Our researchers create state-of-the-art systems to better recognize objects, people, scenes, behaviors and more, with applications in health-care, gaming, tagging systems and more.
Our interdisciplinary work intersects with research in graphics, neuroscience, and computational photography.
First-ever imaging privacy threats from non-imaging components (ambient light sensors) of smart devices. An idea inspired by George Orwell's dystopian novel 1984 -- "Big Brother is watching you" through a two-way TV-like telescreen.
First-ever imaging privacy threats from non-imaging components (ambient light sensors) of smart devices. An idea inspired by George Orwell's dystopian novel 1984 -- "Big Brother is watching you" through a two-way TV-like telescreen.
We develop novel algorithms for neuroimage segmentation, registration, generative modeling, and joint modeling of brain structure with clinical and genetic data.
We aim to create a virtual environment where agents learn to perform human tasks by executing programs. Furthermore, we aim to develop models that can generate such programs from video or text, enabling agents to understand and imitate such activities.
Places is a 10 million image database for scene recognition. It contains images from more than 400 scene categories. Places-CNNs are trained to recognize scene context in human-level accuracy.
We build up a pixel-wise annotated image dataset for scene parsing. Scene parsing network are also proposed to detect and segment visual concepts from any input images.
Our aim is to improve the interpretability of deep neural networks to make it possible to understand their decisions, debug their errors, and make systematic improvements.
Our goal is to build a system that predicts where people are looking in images. Given an image and the location of a head, our approach follows the gaze of the person and identifies the object being looked at.
We aim to understand 3D object structure from a single image. We propose an end-to-end framework which sequentially estimates 2D keypoint heatmaps and 3D object structure, by training it on both real 2D-annotated images and synthetic 3D data and by integrating a 3D-to-2D projection layer.
Our goal is to understand the illumination of an environment. By disentangling the illumination effect from other intrinsic properties (e.g. geometry, texture, color), we can better understand how human perceive the world. It also has several applications such as single image relighting, color editing, etc.
We study the problem of 3D object generation. We propose a novel framework, 3D Generative Adversarial Network (3D-GAN), leveraging recent advances in volumetric convolutional networks and generative adversarial nets.
Our goal is to understand what drivers can perceive when they are not attending to the road ahead and the consequences of their perceptions for safety.
The goal of this project is to develop and test a wearable ultrasonic echolocation aid for people who are blind and visually impaired. We combine concepts from engineering, acoustic physics, and neuroscience to make echolocation accessible as a research tool and mobility aid.
New dataset of “illusory” faces reveals differences between human and algorithmic face detection, links to animal face recognition, and a formula predicting where people most often perceive faces.
DenseAV, developed at MIT, learns to parse and understand the meaning of language just by watching videos of people talking, with potential applications in multimedia search, language learning, and robotics.
“Alchemist” system adjusts the material attributes of specific objects within images to potentially modify video game models to fit different environments, fine-tune VFX, and diversify robotic training.
MIT CSAIL researchers enhance robotic precision with sophisticated tactile sensors in the palm and agile fingers, setting the stage for improvements in human-robot interaction and prosthetic technology
Novel method makes tools like Stable Diffusion and DALL-E-3 faster by simplifying the image-generating process to a single step while maintaining or enhancing image quality.
MIT CSAIL scientists created an algorithm to solve one of the hardest tasks in computer vision: assigning a label to every pixel in the world, without human supervision.
