If AI-powered robots are ever going to help us out around the house, they’re going to need a lot of experience navigating human environments. Simulators, virtual worlds that look and behave just like real life are the best place for them to learn, and Facebook has created one of the most advanced such systems yet.
Teaching a robot to navigate a realistic world and accomplish simple tasks is a process that takes a considerable amount of time, so doing it in a physical space with an actual robot is impractical. It might take hundreds of hours, even years of real time, to learn over many repetitions how best to get from one place to another, or how to grip and pull a drawer.
Instead, the robot’s AI can be placed in a virtual environment that approximates the real one, and the basics can be hashed out as fast as the computer can run the calculations that govern that 3D world. That means you can achieve hundreds or thousands of hours of training in just a few minutes of intense computing time.
Habitat is not itself a virtual world itself, but rather a platform on which such simulated environments can run. It is compatible with several existing systems and environments (SUNCG, MatterPort3D, Gibson and others), and is optimized for efficiency so researchers can run it at hundreds of times real world speeds.
But Facebook also wanted to advance the state of the art in virtual worlds, and so created Replica, a database for Habitat that includes a number of photorealistic rooms organized into a whole house: a kitchen, bathroom, doors, a living room with couches, and everything. It was created by Facebook’s Reality Labs, and is the result of painstaking photography and depth mapping of real environments.