Researchers at Facebook AI have developed a home robot that will move through an unmapped environment to seek a sound-emitting object, like a phone ringing. ‘SoundSpaces,’ the primary audio-visual platform for embodied AI, can accomplish a task like checking to ascertain whether you locked the front entrance or retrieving a telephone that’s ringing in an upstairs bedroom.
AI assistants of the future must learn to plan their route, navigate effectively, look around their physical environment, listen to what’s happening around them, and build memories of the 3D space. These smarter assistants will require new advances in embodied AI, which seeks to teach machines to understand and interact with the physical world’s complexities as people do.
SoundSpaces
Leveraging SoundSpaces, Facebook introduced a replacement task for embodied AI: AudioGoal. Moreover, Facebook AI said in a statement that according to their knowledge, it is often the primary plan to train deep reinforcement learning agents that can both see and hear, to map novel environments and localize targets that emit sounds. With this approach, the researchers achieved faster training and higher navigation accuracy than with single modality counterparts. ‘SoundSpaces’ provides a new audio sensor, making it possible to insert high-fidelity, realistic simulations of any sound source in an array of real-world scanned environments.
Unlike traditional navigation systems that tackle point-goal navigation, the house robot doesn’t require a pointer to the goal location. This means an agent can now influence “go find the ringing phone” instead of “go to the phone that’s 25 feet southwest of your current position.” Therefore, it can now find the desired position on its own using multi-modal sensing.
To build ‘SoundSpaces,’ Facebook used a state-of-the-art algorithm for room acoustics modeling and a “bidirectional path tracing algorithm” to model sound reflections within the room geometry. “With ‘SoundSpaces,’ researchers can train an agent to identify and move toward a sound source even if it’s behind a couch, for example, or to respond to sounds it has never even heard before”, said Facebook AI.
