Dobb-E

Dobb·E is a user-friendly open-source tool that helps teach robots how to handle household chores by mimicking human actions.

At its core, Dobb·E tackles the challenges faced by home robots by offering a budget-friendly and ergonomic way to gather demonstrations of household tasks. One of the key components of this framework is a handy tool known as the Stick. This innovative device combines a $25 reacher-grabber stick, some 3D printed parts, and an iPhone to facilitate the learning process.

The Stick plays an important role in collecting valuable data from a dataset called Homes of New York (HoNY). This dataset features around 13 hours of recorded activities across 22 different homes in New York City. It includes a mix of RGB and depth videos, as well as detailed annotations on the gripper's position and how it opens, all crucial for training the robots.

Using the data gathered from these demonstrations, Dobb·E trains a model known as Home Pretrained Representations (HPR). This model is built on the ResNet-34 architecture and utilizes self-supervised learning methods to effectively prepare a robot for tackling new tasks in different environments.

Impressively, Dobb·E has shown it can achieve an average success rate of 81% when completing new tasks, all within just 15 minutes, relying on only five minutes of previously collected demonstration data in an unfamiliar home setting.

If you're interested in diving deeper, Dobb·E makes it easy to access pre-trained models, source code, and comprehensive documentation through GitHub. Plus, there's an open-access paper titled "On Bringing Robots Home" that elaborates on the methods behind Dobb·E and its accomplishments.