SSP Forum: Jeannette Bohg on Language and Robotics

Monday, May 18, 2020
Zoom Meeting - Online
Photo of Jeannette Bohg

The
Symbolic Systems Forum
presents

Leveraging Language in Learning Robot Manipulation Skills
Jeannette Bohg
Computer Science Department

Monday, May 18, 2020
12:30-1:20 pm
Join from PC, Mac, Linux, iOS or Android: https://stanford.zoom.us/j/92146755757
    Meeting ID: 921 4675 5757
    SIP: 92146755757 [at] zoomcrc.com (92146755757[at]zoomcrc[dot]com)

ABSTRACT:

Humans have gradually developed language, mastered complex motor skills, created and utilized sophisticated tools. The act of conceptualization is fundamental to these abilities because it allows humans to mentally represent, summarize and abstract diverse knowledge and skills. By means of abstraction, concepts that we learn from a limited number of examples can be extended to a potentially infinite set of new and unanticipated situations. My long-term goal is to endow robots with this generalization ability. In this talk, I will present work that gives robots the ability to acquire a variety of manipulation concepts that act as mental representations of verbs in a natural language instruction. We propose to use learning from human demonstrations of manipulation actions as recorded in large-scale video data sets that are annotated with natural language instructions. Specifically, we propose to use a video classifier that scores how well the robot imitates the human actions. This approach alleviates the need for hand-designing rewards and for time-consuming processes such as teleoperation or kinesthetic teaching. In extensive simulation experiments, we show that the policy learned in the proposed way can perform a large percentage of the 78 different manipulation tasks on which it was trained. The tasks are of greater variety and complexity than previously considered collections of robotic manipulation tasks. We show that the policy generalizes over variations of the environment. We also show examples of successful generalization over novel but similar instructions.