Beyond Reward Engineering: Teaching, Learning, and Optimizing Complex Objectives

Organizers: Dylan Hadfield-Menell, Anca Dragan, Marco Pavone and Pieter Abbeel


Determining a good reward function is a challenge that arises in almost any application of robotics. A good reward function needs to be general enough that it prescribes the correct action across a wide range of environments and users. At the same time, it needs to be amenable to planning or optimization so that the robot in question actually takes actions that optimize the desired objective. Furthermore, we often design robots to help end-users, in which case the true reward function is to do whatever the user wants and some form of objective learning or inference is necessary. This workshop will bring together experts in human-robot interaction, inverse reinforcement learning, machine teaching, assistive robotics, and reinforcement learning from the robotics community to answer the following questions:

  1. How can we efficiently teach or specify objectives for robots?
  2. What are scalable and efficient inference methods for inferring objectives from observation?
  3. What are the most reliable and efficient methods to elicit objectives from users?