Publication Details

Monica Nicolescu, Natalie Arnold, Janelle Blankenburg, David Feil-Seifer, Santosh Balajee Banisetty, Mircea Nicolescu, Andrew Palmer, and Thor Monteverde. "Learning of Complex-Structured Tasks from Verbal Instruction." In International Conference on Humanoid Robots, Toronto, Canada, Oct 2019. ( pdf )


  • Keywords:
  • task allocation

Abstract

This paper presents a novel approach to robot task learning from language-based instructions, which focuses on increasing the complexity of task representations that can be taught through verbal instruction. The major proposed contribution is the development of a framework for directly mapping a complex verbal instruction to an executable task representation, from a single training experience. The method can handle the following types of complexities: 1) instructions that use conjunctions to convey complex execution constraints (such as alternative paths of execution, sequential or nonordering constraints, as well as hierarchical representations) and 2) instructions that use prepositions and multiple adjectives to specify action/object parameters relevant for the task. Specific algorithms have been developed for handling conjunctions, adjectives and prepositions as well as for translating the parsed instructions into parameterized executable task representations. The paper describes validation experiments with a PR2 humanoid robot learning new tasks from verbal instruction, as well as an additional range of utterances that can be parsed into executable controllers by the proposed system.

Author Details

Name: Monica Nicolescu
Status: Active

Name: Natalie Arnold
Status: Inactive

Name: Janelle Blankenburg
email: jjblankenburg@nevada.unr.edu
Website: jblankenburg.github.io
Status: Active

Name: David Feil-Seifer
email: dave@cse.unr.edu
Website: http://cse.unr.edu/~dave
Phone: (775) 784-6469
Status: Active

Name: Santosh Balajee Banisetty
email: santoshbanisetty@nevada.unr.edu
Website: http://www.santoshbanisetty.com/
Status: Active

Name: Mircea Nicolescu
Status: Inactive

Name: Andrew Palmer
Status: Active

Name: Thor Monteverde
Status: Inactive

BibTex Reference

@inproceedings{nicolescu2019learning,
  title={Learning of Complex-Structured Tasks from Verbal Instruction},
  author={Monica Nicolescu and Natalie Arnold and Janelle Blankenburg and David Feil-Seifer and Santosh Balajee Banisetty and Mircea Nicolescu and Andrew Palmer and Thor Monteverde},
  year={2019},
  month={October},
  address={Toronto, Canada},
  booktitle={International Conference on Humanoid Robots},
}

HTML Reference

<span class="authors">Monica Nicolescu, Natalie Arnold, Janelle Blankenburg, David Feil-Seifer, Santosh Balajee Banisetty, Mircea Nicolescu, Andrew Palmer, and Thor Monteverde</span>. <span class="title">"Learning of Complex-Structured Tasks from Verbal Instruction." </span> In <span class="booktitle">International Conference on Humanoid Robots</span>, <span class="address">Toronto, Canada</span>, <span class="month">Oct</span> <span class="year">2019</span>. <span class="ending"></span>

Support

REU Site: Collaborative Human-Robot Interaction, National Science Foundation PI: David Feil-Seifer, co-PI: Shamik Sengupta, Amount: $360,000, Feb. 1, 2018 - Jan. 31, 2021

Designing Collaborator Robots for Highly-Dynamic Multi-Human, Multi-Robot Teams, Office of Naval Research (ONR) PI: Monica Nicolescu, co-PI: David Feil-Seifer, Amount: $656,511, April 1, 2016 - March 31, 2019

DURIP: Humanoid Platforms for Human-Robot Collaboration, Office of Naval Research (ONR) PI: Monica Nicolescu, co-PI: David Feil-Seifer, Mircea Nicolescu, Amount: $312,000, May 1, 2014 - June 30, 2015