Designing Collaborator Robots for Highly-Dynamic Multi-Human, Multi-Robot Teams

Abstract: The goal of this proposal is to establish at the University of Nevada, Reno a robotic infrastructure that would enable groundbreaking research in a wide range of topics related to mobile articulated robots. The proposed system consists of the Baxter humanoid robot and the PR2 robotic platform, whose extensive sensory, motor and computational capabilities open up the possibility of addressing research questions that could not have even been formulated in the context of the current publicly available robotic systems.

Details

  • Organization: Office of Naval Research (ONR)
  • Award #: N00014-16-1-2312
  • Amount: $656,511
  • Date: April 1, 2016 - March 31, 2019
  • PI: Dr. Monica Nicolescu
  • Co-PI: Dr. David Feil-Seifer

Supported Publications

  • Banisetty, S., Forer, S., Yliniemi, L., Nicolescu, M., & Feil-Seifer, D. Socially-Aware Navigation: A Non-linear Multi-Objective Optimization Approach. In ACM Transactions on Interactive Intelligent Systems, 11(15), Jul 2021. ( details ) ( .pdf )
  • Blankenburg, J., Zagainova, M., Simmons, M. S., Talavera, G., Nicolescu, M., & Feil-Seifer, D. Human-Robot Collaboration and Dialogue for Fault Recovery on Hierarchical Tasks. In International Conference on Social Robotics (ICSR), CO, Oct 2020. ( details ) ( .pdf )
  • Chowdhury, M., Paul, K. S., Nicolescu, M., Nicolescu, M., Feil-Seifer, D., & Dascalu, S. Computation of Suitable Grasp Pose for Usage of Objects Based on Predefined Training and Real Time Pose Estimation. In International Conference on Autonomic and Autonomous Systems, Lisbon, Portugal, Sep 2020. ( details ) ( .pdf )
  • Paul, K. S., Chowdhury, M., Nicolescu, M., Nicolescu, M., & Feil-Seifer, D. Object Detection and Pose Estimation from RGB and Depth Data for Real-time, Adaptive Robotic Grasping. In International Conference on Image Processing, Computer Vision & Pattern Recognition, Las Vegas, NV, Jul 2020. ( details ) ( .pdf )
  • Banisetty, S., Multi-Context Socially-Aware Navigation Using Non-Linear Optimization. University of Nevada Reno, May 2020. ( details ) ( .pdf )
  • Anima, B., Blankenburg, J., Zagainova, M., Hoseini, S., Chowdhury, M., Feil-Seifer, D., Nicolescu, M., & Nicolescu, M. Collaborative Human-Robot Hierarchical Task Execution with an Activation Spreading Architecture. In International Conference on Social Robotics, page 301-310, Madrid, Spain, Nov 2019. ( details ) ( .pdf )
  • Nicolescu, M., Arnold, N., Blankenburg, J., Feil-Seifer, D., Banisetty, S., Nicolescu, M., Palmer, A., & Monteverde, T. Learning of Complex-Structured Tasks from Verbal Instruction. In International Conference on Humanoid Robots, Toronto, Canada, Oct 2019. ( details ) ( .pdf )
  • Hoseini, S., Blankenburg, J., Nicolescu, M., Nicolescu, M., & Feil-Seifer, D. An Active Robotic Vision System with a Pair of Moving and Stationary Cameras. In Proceedings of the International Symposium on Visual Computing (ISVC), Lake Tahoe, NV, Oct 2019. ( details ) ( .pdf )
  • Hoseini, S., Blankenburg, J., Nicolescu, M., Nicolescu, M., & Feil-Seifer, D. Active Eye-in-Hand Data Management to Improve the Robotic Object Detection Performance. In Computers, 8(4), Sep 2019. ( details ) ( .pdf )
  • Banisetty, S. & Feil-Seifer, D. Towards a Unified Planner For Socially-Aware Navigation. In AAAI Fall Symposium Series: AI-HRI Artificial Intelligence for Human-Robot Interaction, Nov 2018. ( details ) ( .pdf )
  • Forer, S., Banisetty, S., Yliniemi, L., Nicolescu, M., & Feil-Seifer, D. Socially-Aware Navigation Using Non-Linear Multi-Objective Optimization. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain, Oct 2018. ( details ) ( .pdf )
  • Blankenburg, J., Banisetty, S., Hoseini, S., Fraser, L., Feil-Seifer, D., Nicolescu, M., & Nicolescu, M. A Distributed Control Architecture for Collaborative Multi-Robot Task Allocation. In International Conference on Humanoid Robots, Birmingham, UK, Nov 2017. ( details ) ( .pdf )
  • Sebastian, M., Banisetty, S., & Feil-Seifer, D. Socially-Aware Navigation Planner Using Models of Human-Human Interaction. In International Symposium on Robot and Human Interactive Communication (RO-MAN), page 405-410, Lisbon, Portugal, Aug 2017. ( details ) ( .pdf )
  • Miller, B. & Feil-Seifer, D. Embodiment, Situatedness and Morphology for Humanoid Interaction, page 1-23. Springer Netherlands, Mar 2017. ( details ) ( .pdf )
  • Banisetty, S., Sebastian, M., & Feil-Seifer, D. Socially-Aware Navigation: Action Discrimination to Select Appropriate Behavior. In 2016 AAAI Fall Symposium Series: AI-HRI Artificial Intelligence for Human-Robot Interaction, Arlington, VA, Nov 2016. ( details ) ( .pdf )
  • Fraser, L., Rekabdar, B., Nicolescu, M., Nicolescu, M., Feil-Seifer, D., & Bebis, G. A compact task representation for hierarchical robot control. In International Conference on Humanoid Robots, page 697-704, Cancun, Mexico, IEEE. Nov 2016. IEEE. ( details ) ( .pdf )
  • Banisetty, S., Sebastian, M., & Feil-Seifer, D. Socially-Aware Navigation: Action Discrimination To Select Appropriate Behavior. Poster Paper in Annual Graduate Poster Symposium (GSA), Nov 2016. ( details ) ( .pdf )
  • Fraser, L., Rekabdar, B., Nicolescu, M., Nicolescu, M., & Feil-Seifer, D. A Hierarchical Control Architecture for Robust and Adaptive Collaborative Robot Task Execution. In Robotics: Science \& Systems: Workshop on Planning for Human-Robot Interaction: Shared Autonomy and , Cambridge, Massachusetts, Jun 2016. ( details ) ( .pdf )

Supported Projects

Distributed Control Architecture for Collaborative Multi-Robot Task Allocation March 1, 2016 - Present

thumbnail for Distributed Control Architecture for Collaborative Multi-Robot Task Allocation Real-world tasks are not only a series of sequential steps, but typically exhibit a combination of multiple types of constraints. These tasks pose significant challenges, as enumerating all the possible ways in which the task can be performed can lead to large representations and it is difficult to keep track of the task constraints during execution. Previously we developed an architecture that provides a compact encoding of such tasks and validated it in a single robot domain. We recently extended this architecture to address the problem of representing and executing tasks in a collaborative multi-robot setting. Firstly, the architecture allows for on-line, dynamic allocation of robots to various steps of the task. Secondly, our architecture ensures that the collaborative robot system will obey all of the task constraints. Thirdly, the proposed architecture allows for opportunistic and flexible task execution given different environmental conditions. We demonstrated the performance of our architecture on a team of two humanoid robots (a Baxter robot and a PR2 robot) performing hierarchical tasks. Further extensions of this architecture are currently being explored. Due to the adaptability of our multi-robot control architecture, this work has become the foundation for several ongoing extensions. The first extension is to modify the architecture to allow for a human to collaborate with the multi-robot team in order to complete the joint task. To allow the human and robots to communicate, this work augments the control architecture with a third robot “brain” which utilizes intent recognition to determine which objects the human is interacting with and updates the state of the task held by the other robots accordingly. Secondly, this architecture is being adapted to allow a human to train a robot to complete a task through natural speech commands. The multi-robot team is then able to complete the newly trained task. The third extension focuses on learning generalized task sequences utilizing the types of hierarchical constraints defined by this control architecture. The last extension to this work is to modify the architecture to allow heterogenous grasp affordances for each item in the task. These affordances allow the robots to incorporate how well they can grasp a specific object based on their gripper and arm mechanics into the task allocation architecture. This allows robots to be assigned tasks which they are better suited for. This work is currently being completed and the aim is to submit this work to DARS 2018. Each of these extensions emphasize the generalizability of this control architecture and together illustrate the vast applications in which further extensions of this architecture can be adapted to.