Human – Large Language Model Interaction

The dawn of a new era or the end of it all?

Post your questions to our Panels

The Human - Large Language Model Interaction workshop aims at bringing together researchers and industry experts across all disciplines in the fields of Human-Robot Interaction and Artificial Intelligence for an interactive and interdisciplinary discussion around the enormous opportunities and challenges that emerge from integrating Large Language Models in the interactive, conversational and reasoning abilities of robots.

The workshop aims to provide an important venue to encourage debate around issues concerning the deployment of Large Language Models empowered solutions for human-robot interaction. This will be an opportunity to share and discuss ideas, worries, strategies, insights and findings around the application of Large Language Models in interaction scenarios.

The primary audience of the workshop are researchers in the field of Human-Robot Interaction, Conversational Agents, Natural Language Processing, Artificial Intelligence, and Social Robotics, but we invite everyone with an interest or concern about the use of LLMs for HRI to join.

The workshop will take place on March 11, 2024 at Boulder, Colorado (USA) during the 19th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024)

Format

This will be a half-day workshop.

The authors of the accepted papers will present their work in the format of lightning talks or posters during the workshop.

The accepted papers will be published on the workshop website.

Schedule

The workshop will take place in the conference venue, room UMC 386 in the University Memorial Center in the University of Colorado Boulder

Schedule:

  • 2:00 WS Intro
  • 2:00 - 3:00 Panel “Reliability, Safety and Trustworthiness of LLMs in HRI”
Panelist:      
Kevin Klyman
(Stanford University)		 Cynthia Matuszek
(University of Maryland, Baltimore County)		 Casey Kennington
(Boise State University)		 Joel Fischer
(University of Nottingham)
  • 3:00 - 3:30 Lightning Talks
Title Authors
Safety and Accountability for Large Language Model Use in HRI Christine T. Chang and Bradley Hayes
Large Language Models Enable Automated Formative Feedback in Human-Robot Interaction Tasks Emily Jensen, Sriram Sankaranarayanan and Bradley Hayes
Reference-Guided Robotic Photography Through Natural Language Interaction Oliver Limoyo, Jimmy Li, Dmitriy Rivkin, Jonathan Kelly and Gregory Dudek
Intelligent Robotic Tutoring: Integrating Verbal Input for Personalizing Learning Responses Puming Jiang and Nicole Salomons
Multimodal Human-Autonomous Agents Interaction Using Pre-Trained Language and Visual Foundation Models Linus Nwankwo and Elmar Rueckert
An Overview of Dobby, A Conversational Service Robot Carson Stark, Bohkyung Chun, Casey Charleston, Varsha Ravi, Luis Pabon, Surya Sunkari, Tarun Mohan, Peter Stone and Justin Hart
Prototype of a Care Documentation Support System Using Audio Recordings of Care Actions and Large Language Models Matthias Hirschmanner, Reinhard Grabler, Helena Anna Frijns, Evelyn Mayer-Haas and Markus Vincze
Toward LLM-Powered Robots In Engineering Education C. Estelle Smith, Alemitu Bezabih, Shadi Nourriz, Bo Wo and Gabe Fierro
Leveraging Vision and Language Models for Zero-Shot, Personalization of Household Multi-Object Rearrangement Tasks Benjamin Newman, Pranay Gupta, Kris M. Kitani, Yonatan Bisk, Chris Paxton and Henny Admoni
Profiled with Purpose: LLMs’ Role in User Profiles for Personalized Conversations Manasi Swaminathan, Long-Jing Hsu, Rasika Muralidharan, Weslie Khoo, David J. Crandall and Selma Šabanović
Enhancing Urban Data Analysis through Large Language Models: A Case Study with NYC 311 Service Requests Hedaya Walter, Emily Portalatin-Mendez, Brian Smith, Jennifer Laird and Jorge Ortiz
Perceived Credibility of LLM Responses in Search Contexts Abhinav Choudhry, Kyrie Zhixuan Zhou and Rachel F. Adler
  • 3:30 - 4:00 Coffee Break
  • 4:00 - 5:00 Panel “Capabilities and Limitations of LLMs in HRI”
Panelist:      
Carolina Parada
(Google DeepMind Robotics)		 Bilge Mutlu
(University of Wisconsin–Madison)		 Micol Spitale
(Politecnico di Milano)		 Tony Belpaeme
(Ghent University)
  • 5:00 - 5:50 Grand Debate LLMs, the dawn of a new era or the end of it all?
  • 5:50 - 6:00 Closure