Human-AI Collaboration

Enriching communication and collaboration with AI-based tools

A major focus of our current work is to explore how to design effective interfaces for human-AI collaboration. This framing of interaction draws focus away from the idea of computation (as in human-computer interaction), and rather on the idea of computation that can assist and amplify human activity through interactive collaboration. Collaboration, as we understand it, involves multiple turns for interacting, as well as a commitment to help and assist one another.

Coordination and Communication based on Human-Human Interaction

Recently, we have been exploring how our understanding of human-human communication and coordination can be used to smooth human-AI interaction. One design resource for this is in studying and understanding how humans work with other humans, and using ideas here to apply to human-AI communication and coordination.

For instance, we have been interested in studying how we can use proxemics to mediate how we interact with robots (Mahadevan, Sousa, Tang, & Grossman, 2021). We have found that when robots can understand how people are using space to coordinate their interactions with one another, they can more straightforwardly anticipate how human collaborators will work. This smooths collaborative activity between humans and robots.

We applied similar ideas in Stargazer (Li, Sousa, Mahadevan, Wang, Aoyagui, Yu, Yang, Balakrishnan, Tang, & Grossman, 2023), where we designed a robotic camera that could understand both gestures and speech from a howto video tutorial producer. The robotic camera could, based on what the producer was saying, make decisions about how to reposition itself, or to zoom in/out.

Broadly, we expect that if we can design AI that understands proxemic behaviours and language, this will make it easier for humans to interact with AI that understands and has embodiment in our physical world. This is not the only way to design for human-AI collaboration, but one tact that is likely to be successful.

Generative Tools for Authors

We have recently begun exploring how generative tools can be designed to support authors. An important aspect of this is understand what kinds of challenges authors have. In one project, we are exploring the kinds of challenges that fiction authors have in crafting their works (Stark, Tang, Kim, Park, & Wigdor, 2023). After having identified some of these challenges, we have begun the process of designing and studying the impacts of tools that support the creative process.

Publications

  1. Jessi Stark, Anthony Tang, Young-Ho Kim, Joonsuk Park, and Daniel Wigdor. (2023). Can AI Support Fiction Writers Without Writing For Them? . In CHI23 Workshop: In2Writing - The Second Workshop on Intelligent and Interactive Writing Assistants. (Gero, Katy Ilonka and Chang, Minsuk and Huang, Ting-Hao ’Kenneth’ and Chung, John Joon Young and Raheja, Vipul and Wambsganss, Thiemo and Kang, Dongyeop and Lee, Mina, Eds.) (workshop).
  2. Jiannan Li, Mauricio Sousa, Karthik Mahadevan, Bryan Wang, Paula Akemi Aoyagui, Nicole Yu, Angela Yang, Ravin Balakrishnan, Anthony Tang, and Tovi Grossman. (2023). Stargazer: An Interactive Camera Robot for Capturing How-To Videos Based on Subtle Instructor Cues. In CHI 2023: Proceedings of the 2023 SIGCHI Conference on Human Factors in Computing Systems. (conference).
  3. Karthik Mahadevan, Mauricio Sousa, Anthony Tang, and Tovi Grossman. (2021). "Grip-that-there": An Investigation of Explicit and Implicit Task Allocation Techniques for Human-Robot Collaboration. In CHI 2021: Proceedings of the 2021 SIGCHI Conference on Human Factors in Computing Systems. (conference).
    Acceptance: 26.3% - 749/2844. Notes: Best Paper Nominee (top 5% of submissions).