Agents Learning With and From Other Agents
Ph.D. dissertation, Department of Computer Science, The University of Texas at Austin.
Advisor: Peter Stone
Committee: Peter Stone, Risto Miikkulainen, Elias Stengel-Eskin, Natasha Jaques
Defended: September 18, 2026
Abstract
Reinforcement learning algorithms enable AI agents to learn complex behaviors in both single and multi-agent scenarios. However, significant challenges remain in how agents can efficiently leverage knowledge produced by others and coordinate with unknown, diverse teammates, particularly under realistic constraints such as limited demonstration data, variable team compositions, and open-ended, non-stationary partner distributions. This dissertation answers the following question: “How can reinforcement learning agents that operate in a cooperative context with other agents learn from and with those agents to efficiently solve shared tasks?” This dissertation approaches the question along two complementary dimensions: (1) learning from other agents by leveraging demonstration data for improved sample efficiency, and (2) learning to coordinate with potentially unknown teammates in cooperative settings. Addressing the first dimension, it introduces an algorithm that enables RL agents to learn from suboptimal and limited demonstration data. Addressing the second dimension, it introduces two novel problem formulations for ad hoc teamwork (AHT): N-agent ad hoc teamwork (NAHT), which generalizes AHT to scenarios involving a varying number of AHT agents and teammates, and a formulation of AHT as an open-ended learning problem. To provide a unified answer to the primary question, this dissertation presents three contributions that address both angles of the thesis question. First, it introduces a decentralized multi-agent reinforcement learning algorithm that relies upon team demonstrations to encourage convergence and improve sample efficiency. Second, it provides an algorithm for NAHT that leverages data generated by unknown teammates to improve sample efficiency of learning. Third, it introduces the first hardware-accelerated benchmark for AHT, with evaluation grounded in human behavioral data. This dissertation advances the field of multi-agent reinforcement learning by bridging the study of learning from demonstrations and coordination with unknown teammates, with the broader goal of developing AI agents capable of flexible and collaborative problem-solving alongside humans and other agents in the real world.
Defense Talk
Publications Arising from the Dissertation
- Ch. 3AAMAS 2023
- Ch. 4AAAI 2023
- Ch. 5NeurIPS 2024
- Ch. 6NeurIPS 2026
- Ch. 7ICLR 2026 MALGAI Workshop