Upcoming Event: Center for Autonomy Seminar

Game Semantics for Robust POMDPs

Eline Bovy, Ph.D. student, Radboud University, Netherlands

Abstract

Partially observable Markov decision processes (POMDPs) model sequential decision-making problems where the agent has partial information about the state. POMDPs rely on the key assumption that probabilities are precisely known. Robust POMDPs (RPOMDPs) alleviate this concern by defining imprecise probabilities, referred to as uncertainty sets. Existing research on RPOMDPs is limited and primarily focuses on algorithmic solution methods. In this talk, I will present our contribution to the theoretical understanding of RPOMDPs. We show that 1) different assumptions on the uncertainty sets affect optimal policies and their values; 2) RPOMDPs have a partially observable stochastic game (POSG) semantic; and 3) the same RPOMDP with different assumptions leads to semantically different POSGs and, thus, different policies and values. These semantics for RPOMDPs give access to results for POSGs, studied in game theory; concretely, we show the existence of a Nash equilibrium. Finally, we classify the existing RPOMDP literature using our semantics, clarifying under which uncertainty assumptions these existing works operate.

Biography

Eline Bovy is a second-year PhD student at the Radboud University in Nijmegen (the Netherlands) working in the Data-Driven Verification and Learning Under Uncertainty (DEUCE) ERC project supervised by Prof. Nils Jansen. Before starting as a PhD student, she studied Mathematical Foundations of Computer Science at the Radboud University.
Eline's research focuses on extending the theoretical understanding of optimal behavior in realistic stochastic sequential decision-making problems modeled by extensions of Markov Decision Processes.