Past Event: CSEM Student Forum

1. Hidden Markov Models for Protein Fluorosequencing and 2. Reward Machines for Cooperative Multi-Agent Reinforcement Learning

1. Matt Smith and 2. Cyrus Neary, CSEM PhD Program, Oden Institute, UT Austin

Abstract

Speaker 1: Matt Smith Title: Hidden Markov Models for Protein Fluorosequencing Abstract: A research group in Dr. Edward Marcotte's lab is working on a new technology called protein fluorosequencing that depends partly on the fast and accurate identification of small strings of amino acids based on time series data. In this talk, we develop a Hidden Markov Model (HMM) to perform this classification and compare it with other methods. We also explore the possibility of combining the HMM with faster methods to increase speed while retaining accuracy for large scale problems. Bio: Matt Smith is a third year student in the CSEM program, advised by Dr. Edward Marcotte. His research is in the computational aspects of protein fluorosequencing. Speaker 2: Cyrus Neary Title: Reward Machines for Cooperative Multi-Agent Reinforcement Learning Abstract: In cooperative multi-agent reinforcement learning, a collection of agents learns to interact in a shared environment to achieve a common goal. We propose the use of reward machines (RM) --- Mealy machines used as structured representations of reward functions --- to encode the team's task. The proposed novel interpretation of RMs in the multi-agent setting explicitly encodes required teammate interdependencies and independencies, allowing the team-level task to be decomposed into sub-tasks for individual agents. We define such a notion of RM decomposition and present algorithmically verifiable conditions guaranteeing that distributed completion of the sub-tasks leads to team behavior accomplishing the original task. Experimental results in three discrete settings exemplify the effectiveness of the proposed RM decomposition approach, which converges to a successful team policy two orders of magnitude faster than a centralized learner and significantly outperforms hierarchical and independent q-learning approaches. Bio: Cyrus Neary is a third-year CSEM PhD student. He works with Dr. Ufuk Topcu in the Autonomous Systems Group. His research currently focuses on structured task representations for reinforcement learning. (The CSEM Student Forum is a seminar series given by current CSEM graduate students to their peers. The aim of the forum is to expose students to each other's research, encourage collaboration, and provide opportunities to practice presentation skills. First- and second-year CSEM students receive seminar credit for attending.)