tl;dr: I had a great time attending AAAI-2023 in Washington, D.C. I met many great people and learned a lot about the broader AI community. I attended all of the “RL Theory” and “RL Algorithms” sessions to find out more about the RL community’s interests. Please find the full paper (arXiv version) at this link: Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning. The poster is here: Poster (pdf).

Me presenting at AAAI-2023 (photo credits: Weiye Zhao)


The Details:

Many folks working on RL theory are interested in the bandit setting which was a bit surprising to me. On the algorithms side, there were a few other groups working on reward-shaping and methods for transfer learning and hierarchical composition.

At the conference, I met:

  • Ron Parr whose recent paper on policy caches was an inspiration for further investigating bounds on the optimal value function in compositional RL (see here for a relevant pre-print). I also really enjoyed his work on radial-basis value functions.

  • Matheus Centa and several others from his group at Inria. We had a great time discussing ideas and I’m looking forward to their future work. The paper he presented at AAAI can be found here. It’s a great read and the fundamental ideas are quite related to my own work.

  • Sam Lobel who, with collaborators, (a) found a very fundamental problem with policy networks in deep RL, and (b) proposed a promising solution to this problem. I highly recommend reading the paper, which can be found here.

During the poster session, I also met:

  • Sreehari Rammohan who is working with Sam Lobel (above) at Brown University. I’m looking forward to seeing his future work and collaborating with him in the future.
  • Weiye Zhao (Carnegie Mellon University) who I can thank for the photo above, taken during the interactive poster session.
  • and many others! (Feel free to reach out if we met so that I can add you to this list!)