Home

Recent Announcements

• I will be in Tubingen, Germany to present our paper, "Average-Reward Soft Actor-Critic" at EWRL!

• I will be at the IAIFI Summer Workshop at Harvard to present our work on average-reward RL.

• I will be in Edmonton to present our paper, "Average-Reward Soft Actor-Critic" for the second Reinforcement Learning Conference!

• I will be in Vancouver for ICML 2025 to present a recent paper "Exploration Behavior of Untrained Policies" at the high-dimensional learning dynamics (HiLD) workshop.

• Our paper, "Saturation of the Cramér-Rao Bound for the Atomic Resonance Frequency with Phased Array of Hyperbolic Secant Pulses" is now available on arXiv! In this work, with Tharon Holdsworth and Professor Girish S. Agarwal, we exploited properties of sech pulses to develop a pulse sequence capable of saturating the Cramér-Rao bound. This allows for the most precise measurements of atomic resonance frequencies.

• Our paper, "Average-reward Soft Actor-Critic" has been accepted at the second annual Reinforcement Learning Conference!

• Traveled to ICLR (Singapore) and Inria (Lille, FR) to present our work on RL for drug dosing and deep average-reward RL algorithms, respectively.

• I was invited to give a guest lecture at UMass Boston's new AI4ALL class. I gave an introduction to RL including discussions of exploration-exploitation, reward hacking, and policy improvement. The students developed custom environments related to real-world problems of their interest, and deployed RL algorithms to understand how agents learn and act. I had a great time teaching and the students did a tremendous job! I think such courses are important for increasing diversity in the field while giving students more tools for their future careers.

• I was invited to give a talk at Cohere 4 AI -- tune in to hear more about our reward shaping research! Here's the YouTube Link.

• The American Physical Society (APS) Group of Data Science (GDS) has awarded me the IMPACT award, given for "noteworthy progress in their academic careers at the intersection of Physics and Data Science". I am grateful to GDS for recognizing and supporting my work! They have played a pivotal role in helping me attend conferences during my PhD.

• Our paper was accepted at ICLR 2025! Very excited to travel to Singapore and present this work with Josiah, let me know if you'll be there!

• I am a finalist for the Student Speaker Award, awarded by the American Physical Society (APS) Group of Statistical and Nonlinear Physics (GSNP) for the "best contributed talk at the March meeting by a graduate student in the area of Statistical and Nonlinear Physics."

• 5 papers accepted to AAAI-2025 and workshops! "Bootstrapped Reward Shaping", "EVAL: EigenVector-based Average-reward Learning", "Average Reward Reinforcement Learning with Entropy-Regularization", "Inferring Transition Dynamics from Value Functions", and "CASH: Cache Alignment with Specified Horizons".

• I will be a reviewer for ICML 2025 📄✍️

• With Josiah Kratz, I presented our work at NeurIPS 2024 in Vancouver, Canada! (poster)

• I presented our work on average-reward RL at MLQT ("Machine Learning for Quantum Technologies") in Erlangen, Germany! (poster) I was able to meet some leaders in the field of Quantum + ML, and saw a lot of exciting new ideas.

• I was accepted for an internship at Sony AI for the summer of 2025! I will be working on deep RL with average-reward objectives for video games. Very excited!

• Reviewing papers for ICLR and AISTATS... 📄✍️

• The work with Josiah based on our meeting at IAIFI has been accepted as a workshop paper in ML4PS @ NeurIPS 2024! "Reinforcement Learning for Optimal Control of Adaptive Cell Populations".

• I presented our work at RLC 2024 in Amherst, MA. The invited talks were amazing! I learned a lot about the current state of the art in reinforcement learning and met many great people. (slides, poster)

• I have been accepted to attend the IAIFI Summer School and Summer Workshop. I worked on a project with Josiah Kratz based on reinforcement learning in biological control. (slides)

• I am a reviewer for NeurIPS 2024 -- this is my first time as a conference reviewer! 📄✍️

• Our paper has been accepted for publication at the (inaugural) Reinforcement Learning Conference!

• I was nominated and inducted with lifetime membership to CSU's chapter of Sigma Pi Sigma.

• I was awarded the Beacon Graduate Student Leadership Award. Thank you to all of those who nominated me!

• I attended the annual APS March Meeting to share our work on deep average-reward RL algorithms developed from our work in PRR. You can find the slides for that talk here.

• I was awarded the GDS IMPACT Award for my work at the intersection of physics and reinforcement learning.

• I've started a deep learning class, free for anyone to attend (pizza and drinks included!), with minimal prerequisites. See here and here for more details. I'm excited to share my enthusiasm about the deep learning (and particularly deep RL) revolution.

• Tharon Holdsworth and I presented a project for Akira Sone's quantum control course, where we study properties of the sech pulse and its use in quantum control. We will be releasing results on arXiv shortly, and code can be found here.

• We updated the arXiv with a newer version of our work on double-sided bounds for the value function in reinforcement learning: arxiv link.

• I attended the IAIFI Summer School and Summer Workshop. I had an amazing time! I met and learned from many brilliant people working at the intersection of fundamental physics and AI - in both theoretical and experimental settings.

• I presented our work on compositionality in reinforcement learning at the 2023 Conference on Uncertainty in Artificial Intelligence (UAI) (poster). My co-author, Argenis Arriojas, received a Poster Spotlight award for our other work: "Bayesian Inference Approach for Entropy Regularized Reinforcement Learning with Stochastic Dynamics".

• I have been awarded the College of Science and Mathematics (CSM) Dean's Doctoral Research Fellowship for Fall 2023. This fellowship will continue to support my research at the interface of physics and reinforcement learning. Thank you to CSM!

• I presented my work at the 7th Annual CSM Student Success Showcase. See my poster here (pdf here).

• I presented my work and future research goals for reinforcement learning at the "Physics Department Retreat" at UMass Boston.

• Our paper titled "Entropy regularized reinforcement learning using large deviation theory" was accepted for publication in Physical Review Research. (A more detailed post is pending.)

• Our papers (1) "Bounding the Optimal Value Function in Compositional Reinforcement Learning" and (2) "Bayesian Inference Approach for Entropy Regularized Reinforcement Learning with Stochastic Dynamics" have both been accepted in UAI 2023.

• I taught a 5 week mini-course titled "Deep Learning and Physics" for all students and faculty at UMass Boston. I am planning to offer a similar, extended course in the future. See the flyer here

• We have added two new papers to the arXiv: Bounding the Optimal Value Function in Compositional Reinforcement Learning and Compositionality and Bounds for Optimal Value Functions in Reinforcement Learning exploring the use of bounds on the Q-function in deep reinforcement learning. (A more detailed post is pending.)

• I attended the 2023 APS March Meeting in Las Vegas, NV. I presented a talk on my research in the statistical mechanics approach to reinforcement learning. (A more detailed post is pending.)

• I had a great time presenting at my first AI conference (AAAI-2023) in Washington, D.C. I met many great people and learned a lot about what the broader community is interested in. (Here is a more detailed post.)

• I have been awarded a College of Science and Mathematics (CSM) Dean's Doctoral Research Fellowship for Spring 2023. This fellowship will support my research in the statistical mechanics approach to reinforcement learning. I am very grateful to the CSM for this award, and I look forward to continuing my research in the coming year.

• Our group's first paper on entropy-regularized reinforcement learning titled "Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning" was accepted at AAAI-2023.

• I gave a talk for my PhD qualifying exam. I passed and was thus promoted to PhD candidacy. Here are the slides.