Hello! My name is Madhur Panwar, and I am a Research Fellow at Microsoft Research India (MSRI) under the mentorship of Dr. Navin Goyal. My research centers on understanding and improving the capabilities of language models. I aim to understand the origins of their emergent abilities, generalization, and inductive biases, tracing these elements to training dynamics, pre-training data, objectives, and architectural designs.

Before joining MSRI, I worked as a Software Development Engineer at Adobe, developing Adobe Journey Optimizer — a comprehensive tool designed to empower marketers to orchestrate seamless customer interactions from start to finish. I graduated from BITS Pilani, India, in 2021 with a B.E. in Computer Science and an M.Sc. in Mathematics. I pursued my Bachelor's thesis on speaker diarization under the co-supervision of Prof. Chng Eng Siong (NTU Singapore) and Prof. Poonam Goyal (BITS Pilani).

In 2020, I interned at the Media and Data Science Research Lab at Adobe, where I collaborated with Balaji Krishnamurthy and Milan Aggarwal to develop topic models. I also had an enriching summer experience in 2019 as a Mitacs Globalink Research Intern at the University of Victoria, Canada. During this time, I focused on developing open-source software that formulates and solves energy optimization problems for buildings, guided by Prof. Ralph Evins.

For further information about my academic and professional journey, please consult my CV. Should you have any inquiries about my research, do not hesitate to contact me via email.

Publications

InversionView: A General-Purpose Method for Reading Information from Neural Activations
Xinting Huang, Madhur Panwar, Navin Goyal, Michael Hahn
Preprint
pdf abstract

Learning Syntax Without Planting Trees: Understanding When and Why Transformers Generalize Hierarchically
Kabir Ahuja, Vidhisha Balachandran, Madhur Panwar, Tianxing He, Noah A. Smith, Navin Goyal, Yulia Tsvetkov
Preprint
pdf abstract

In-Context Learning through the Bayesian Prism
Madhur Panwar*, Kabir Ahuja*, Navin Goyal
ICLR 2024
pdf abstract

TAN-NTM: Topic Attention Networks for Neural Topic Modeling
Madhur Panwar*, Shashank Shailabh*, Milan Aggarwal*, Balaji Krishnamurthy
ACL 2021 (Oral)
pdf abstract

In-Context Learning and Bayesian Inference
Madhur Panwar*, Kabir Ahuja*, Navin Goyal
R0-FoMo Workshop at NeurIPS 2023
pdf abstract

Surprising Deviations from Bayesian View in In-Context Learning
Madhur Panwar, Kabir Ahuja, Navin Goyal
ICBINB Workshop at NeurIPS 2023
pdf abstract

Transformers Can Learn To Solve Linear-Inverse Problems In-Context
Kabir Ahuja*, Madhur Panwar*, Navin Goyal
Deep Inverse Workshop at NeurIPS 2023
pdf abstract

[Re] AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
Anirudh Buvanesh*, Madhur Panwar*
NeurIPS 2022 Journal Track (Spotlight)  |  ReScience C Journal, 2022
pdf

Offline Speaker Diarization of Single-Channel Audio
Madhur Panwar
Bachelor's Thesis, 2021
pdf abstract