Sushil Khyalia

I am an ML Researcher on the LLM team at Sarvam AI, where I work on post-training. I was a core contributor to Sarvam 30B and 105B, India's first open-source reasoning MoE models, working mostly on reinforcement learning. I also worked on the SFT and RLVR pipelines for Sarvam-M.

I completed my Master's in Machine Learning at Carnegie Mellon University in December 2024, where I worked with Professors Laszlo Jeni and Louis-Philippe Morency on affect recognition in video and on benchmarking multimodal video understanding models.

Before that, I spent two and a half years at Samsung Research Headquarters in South Korea. On the Language and Voice Team, under Dr. Chanwoo Kim, I worked on a signal-propagation theory of transformers and the initialization scheme it implies, on lexicon-free text-to-speech, and on open-domain question answering. Earlier, on the Data Analysis Team, I worked on time-series imputation and session-based recommendation.

I received my undergraduate degree from IIT Bombay, where I was advised by Prof. Ganesh Ramakrishnan and Prof. Preethi Jyothi on meta-learning strategies for knowledge transfer in natural language understanding. I also worked with Prof. Shivaram Kalyanakrishnan on PAC mode estimation and on complexity upper bounds for policy iteration on deterministic MDPs.

Email  /  CV  /  Scholar  /  LinkedIn  /  Github

Research

I am interested in post-training for large language models — reinforcement learning, verifiable environments, and reward design — and in what makes reasoning and agentic behaviour emerge and stay stable at scale. I am also drawn to the theory underneath it: signal propagation and training dynamics in deep networks, and the complexity of sequential decision-making.

A novel initialization and output scaling scheme that enables stable training of transformer networks with up to 1000 layers.

Upper bounds on Policy Iteration algorithms' running time for deterministic MDPs derived using graph-theoretic analysis.

A 2,556-question audio-visual benchmark whose questions cannot be answered from either modality alone, forcing genuine cross-modal reasoning.

A novel stopping rule based on PPR martingale confidence sequences for PAC mode estimation, offering asymptotic optimality and improved sample efficiency for categorical distributions.

A meta-learning framework that enhances multi-task and multilingual models by optimizing knowledge transfer across diverse tasks and languages.

Data-driven grapheme-to-phoneme representations using self-supervised learning eliminating the need for lexicons in text-to-speech systems while achieving competitive performance.

A GAN-based self-attention network for imputing missing values in multivariate time series data, enhancing accuracy through bidirectional RNNs and novel attention mechanisms.

Improving session-based recommendation by combining contrastive learning with global context-aware augmentations, leading to more accurate and robust predictions.

I used Jon Barron's website template source code as a base.