Sushil Khyalia

I am a second-year Master's student in Machine Learning at Carnegie Mellon University, where I focus on affect recognition in videos by integrating vision, audio, and text data. I work under the guidance of Professors Laszlo Jeni and Louis-Philippe Morency. In addition, I am involved in benchmarking video understanding models.

I received my undergraduate degree from IIT Bombay, where I was advised by Prof. Ganesh Ramakrishnan and Prof. Preethi Jyothi on exploring meta-learning strategies for effective knowledge transfer in natural language understanding tasks. I also had the opportunity to work with Prof. Shivaram Kalyanakrishnan on PAC mode estimation and improving the upper bound complexity for policy iteration on 2-action MDPs.

Previously, I spent an enriching year as part of the Language and Voice Team at Samsung Research Headquarters in South Korea, where I had the privilege of working under the leadership of Dr. Chanwoo Kim. My work focused on open-domain question-answering, lexicon-free text-to-speech, and large language modeling. Prior to that, I contributed to the Data Analysis Team, where I worked on data imputation and session-based recommendation systems.

Email / CV / Scholar / LinkedIn / Github

Research

My research interests include natural language processing, synthetic data generation, data compression, multimodal data integration, and enhancing robustness for low-resource languages in machine learning.

	Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models Akhil Kedia^⋆, Mohd Abbas Zaidi^⋆, Sushil Khyalia^⋆, Jungho Jung, Harshith Goka, Haejun Lee ICML, 2024 code / pdf / poster A novel initialization and output scaling scheme that enables stable training of transformer networks with up to 1000 layers.
	Meta-Learning for Effective Multi-task and Multilingual Modelling Ishan Tarunesh, Sushil Khyalia, Vishwajeet Kumar, Ganesh Ramakrishnan, Preethi Jyothi EACL, 2021 code / pdf A meta-learning framework that enhances multi-task and multilingual models by optimizing knowledge transfer across diverse tasks and languages.
	Upper Bounds for All and Max-gain Policy Iteration Algorithms on Deterministic MDPs Ritesh Goenka, Eashan Gupta^, Sushil Khyalia^, Pratyush Agarwal^†, Mulinti Shaik Wajid^†, Shivaram Kalyanakrishnan arXiv, 2022 pdf Upper bounds on Policy Iteration algorithms' running time for deterministic MDPs derived using graph-theoretic analysis.
	Data Driven Phoneme Representations for a Lexicon Free Text to Speech of Low-Resource Languages Abhinav Garg, Jiyeon Kim, Sushil Khyalia, Chanwoo Kim, Dhananjaya Gowda ICASSP, 2024 pdf / samples Data-driven grapheme-to-phoneme representations using self-supervised learning eliminating the need for lexicons in text-to-speech systems while achieving competitive performance.
	STING: Self-attention based Time-series Imputation Networks using GAN Eunkyu Oh, Taehun Kim, Yunhu Ji, Sushil Khyalia ICDM, 2021 pdf A GAN-based self-attention network for imputing missing values in multivariate time series data, enhancing accuracy through bidirectional RNNs and novel attention mechanisms.
	SR-GCL: Session-Based Recommendation with Global Context Enhanced Augmentation in Contrastive Learning Eunkyu Oh, Taehun Kim, Minsoo Kim, Yunhu Ji, Sushil Khyalia DLG-AAAI, 2022 pdf Improving session-based recommendation by combining contrastive learning with global context-aware augmentations, leading to more accurate and robust predictions.
	PAC Mode Estimation using PPR Martingale Confidence Sequences Shubham Anand Jain^, Rohan Shah^, Sanit Gupta^†, Denil Mehta^†, Inderjeet J. Nair^†, Jian Vora^†, Sushil Khyalia, Sourav Das, Vinay J. Riberio, Shivaram Kalyanakrishnan AISTATS, 2022 pdf A novel stopping rule based on PPR martingale confidence sequences for PAC mode estimation, offering asymptotic optimality and improved sample efficiency for categorical distributions.

I used Jon Barron's website template source code as a base.