Nithin Rao Koluguri

Featured In

• New Standard for Speech Recognition and Translation from the NVIDIA NeMo Canary Model
• Pushing the Boundaries of Speech Recognition with NVIDIA NeMo Parakeet ASR Models
• Speech AI Expands Global Reach With Telugu Language Breakthrough

Industry Experience

Since June 2020, I've been fortunate to work at NVIDIA on some amazing projects in speech AI. I created the TitaNet architecture for speaker recognition, which has become a go-to model across the field and still averages around 1.5 million downloads per month on Hugging Face. I also co-built the first speaker diarization modules in NeMo, using TitaNet embeddings and NME-SC clustering.

Beyond speaker systems, I led the billion-parameter scaling for FastConformer ASR models, helping push the boundaries of what's possible in speech recognition. I've also been leading the development of the Parakeet model series — including parakeet-tdt-0.6b-v2, which currently holds the #1 spot on the Hugging Face Open-ASR leaderboard.

These days, my work is focused on building the next generation of SpeechLM models, pushing the limits of what speech and language models can achieve.

Prior to NVIDIA, I gained valuable experience as an Applied Machine Learning Intern at the Bose CE Applied Research Group during the summer of 2019. There, I developed and deployed NLP and Computer Vision ML systems on mobile devices (Google Pixel) and Bose wearables.

My early industry experience includes working as a Software Developer at Robert Bosch Bangalore (July 2016 - January 2018). In this role, I contributed to Telematics Projects, focusing on customer and production diagnosis and designing features for Daimler. I also worked on integrating Text-to-Speech (TTS) functionalities for various in-car multimedia features like navigation and SMS readout.

Academic Experience

During my Master's degree at the University of Southern California (USC), I worked as a Graduate Research Assistant at the Signal Analysis and Interpretation Laboratory (SAIL) from September 2018 to December 2020. Under the guidance of Prof. Shrikanth Narayanan and Prof. Panayiotis Georgiou, my research explored innovative approaches to speaker diarization using prototypical networks, leveraging few-shot meta-learning techniques. I also contributed to improving Automatic Speech Recognition (ASR) by generating n-best path lists from speech recordings under various noise conditions using Kaldi, aiming to reduce ASR error rates.

Before my Master's studies, I served as a Research Assistant at the SPIRE LAB at the Indian Institute of Science (IISc), Bangalore, from January to July 2018. Advised by Prof. Prasanta Kumar Ghosh, my work focused on leveraging voice as a biomarker. I built speech classifiers to aid in the detection of Amyotrophic Lateral Sclerosis (ALS) and Parkinson's (PD) diseases and analyzed the effectiveness of different speech stimuli and recording devices for this diagnostic goal.

Research

I am interested in understanding signal level properties of audio,speech and Image. My research experiences thus far delve in natural language processing and applying machine learning algorithms on speech & image.

Academic Reviewer

Blogs

inspired from this website