Research

What I work on

I work broadly at the intersection of Natural Language Processing and Machine Learning. I have particular interest in problems pertaining to scenarios where scaling datasets is not a viable solution, most notably in multilingual models. A significant part of my research is motivated by challenges specifc to my mother tongue, Gujarati. The research questions that interest me the most are described below:

Architectural Improvements for Multilingual Models

Can we adapt the transformer architecture and its training procedure to better represent constituent languages?
Can smarter design choices alleviate the 'curse of multinguality'?

Balanced Tokenization Across Languages

Can we effectively retrofit tokenizers and model weights to reduce 'token over-fragmentation'?

Dataset Creation in Low-Resource Scenarios

What are the challenges in creating and verifying labeled datasets for low-resource languages and specialized domains?
Can LLMs be leveraged to aid annotation in such scenarios?