Xingjian Danny Dong

xdong404 at usc dot edu

Los Angeles, CA

Welcome! I am Xingjian (Danny) Dong 董行健, a Computer Science Master’s student at University of Southern California, and advised by Gregory Yauney and Prof. Swabha Swayamdipta at the DILL Lab. Previously, I completed my undergrad as an honor graduate at Harbin Institute of Technology (HIT). I was a visiting student at UC Berkeley where I was selected in the Undergraduate Research Apprentice Program (URAP) advised by Prof. Gašper Beguš. Feel free to check out my CV.

I am broadly interested in Machine Learning and Natural Language Processing, and specially in Data-centric NLP. I’d like to explore how data affects the behavior of models in training and inference phases for LLMs. I’m also interested in LLM privacy and the efficiency, and am open to multiple new ideas at the era of LLMs.

I am actively looking for NLP PhD position for 26 Fall as well as research-based industry position. Feel free to contact me if you’re interested or just chat!

News

Nov 11, 2024	Exicited to attend EMNLP 2024 in Miami, FL! Will be presenting our paper about OOD Detection.
Oct 07, 2024	Attend COLM 2024 at University of Pennsylvania, Philadelphia, PA.
Sep 19, 2024	🎉 First research paper was accepted by EMNLP 2024 Findings! Happy to share new method for OOD Detection with NNK-Means.
Jan 08, 2024	Exicited to start working as a Graduate Research Assistant at DILL Lab.

Selected Publications

EMNLP

Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression

Aryan Gulati , Xingjian Dong , Carlos Hurtado , and 3 more authors

In Findings of EMNLP, 2024

Abstract

As language models become more general purpose, increased attention needs to be paid to detecting out-of-distribution (OOD) instances, i.e., those not belonging to any of the distributions seen during training. Existing methods for detecting OOD data are computationally complex and storage-intensive. We propose a novel soft clustering approach for OOD detection based on non-negative kernel regression. Our approach greatly reduces computational and space complexities (up to 11\times improvement in inference time and 87% reduction in storage requirements) and outperforms existing approaches by up to 4 AUROC points on four different benchmarks. We also introduce an entropy-constrained version of our algorithm, which leads to further reductions in storage requirements (up to 97% lower than comparable approaches) while retaining competitive performance. Our soft clustering approach for OOD detection highlights its potential for detecting tail-end phenomena in extreme-scale data settings.