Xiaodan Du

Teaching an Agent to Sketch One Part at a Time

Xiaodan Du*, Ruize Xu*, David Yunis*, Yael Vinker, Greg Shakhnarovich

Project Page | arXiv | Dataset | Checkpoints | Code
The 19th European Conference on Computer Vision (ECCV 2026)

We develop a method for producing vector sketches one part at a time. To do this, we train a VLM agent using a novel multi-turn process-reward GRPO following SFT. We also introduce a new dataset ControlSketch-Part, containing rich part-level annotations for sketches, obtained using a novel, generic automatic annotation pipeline.

SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction

Shester Gueuwou, Xiaodan Du, Greg Shakhnarovich, Karen Livescu, Alexander H. Liu

Project Page | arXiv | Code
The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025 Oral)

We introduce SHuBERT, a self-supervised representation learning approach that adapts the masked prediction for multi-stream visual sign language input, learning to predict multiple targets for corresponding to clustered hand, face, and body pose streams. SHuBERT achieves state-of-the-art performance on multiple sign language benchmarks.

SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale

Shester Gueuwou, Xiaodan Du, Greg Shakhnarovich, Karen Livescu

Project Page | arXiv
The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

We introduce SignMusketeers for Sign Language Translation at Scale. Using only 3% of compute, 41x less pretraining data, and 160x less pretraining epochs, it achieves competitive performance(-0.4 BLEU) compared to the recent ASL-English Translation SOTA.

Generative Models: What do they know? Do they know things? Let's find out!

Previous title: Intrinsic LoRA: A Generalist Approach for Discovering Knowledge in Generative Models

Xiaodan Du, Nicholas Kolkin, Greg Shakhnarovich, Anand Bhattad

Project Page | arXiv | Code
IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) workshops, 2024

We introduce Intrinsic LoRA (I-LoRA), a universal, plug-and-play approach that transforms any generative model into a scene intrinsic predictor, capable of extracting intrinsic scene maps directly from the original generator network.

Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation

Haochen Wang*, Xiaodan Du*, Jiahao Li*, Raymond A. Yeh, Greg Shakhnarovich

Project Page | arXiv | Code
IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023

Generating 3D objects from 2D diffusion models by chaining scores with NeRF gradients.

Text-Free Learning of a Natural Language Interface for Pretrained Face Generators

Xiaodan Du, Raymond A. Yeh, Nicholas Kolkin, Eli Shechtman, Greg Shakhnarovich

arXiv | Code
arXiv, 2022

We propose Fast text2StyleGAN, a natural language interface that adapts pre-trained GANs for text-guided human face synthesis.