Yu Zeng’s homepage

I am a Research Scientist at NVIDIA Research. My research interest lies in computer vision and deep learning. I have focused on two main areas: (1) multimodal generative AI for visual content creation, and (2) label-efficient deep learning. By combining these research areas, I aim to bridge human creativity and machine intelligence through user-friendly and socially responsible models while minimizing the need for intensive human supervision.

Before joining NVIDIA, I obtained my PhD from Johns Hopkins University. I also worked with the research teams at Adobe during my PhD and Master’s studies.

News

• Our work on text-to-image generation was accepted by CVPR 2024 (paper, page)
• Our work on portrait relighting was accepted by CVPR 2024 (paper , page)
• I was selected as one of the Rising Stars in AI by KAUST AI Initiative
• Our work on image inpainting was accepted by AAAI 2024 (coming soon)

Selected Honors & Awards

• KAUST AI Initiative Rising Stars in AI 
• JHU ECE Kewei Yang and Grace Xin Fellowship
• China National Scholarship
• Third place in 2018 OPPO Top AI Competition (Portrait Segmentation), 50,000 RMB bonus (3rd from 456 teams)

Selected Articles | Full List | Google Scholar

One-step diffusion policy: Fast visuomotor policies via diffusion distillation” arXiv preprint, 2024.
Z. Wang, Z. Li, A. Mandlekar, Z. Xu, J. Fan, Y. Narang, L. Fan, Y. Zhu, Y. Balaji, M. Zhou, M.-Y. Liu, Y. Zeng
[pdf] | [Porject Page]

JeDi: Joint-Image Diffusion Models for Finetuning-free Personalized Text-to-Image generation. CVPR. 2024.
Yu Zeng, Vishal M. Patel, Haocheng Wang, Xun Huang, Ting-Chun Wang, Ming-Yu Liu, Yogesh Balaji
pdf | Project Page

SceneComposer: Any-Level Semantic Image Synthesis. CVPR. 2023. (Highlight, top 2.5% submission)
Yu Zeng, Zhe Lin, Jianming Zhang, Qing Liu, John Collomosse, Jason Kuen, Vishal M. Patel
pdf | Project Page

SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches. CVPR. 2022.
Yu Zeng, Zhe Lin, Vishal M. Patel
pdf | Project | Demo | Code

CR-Fill: Generative Image Inpainting with Auxiliary Contextual Reconstruction. ICCV. 2021.
Yu Zeng, Zhe Lin, Huchuan Lu, Vishal M. Patel
pdf | Code

High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling. ECCV. 2020.
Yu Zeng, Zhe Lin, Jimei Yang, Jianming Zhang, Eli Shechtman, Huchuan Lu
pdf | Project | Demo

Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation. ICCV. 2019.
Zeng, Yu and Zhuge, Yunzhi and Lu, Huchuan and Zhang, Lihe
pdf | Code

Multi-source Weak Supervision for Saliency Detection. CVPR. 2019.
Zeng, Yu and Zhuge, Yunzhi and Lu, Huchuan and Zhang, Lihe and Qian, Mingyang and Yu, Yizhou
pdf | Code
Learning to Detect Salient Object with Multi-source Weak Supervision. TPAMI. 2021.
H Zhang, Y Zeng, H Lu, L Zhang, J Li, J Qi
pdf | Code

Learning to Promote Saliency Detectors. CVPR. 2018.
Yu Zeng, Huchuan Lu, Lihe Zhang, Mengyang Feng, Ali Borji
pdf | Code