Dear All,

You are cordially invited to the "AI Paper Story Sharing" online lecture to be delivered by Dr. Yu DENG on June 17 (Friday). We are looking forward to seeing you at this lecture!

Lecture Information

Time & Date: 14:30 - 16:00, June 17 (Friday)

Venue: Bilibili Live Stream

Language: Chinese

Speaker: Dr. Yu DENG 

(Institute for Advanced Study, Tsinghua University)

Abstract: 3D-awareGANs are capable of synthesizing multi-view images of an object given only monocular 2D images as training data. The key to achieve this is an incorporation of an underlying 3D representation, where state-of-the-artmethods leverage neural radiance field (NeRF) due to its superiority shown in novel view synthesis. However, the high computational cost of NeRF in a GAN training paradigm great restricts the image generation quality of these methods. Under limited memory budgets, they can only afford sampling dozens of points for NeRF’s volumetric rendering process, which not only limits the expressive power of the generator to handle fine details but also impedes effective GAN training due to the noise caused by unstable Monte Carlo sampling. In this talk, I will present a recent work of us that tackles this problem by introducing a novel 3D representation called radiance manifolds. By regulating point sampling and radiance field learning on 2D surface manifolds in the 3D space, our generator can produce high quality images with realistic fine details and strong visual 3D consistency that significantly outperforms previous methods. I will also tell the story behind this work that how we came up with the idea and the difficulties we encountered in the entire process.

Biography

Yu Deng received his Ph.D. degree at Tsinghua University under the supervision of Prof. Harry Shum. He is currently working as a research intern at visual computing group in MSRA and is about to join Xiaobing as a researcher. His research interest includes 3D reconstruction, 3D representation learning, and neural rendering. He has published several papers on CVPR during his Ph.D. study. Before that he also received his B.S. degree from Tsinghua University.