RGBD Image Processing for 3D Modeling and Texturing

Hwasup Lim, Seungkyu Lee, and Jongwoo Lim


Commodity depth cameras become increasingly popular for the last couple of years in that they usually can capture both color and depth images with affordable quality in real-time. The main application of depth cameras targets natural user interface for games or human computer interaction. Recently new opportunities of using depth cameras are opening for 3D modeling and texturing through interests of research communities. Many researchers, however, are employing depth cameras without in-depth understanding of their characteristics and without good guides of suitable algorithms for their purposes. Main concerns of using depth cameras are their limited depth accuracy caused by their methods for distance measurement (e.g., time of flight, structured light, or depth from focus), and low quality color image by cheap color sensors. In this tutorial, we will start with various depth enhancement techniques according to their depth sensing mechanisms. Then, we will address the practical issues in the use of depth cameras and appropriate solutions to overcome their limitations especially for surface reconstruction and texture map generation. The main focuses will be put on the integration of multiple color-depth images for depth accuracy enhancement and the use of multiple color images by external color sensors for texture quality enhancement. Finally new research possibilities on deformable object modeling using depth cameras will be introduced with the comparison of the past and current state of the art algorithms. Unlike previous tutorials with similar topics, this tutorial presents research issues on the full pipeline from depth acquisition to 3D textured model generation.


Dr. Hwasup Lim received his Ph.D. degree in Electrical Engineering from the Pennsylvania State University in 2007. He was a R&D staff member at Advanced Media Lab, Samsung Advanced Institute of Technology (SAIT) from 2007 to 2011, where his research focused on TOF depth image processing and 3D human modeling & tracking. He is currently a senior research scientist at Imaging Media Research Center, Korea Institute of Science and Technology (KIST). His research interests include RGBD image-based 3D modeling, hand tracking, and object recognition.

Seungkyu Lee received his PhD degree in computer science and engineering from Penn State University, US. He has been a research engineer in Korea Broadcasting System Technical Research Institute where he carried out research on HD image processing, MPEG4-AVC, and the standardization of Terrestrial-Digital Mobile Broadcasting. He has been a principal research scientist, Advanced Media Lab., Samsung Advanced Institute of Technology. He is an assistant professor at Kyung Hee University. His research interests include depth sensor, color/depth image processing, symmetry-based computer vision, and 3-D modeling/reconstruction.

Jongwoo Lim graduated from Seoul National University, Seoul, Korea in 1997, and received M.S. degree in 2003 and Ph.D. degree in 2005 from University of Illinois at Urbana-Champaign (UIUC), USA. He worked at Honda Research Institute USA, Mountain View, CA, USA. as a senior scientist from 2005 to 2011, and at Google inc. as a software engineer from 2011 to 2012. Since 2012 he is an assistant professor of Computer Science & Engineering division at Hanyang university, Seoul, Korea. His research interests include computer vision, robotics, and machine learning.