Christian Walder (1,2), Martin Breidt (1), Heinrich Bülthoff (1), Bernhard Schölkopf (1),
and Cristóbal Curio (1)
(1) Max Planck Institute for Biological Cybernetics, Tübingen, Germany
(2) Informatics and Mathematical Modelling, Technical University of Denmark
We present a novel algorithm for the marker-less tracking of deforming surfaces such as faces. We acquire a sequence of 3D scans along with color images at 40Hz. The data is then represented by implicit surface and color functions, using a novel partition-of-unity type method of efficiently combining local regressors using fast nearest neighbour searches. Both of these functions act on the 4D space of 3D plus time, and use temporal information to handle the noise in the individual scans. A template mesh may be interactively or automatically registered to the first frame in the sequence, and is then automatically deformed to track the scanned surface. This deformation is obtained as the solution of an energy minimization problem incorporating the shape, color and dynamics of the surface. Our unoptimised prototype system yields high-quality animated 3D models in correspondence, at a rate of approximately twenty seconds per frame. Tracking results for faces and other deformable objects are presented.
Presented at DAGM 2009 in Jena & received DAGM 2009 Prize .
Supported by DFG grant Perceptual Graphics and EU project BACS FP6-IST-027140
Video: Martin Breidt - Narrator: Roland Fleming
Publication record, PDF, conference proceedings.
Last updated: 05.10.09 (mb)
