Photo-Inspired Model-Driven 3D Object Modeling

Kai Xu1,2, Hanlin Zheng3, Hao Zhang2, Daniel Cohen-Or4, Ligang Liu3, Yueshan Xiong1

1National Univ. of Defense Technology, 2Simon Fraser Univ., 3Zhejiang Univ., 4Tel Aviv Univ.

ACM Transactions on Graphics (SIGGRAPH 2011), 30(4)


Figure 1: Photo-inspired 3D modeling of a chair from four different 3D candidates (cyan). The new models (yellow) are created as geometric variations of the candidates to fit the target object in the photo while preserving the 3D structure of the candidates.

We introduce an algorithm for 3D object modeling where the user draws creative inspiration from an object captured in a single photograph. Our method leverages the rich source of photographs for creative 3D modeling. However, with only a photo as a guide, creating a 3D model from scratch is a daunting task. We support the modeling process by utilizing an available set of 3D candidate models. Specifically, the user creates a digital 3D model as a geometric variation from a 3D candidate. Our modeling technique consists of two major steps. The first step is a user-guided image-space object segmentation to reveal the structure of the photographed object. The core step is the second one, in which a 3D candidate is automatically deformed to fit the photographed target under the guidance of silhouette correspondence. The set of candidate models have been pre-analyzed to possess useful high-level structural information, which is heavily utilized in both steps to compensate for the ill-posedness of the analysis and modeling problems based only on content in a single image. Equally important, the structural information is preserved by the geometric variation so that the final product is coherent with its inherited structural information readily usable for subsequent model refinement or processing.

Paper PDF

PDF, 12.6MB

Conference Slides

PPTX, 14.3MB


MOV, 47.9MB

Algorithm Overview

Figure 2: Algorithm overview. Given a single input photograph, a model-driven approach is applied to obtain a labeled segmentation of the photographed object (step 1). The candidate to be deformed can be randomly chosen or retrieved from the candidate set (step 2). The chosen candidate is deformed to achieve a fit in the silhouettes while preserving its structure (step 3).

Structure Optimization

Figure 3: One iteration step of controller optimization when deforming a 3D candidate (b) to fit a photo (a). The result of reconstructing external controllers (c), though fitting well to the photo silhouette, violates the inherent structure of the candidate, e.g., proximity and symmetry (see insert) between the controllers. We first symmetrize the individual controllers (d) and then optimize the structure using symmetry (e) and proximity constraints (f). The final controllers are well structured (g) and the underlying geometry is deformed accordingly (h).

The Google chair challenge

Figure 4: The "Google chair challenge": modeling chairs after Google image search returns on “chair”. Top row lists the top 11 returned images from the search. We are unable to model objects contained in the images marked by red boxes. 3D models created from other photos (yellow) are shown below where the candidates were retrieved from the database.


Figure 5: A gallery of 3D model creations for different object classes and from varying photographic inspirations. In each case, one 3D candidate is deformed to fit three photographed objects. Note that the model created out of the marked chair photo does not match well the image silhouette since its cuboid controllers are not allowed to bend. The 3D candidate was chosen randomly from the chair set.

Readily Usability

Figure 6: The structure-preserving deformation retains the structural information in the candidate so that the produced variation (b) remains readily usable. Based on the structural information (c), the user can perform further editing (d-f) using the structure-preserving shape manipulation of [Zheng et al. 2011].

We first thank the anonymous reviewers for their valuable comments and suggestions. We are grateful to the authors of [Zheng et al. 2011] for sharing an early manuscript of their work. Thanks also go to Aiping Wang from NUDT for fruitful discussion. This work is supported in part by grants from NSERC (No. 611370), a research fund for the Doctoral Program of Higher Education of China (No. 20104307110003), the Israeli Ministry of Science, the Israel Science Foundation, the National Science Foundation of China (61070071), and the 973 National Key Basic Research Foundation of China (No. 2009CB320801).

@article {xu_sig11,
    title = {Photo-Inspired Model-Driven 3D Object Modeling},
    author = {Kai Xu and Hanlin Zheng and Hao Zhang and Daniel Cohen-Or and Ligang Liu and Yueshan Xiong}
    journal = {ACM Transactions on Graphics, (Proc. of SIGGRAPH 2011)},
    volume = {30},
    number = {4},
    pages = {80:1--80:10},
    year = {2011}

    Back to TopHome