Mapping high-fidelity 3D geometry to a representation that allows for intuitive edits remains an elusive goal in computer vision and graphics. The key challenge is the need to model both continuous and discrete shape variations. Current approaches, such as implicit shape representation, lack straightforward interpretable encoding, while others that employ procedural methods output coarse geometry. We present GeoCode, a technique for 3D shape synthesis using an intuitively editable parameter space. We build a novel program that enforces a complex set of rules and enables users to perform intuitive and controlled high-level edits that procedurally propagate at a low level to the entire shape. Our program produces high-quality mesh outputs by construction. We use a neural network to map a given point cloud or sketch to our interpretable parameter space. Once produced by our procedural program, shapes can be easily modified. Empirically, we show that GeoCode can infer and recover 3D shapes more accurately compared to existing techniques and we demonstrate its ability to perform controlled local and global shape manipulations.
We build a novel program that enforces a complex set of rules and enables users to perform intuitive and controlled high-level edits that procedurally propagate at a low-level to the entire shape. In our program, changing a human-interpretable parameter produces a set of low-level instructions which ensure the modified shape is structurally valid. Our program produces a high-quality and intuitively editable 3D mesh by construction.
For example, given the input shape (left), modifying the seat in isolation will lead to an undesirable result (middle). Our program (right) properly propagates the edit to the remainder of the shape.
Our program is implemented as a directed acyclic graph (DAG) comprised of operation nodes and edges. Selected operation nodes are parameterized by the input parameters, allowing the user to interact with the program and control the resulting shape. The operations are chained together, receive, process and then pass the information along their edges, and model shape elements and inter-part influences. The program has a single final node, which outputs a mesh.
GeoCode learns to map a point cloud or a sketch input to an intuitively editable parameter space. The input passes through the corresponding encoder to obtain an embedding vector which is then fed to a set of decoders that predict the interpretable parameters. The program enforces a set of rules that, given a parameter representation, produces a high-quality shape by construction.
@article{pearl2022geocode,
title={GeoCode: Interpretable Shape Programs},
author={Pearl, Ofek and Lang, Itai and Hu, Yuhua and Yeh, Raymond A. and Hanocka, Rana},
booktitle={arXiv preprint arxiv:2212.11715},
year={2022}
}