Point2Skeleton: Learning Skeletal Representations from Point Clouds

Point2Skeleton: Learning Skeletal Representations from Point Clouds

Cheng Lin¹, Changjian Li², Yuan Liu¹, Nenglun Chen¹,
Yi-King Choi¹, Wenping Wang¹

¹The University of Hong Kong, ²University College London

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

Oral Presentation, Best Paper Candidate

Fig. 1. We introduce an unsupervised method to learn skeletal meshes from point clouds. The skeletal meshes contain both 1D curve segments and 2D surface sheets which can represent underlying structures of various shapes.

Abstract

We introduce Point2Skeleton, an unsupervised method to learn skeletal representations from point clouds. Existing skeletonization methods are limited to tubular shapes and the stringent requirement of watertight input, while our method aims to produce more generalized skeletal representations for complex structures and handle point clouds. Our key idea is to use the insights of the medial axis transform (MAT) to capture the intrinsic geometric and topological natures of the original input points. We first predict a set of skeletal points by learning a geometric transformation, and then analyze the connectivity of the skeletal points to form skeletal mesh structures. Extensive evaluations and comparisons show our method has superior performance and robustness. The learned skeletal representation will benefit several unsupervised tasks for point clouds, such as surface reconstruction and segmentation.

Paper [PDF]

Code [Github]

Citation:
Lin, Cheng, Changjian Li, Yuan Liu, Nenglun Chen, Yi-King Choi, and Wenping Wang. "Point2Skeleton: Learning Skeletal Representations from Point Clouds." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4277-4286. 2021. (bibtex)

Network Overview

Fig. 2. An overview of our Point2Skeleton pipeline. Given a point cloud as input, first, we learn a geometric transformation via convex combinations to predict the skeletal points together with their radii. Second, we connect the skeletal points to form a mesh structure; we initialize a graph structure using two simple priors and formulate a link prediction problem using a graph auto-encoder to obtain a complete skeletal mesh.

Representative Results

Comparison

Fig. 3. Qualitative comparison with the competitive point cloud skeletonization methods, i.e., L1-medial skeleton and deep point consolidation (DPC).

Applications - Reconstruction

Fig. 4. Unsupervised surface reconstruction from point clouds. Our method can produce watertight surfaces without the need of surface normals and can capture the details of thin structures.

Applications - Segmentation

Fig. 5. Unsupervised structural decomposition for point clouds by detecting dimensional changes and non-manifold branches on the skeletal mesh.

Applications - Inpaiting

Fig. 6. Skeletal mesh prediction for reconstructing complete surfaces from point clouds with missing regions.