Point2Skeleton: Learning Skeletal Representations from Point Clouds
Cheng Lin1Changjian Li2Yuan Liu1,  Nenglun Chen1
Yi-King Choi1Wenping Wang1
1The University of Hong Kong,   2University College London
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Oral Presentation, Best Paper Candidate
Paper teaser
Fig. 1. We introduce an unsupervised method to learn skeletal meshes from point clouds. The skeletal meshes contain both 1D curve segments and 2D surface sheets which can represent underlying structures of various shapes.
We introduce Point2Skeleton, an unsupervised method to learn skeletal representations from point clouds. Existing skeletonization methods are limited to tubular shapes and the stringent requirement of watertight input, while our method aims to produce more generalized skeletal representations for complex structures and handle point clouds. Our key idea is to use the insights of the medial axis transform (MAT) to capture the intrinsic geometric and topological natures of the original input points. We first predict a set of skeletal points by learning a geometric transformation, and then analyze the connectivity of the skeletal points to form skeletal mesh structures. Extensive evaluations and comparisons show our method has superior performance and robustness. The learned skeletal representation will benefit several unsupervised tasks for point clouds, such as surface reconstruction and segmentation.
  Paper [PDF]

Code [Github]

Lin, Cheng, Changjian Li, Yuan Liu, Nenglun Chen, Yi-King Choi, and Wenping Wang. "Point2Skeleton: Learning Skeletal Representations from Point Clouds." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4277-4286. 2021.  (bibtex)

Network Overview
Fig. 2. An overview of our Point2Skeleton pipeline. Given a point cloud as input, first, we learn a geometric transformation via convex combinations to predict the skeletal points together with their radii. Second, we connect the skeletal points to form a mesh structure; we initialize a graph structure using two simple priors and formulate a link prediction problem using a graph auto-encoder to obtain a complete skeletal mesh.
Representative Results
Fig. 3. Qualitative comparison with the competitive point cloud skeletonization methods, i.e., L1-medial skeleton and deep point consolidation (DPC).
Applications - Reconstruction
Fig. 4. Unsupervised surface reconstruction from point clouds. Our method can produce watertight surfaces without the need of surface normals and can capture the details of thin structures.
Applications - Segmentation
Fig. 5. Unsupervised structural decomposition for point clouds by detecting dimensional changes and non-manifold branches on the skeletal mesh.
Applications - Inpaiting
Fig. 6. Skeletal mesh prediction for reconstructing complete surfaces from point clouds with missing regions.
©Changjian Li. Last update: June 23, 2021.