Wenhao Li, Zhiyuan Yu, Qijin She, Zhinan Yu, Yuqing Lan, Chenyang Zhu, Ruizhen Hu, Kai Xu*, "LLM-enhanced Scene Graph Learning for Household Rearrangement", SIGGRAPH Asia 2024. [Paper | Project page]

The household rearrangement involves both common-sense knowledge on the objective side and human user preference on the subjective side.

We propose to mine object functionality with user preference alignment directly from the scene itself through LLM-enhanced scene graph learning which transforms the input scene graph into an affordance-enhanced graph with information-enhanced nodes and newly discovered edges.

Hang Zhao, Zherong Pan, Yang Yu, Kai Xu*, "Learning Physically Realizable Skills for Online Packing of General 3D Shapes", ACM Transactions on Graphics (presented at SIGGRAPH 2024). [Paper | Project page | Code & data]

We study the problem of learning online packing skills for irregular 3D shapes where we take physical realizability into account, involving physics dynamics and constraints of a placement. The complex irregular geometry and imperfect object placement together lead to huge solution space. Direct training in such space is prohibitive. We propose a theoretically-provable method for candidate action generation to reduce the action space of RL and the learning burden.

Qijin She, Shishun Zhang, Yunfan Ye, Ruizhen Hu, Kai Xu*, "Learning Cross-hand Policies for High-DOF Reaching and Grasping", ECCV 2024. [Paper | Project page | Code]

We propose a method that can learn a unified reaching-and-grasping policy that can be easily transferred to different dexterous grippers, based on the IBS representation of dynamic grasping.

We adopt a decoupled learning scheme: 1) a gripper-agnostic policy model that predicts the displacements of pre-defined key points on the gripper, and 2) a gripper-specific adaptation model that translates these displacements into adjustments for controlling the grippers' joints.

Sisi Dai, Wenhao Li, Haowen Sun, Haibin Huang, Chongyang Ma, Hui Huang, Kai Xu*, Ruizhen Hu, "InterFusion: Text-Driven Generation of 3D Human-Object Interaction", ECCV 2024. [Paper | Project page | Code]

We tackle the generating 3D human-object interactions from textual descriptions in a zero-shot text-to-3D manner. We address two key challenges: the unsatisfactory outcomes of direct text-to-3D methods in HOI, largely due to the lack of paired text-interaction data, and the inherent difficulties in simultaneously generating multiple concepts with complex spatial relationships.

Zhiyuan Yu, Zheng Qin, Lintao Zheng, Kai Xu*, "Learning Instance-Aware Correspondences for Robust Multi-Instance Point Cloud Registration in Cluttered Scenes", CVPR 2024. [Paper | Code]

Multi-instance point cloud registration estimates the poses of multiple instances of a model point cloud in a scene point cloud.

We propose MIRETR, Multi-Instance REgistration TRansformer, a coarse-to-fine approach to the extraction of instance-aware correspondences. At the coarse level, it jointly learns instance-aware superpoint features and predicts per-instance masks. With instance masks, the influence from outside of the instance being concerned is minimized, such that highly reliable superpoint correspondences can be extracted.

Junwen Huang, Alexey Artemov, Yujin Chen, Shuaifeng Zhi, Kai Xu, Matthias Nießner, "SSR-2D: Semantic 3D Scene Reconstruction from 2D Images", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024. [Paper | Code]

We explore a central 3D scene modeling task, namely, semantic scene reconstruction without using any 3D annotations. The key idea of our approach is to design a trainable model that employs both incomplete 3D reconstructions and their corresponding source RGB-D images, fusing cross-domain features into volumetric embeddings to predict complete 3D geometry, color, and semantics with only 2D labeling which can be either manual or machine-generated.

Wenhao Li, Shishun Zhang, Sisi Dai, Hui Huang, Ruizhen Hu, Xiaohong Chen, Kai Xu*, "Synchronized Dual-arm Rearrangement via Cooperative mTSP", ICRA 2024. [Paper | Code]

We formulated the problem of synchronized dual-arm rearrangement as cooperative mTSP, a variant of mTSP where agents share cooperative costs, and utilized reinforcement learning for its solution. We devise an attention-based network working on task state graph for task scheduling.

Shishun Zhang, Qijin She, Wenhao Li, Chenyang Zhu, Yongjun Wang, Ruizhen Hu, Kai Xu*, "Learning Dual-arm Object Rearrangement for Cartesian Robots", ICRA 2024. [Paper | Code]

This work focuses on the dual-arm object rearrangement problem abstracted from a realistic industrial scenario of Cartesian robots. The goal of this problem is to transfer all the objects from sources to targets with the minimum total completion time.

Yihan Cao, Jiazhao Zhang, Zhinan Yu, Kai Xu*, "Neural Observation Field Guided Hybrid Optimization of Camera Placement", The IEEE Robotics and Automation Letters, 2024. [Paper | Code]

Camera placement is crutial in multi-camera systems. Its challenge lies in the nonlinear nature of high-dim parameters and the unavailability of gradients for target functions like coverage and visibility. We present a hybrid method incorporating both gradient-based and non-gradient-based optimizations, enjoying the advantages of both smooth convergence and robustness.

Yunfan Ye, Kai Xu*, Yuhang Huang, Renjiao Yi, Zhiping Cai, "DiffusionEdge: Diffusion Probabilistic Model for Crisp Edge Detection", AAAI 2024. [Paper | Code]

We propose the first diffusion model for the task of general edge detection, which we call DiffusionEdge. To avoid expensive computational resources while retaining the generation performance, we apply our recently proposed diffusion model ADM in the latent space and enable the classic cross-entropy loss which is uncertainty-aware in pixel level to directly optimize the parameters in latent space in a distillation manner. We also adopt a decoupled architecture to speed up the denoising process and propose a corresponding adaptive Fourier filter to adjust the latent features of specific frequencies.

Yuhang Huang, Zheng Qin, Xinwang Liu, Kai Xu*, "Simultaneous Image-to-Zero and Zero-to-Noise: Diffusion Models with Analytical Image Attenuation", arXiv:2306.13720. [Paper | Code]

We propose an extrememly powerful diffusion model ADM. ADM turns the forward image-to-noise mapping into image-to-zero mapping and zero-to-noise mapping. It achieves high-quality results in several generative tasks with much less diffusion steps, thus greatly improving the generation speed. PLEASE TRY IT!

Hui Tian, Chenyang Zhu, Yifei Shi, Kai Xu*, "SuperUDF: Self-supervised UDF Estimation for Surface Reconstruction", IEEE Transactions on Visualization and Computer Graphics (TVCG), 2024. [Paper | Code]

SuperUDF is a self-supervised UDF learning which exploits a learned geometry prior for efficient training and a novel regularization for robustness to sparse sampling. The core idea draws inspiration from the classical surface approximation operator of locally optimal projection (LOP).

Xiong-Hui Chen, Junyin Ye, Hang Zhao, Yi-Chen Li, Xu-Hui Liu, Haoran Shi, Yu-Yan Xu, Zhihao Ye, Si-Hang Yang, Yang Yu, Anqi Huang, Kai Xu, Zongzhang Zhang, "Deep Demonstration Tracing: Learning Generalizable Imitator Policy for Runtime Imitation from a Single Demonstration", ICML 2024. [Paper | Code]

One-shot imitation learning (OSIL) is to learn an imitator agent that can execute multiple tasks with only a single demonstration. In real-world scenario, the environment is dynamic, e.g., unexpected changes can occur after demonstration. Thus, achieving generalization of the imitator agent is crucial as agents would inevitably face situations unseen in the provided demonstrations. We present Deep Demonstration Tracing (DDT), a demonstration transformer architecture to encourage agents to adaptively trace suitable states in demonstrations.

Xiaogang Wang, Yuhang Cheng, Ziyang Fan, Kai Xu, "Learning to Transfer Heterogeneous Translucent Materials from a 2D Image to 3D Models", ACM Multimedia 2024. [Paper]

Great progress has been made in rendering translucent materials in recent years, but automatically estimating parameters for heterogeneous materials such as jade and human skin remains a challenging task, often requiring specialized and expensive physical measurement devices. In this paper, we present a novel approach for estimating and transferring the parameters of heterogeneous translucent materials from a single 2D image to 3D models.

Xuefeng Yin, Chenyang Zhu, Shanglai Qu, Yuqi Li, Kai Xu, Baocai Yin, Xin Yang, "CSO: Constraint-guided Space Optimization for Active Scene Mapping", ACM Multimedia 2024. [Paper]

Simultaneously mapping and exploring a complex unknown scene is an NP-hard problem. We present CSO, a deep reinforcement learning-based framework for efficient active scene mapping.

Xiaogang Wang, Liang Wang, Hongyu Wu, Guoqiang Xiao, Kai Xu*, "Parametric Primitive Analysis of CAD Sketches With Vision Transformer", IEEE Transactions on Industrial Informatics, 2024. [Paper]

The interpretation of CAD sketches plays a crucial role in industrial product design. To address the error accumulation in autoregressive models and the complexities associated with self-supervised model design, we propose a two-stage network framework. It consists of a primitive network and a constraint network, transforming the sketch analysis task into a set prediction problem to enhance the effective handling of primitives and constraints. By decoupling target types from parameters, it gains increased flexibility and optimality while reducing complexity.

Yijie Tang, Jiazhao Zhang, Zhinan Yu, He Wang, Kai Xu*, "MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction", ACM Transactions on Graphics (SIGGRAPH Asia 2023), 42(6). [Paper | Project page | Code & data]

We introduce MIPS-Fusion, a robust and scalable online RGB-D reconstruction method based on a novel neural implicit representation – multi-implicit-submap. Neural submaps are allocated incrementally alongside the scanning trajectory, learned efficiently with local bundle adjustments, refined distributively in a back-end optimization, and optimized globally in realizing submap-level loop closure. We also propose a hybrid tracking approach combining randomized and gradient-based pose optimizations. For the first time, randomized optimization is made possible in neural tracking with several key designs to the learning process, enabling efficient and robust tracking even under fast camera motions.

Boyan Wan, Yifei Shi, Kai Xu*, "SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations", ICCV 2023. [Paper | Code]

We propose SOCS for category-level 6D pose estimation. SOCS is semantically coherent: Any point on the surface of a object can be mapped to a semantically meaningful location in SOCS, allowing for accurate pose and size estimation under large shape variations. Our method is well-generalizing for large intra-category shape variations and robust to inter-object occlusions

Jingjia Shi, Shuaifeng Zhi, Kai Xu*, "PlaneRecTR: Unified Query learning for 3D Plane Recovery from a Single View", ICCV 2023. [Paper | Project page | Youtube / Bilibli | Code]

PlaneRecTR is a vision transformer architecture with query-based learning, and for the first time unifies all subtasks of single-view plane recovery with a single compact model. Mutual benefits between planar geometry and segmentation lead to SOTA performance.

Minhao Li, Zheng Qin, Zhirui Gao, Renjiao Yi, Chenyang Zhu, Yulan Guo, Kai Xu*, "2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds", ICCV 2023. [Paper | Code]

The commonly adopted detect-then-match approach to registration finds difficulties in the cross-modality cases due to the incompatible keypoint detection and inconsistent feature description. We propose, 2D3D-MATR, a detection-free method for accurate and robust registration between images and point clouds.

Yifei Shi, Junhua Xi, Dewen Hu, Zhiping Cai, Kai Xu*, "RayMVSNet++: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereo", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023. [Paper | Code]

This is an enhancement of our RayMVSNet paper with contextual feature aggregation for each ray. We leverage an attentional gating unit for selecting semantically relevant neighboring rays within the local frustum around a ray. It improves the performance on more challenging datasets (e.g. low-quality images caused by poor lighting conditions or motion blur).

Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, Slobodan Ilic, Dewen Hu*, Kai Xu*, "GeoTransformer: Fast and Robust Point Cloud Registration with Geometric Transformer", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023. [Paper | Code]

This is the journal extension of our CVPR 2022 paper with 1) significantly reduced (17%) memory footprint and computational cost, 2) handling of non-rigid registration, and 3) more thorough evaluations and in-depth analysis.

Yunfan Ye, Renjiao Yi, Zhirui Gao, Zhiping Cai*, Kai Xu*, "Delving into Crispness: Guided Label Refinement for Crisp Edge Detection", IEEE Transactions on Image Processing (TIP), 2023. [Paper | Code]

We find that label quality is more important than model design to achieving crisp edge detection. We propose an iterative Canny-guided refinement of human-labeled edges whose result can be used to train crisp edge detectors.

Yunfan Ye, Renjiao Yi, Zhiping Cai*, Kai Xu*, "STEdge: Self-training Edge Detection with Multi-layer Teaching and Regularization", IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2023. [Paper | Code]

We propose self-training edge detection, leveraging the untapped wealth of large-scale unlabeled image datasets. We design a self-supervised framework which achieves significant performance boost over supervised methods with lightweight finetuning on the target dataset.

Hui Tian, Zheng Qin, Renjiao Yi, Chenyang Zhu, Kai Xu*, "Tensorformer: Normalized Matrix Attention Transformer for High-quality Point Cloud Reconstruction", IEEE Transactions on Multimedia (TMM), 2023. [Paper | Code]

Transformer-based methods to point cloud reconstruction can work without normals but without rich local details. We introduce a novel normalized matrix attention transformer, Tensorformer. It allows for simultaneous point-wise and channel-wise message passing. It brings more degrees of freedom in feature learning and thus facilitates better modeling of local geometric details.

Zheng Qin, Changjian Wang, Yuxing Peng, Kai Xu*, "CasViGE: Learning robust point cloud registration with cascaded visual-geometric encoding", Computer Aided Geometric Design (CAGD), Volume 104, 2023. [Paper | Code]

Recent methods to point cloud registration attempt to inject the visual information from RGB images to obtain more accurate correspondences. However, as 2D and 3D convolutions have different inductive biases, this simplistic method ignores the intrinsic correlation between the two modalities, which harms the distinctiveness of the point descriptors. CasViGE iteratively fuses the inter-modality features by leveraging the inductive biases of both 2D and 3D convolutions, which better considers the correlation between the two modalities. As a plug-and-play module, it attains significant improvements on various registration methods.

Yunfan Ye, Renjiao Yi, Zhirui Gao, Chenyang Zhu, Zhiping Cai, Kai Xu*, "NEF: Neural Edge Fields for 3D Parametric Curve Reconstruction from Multi-view Images", CVPR 2023. [Paper | Project page | Code]

We study the problem of reconstructing 3D feature curves of an object from a set of calibrated multi-view images. To do so, we learn a neural implicit field representing the density distribution of 3D edges which we refer to as Neural Edge Field (NEF). Inspired by NeRF, NEF is optimized with a view-based rendering loss where a 2D edge map is rendered at a given view and is compared to the ground-truth edge map extracted from the image of that view.

Zheng Qin, Hao Yu, Changjian Wang, Yuxing Peng, Kai Xu*, "Deep Graph-based Spatial Consistency for Robust Non-rigid Point Cloud Registration", CVPR 2023. [Paper | Code]

We study the problem of outlier correspondence pruning for non-rigid point cloud registration. We propose Graph-based Spatial Consistency Network (GraphSCNet) to filter outliers for non-rigid registration. Our method is based on the fact that non-rigid deformations are usually locally rigid, or local shape preserving.

Renjiao Yi, Chenyang Zhu, Kai Xu*, "Self-supervised Non-Lambertian Single-view Image Relighting", CVPR 2023. [Paper | Code]

We present a learning-based approach to relighting a single image of non-Lambertian objects involving both inverse rendering and re-rendering. We propose a self-supervised method for inverse rendering with a low-rank constraint. To facilitate the learning, we contribute Relit, a large-scale dataset of videos with aligned objects under changing illuminations.

Jiazhao Zhang, Liu Dai, Fanpeng Meng, Qingnan Fan, Xuelin Chen, Kai Xu, He Wang, "3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification", CVPR 2023. [Paper | Code]

We propose a framework for the challenging 3D-aware ObjectNav based on two straightforward sub-policies. The two sub-polices, namely corner-guided exploration policy and category-aware identification policy, simultaneously perform by utilizing online fused 3D points as observation.

Sheng Ao, Qingyong Hu, Hanyun Wang, Kai Xu, Yulan Guo, "BUFFER: Balancing Accuracy, Efficiency, and Generalizability in Point Cloud Registration", CVPR 2023. [Paper | Code]

An ideal point cloud registration framework should have superior accuracy, acceptable efficiency, and strong generalizability. We propose BUFFER, a point cloud registration method for balancing accuracy, efficiency, and generalizability. The key is to take advantage of both point-wise and patch-wise techniques, while overcoming the inherent drawbacks simultaneously.

Yaqiao Dai, Renjiao Yi, Chenyang Zhu, Hongjun He, Kai Xu* "Multi-resolution Monocular Depth Map Fusion by Self-supervised Gradient-based Composition", AAAI 2023, Oral presentation. [Paper | Code]

We propose a novel depth map fusion module to combine the advantages of estimations with multi-resolution inputs. Instead of merging the low- and high-resolution estimations equally, we adopt the core idea of Poisson fusion, trying to implant the gradient domain of high-resolution depth into the low-resolution depth.

Zeyu Huang, Juzhan Xu, Sisi Dai, Kai Xu, Hao Zhang, Hui Huang, Ruizhen Hu, "NIFT: Neural Interaction Field and Template for Object Manipulation", ICRA 2023. [Paper | Code]

We introduce NIFT, Neural Interaction Field and Template, a descriptive and robust interaction representation of object manipulations to facilitate imitation learning. Given a few object manipulation demos, NIFT guides the generation of the interaction imitation for a new object instance by matching the Neural Interaction Template (NIT) extracted from the demos in the target Neural Interaction Field (NIF) defined for the new object. Specifically, NIF is a neural field that encodes the relationship between each spatial point and a given object.

Jiazhao Zhang, Yijie Tang, He Wang, Kai Xu*, "ASRO-DIO: Active Subspace Random Optimization based Depth Inertial Odometry", ICRA 2023 (IEEE T-RO paper track). [Paper | Code]

ASRO-DIO enables real-time RGB-D reconstruction under extremely fast camera motions. To the center of ASRO-DIO is the fast and robust Depth-IMU odometry with efficient active subspace randomized optimization in the 18D state space of IMU tracking.

Liqiang Lin, Pengdi Huang, Chi-Wing Fu, Kai Xu, Hao Zhang, Hui Huang, "On Learning the Right Attention Point for Feature Enhancement", Science China (Information Sciences), 2023, 66: 112107. [Paper]

An attention-based mechanism to learn enhanced point features for point cloud processing tasks. Unlike prior studies, which were trained to optimize the weights of a pre-selected set of attention points, our approach learns to locate the best attention points to maximize the performance of a specific task, e.g., point cloud classification. Importantly, we advocate the use of single attention point to facilitate semantic understanding in point feature learning.

Qijin She, Ruizhen Hu, Junzhan Xu, Min Liu, Kai Xu*, Hui Huang*, "Learning High-DOF Reaching-and-Grasping via Dynamic Representation of Gripper-Object Interaction", ACM Transactions on Graphics (SIGGRAPH 2022). [Paper | Project page | Code]

We represent a grasp with Interaction Bisector Surface and find that it is surprisingly effective as a state representation since it well informs the fine-grained control of each finger with spatial relation against the target object. It facilitates learning a strong control model of high-DOF grasping with good sample efficiency, dynamic adaptability, and cross-category generality.

Jiazhao Zhang, Yijie Tang, He Wang, Kai Xu*, "ASRO-DIO: Active Subspace Random Optimization based Depth Inertial Odometry", IEEE Transactions on Robotics (TRO). [Paper | Code]

This is an extension of ROSEFusion which enables realtime RGB-D reconstruction under fast camera motion via random optimization. ASRO-DIO achieves robust Depth-IMU odometry and supports even faster camera motion! To realize efficient random optimization in the 18D state space of IMU tracking, we propose to identify and sample particles from active subspace.

Yifei Shi, Xin Xu, Junhua Xi, Xiaochang Hu, Dewen Hu, Kai Xu*, "Learning to Detect 3D Symmetry from Single-view RGB-D Images with Weak Supervision". IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). [Paper | Code]

This is an extension SymmetryNet which detects object-level symmeries from a single-view RGB-D image with strong supervision. In this work, we present a 3D symmetry detection approach to detect symmetry from single-view RGB-D images without using symmetry supervision. The key idea is to train the network in a weakly-supervised learning manner to complete the shape based on the predicted symmetry such that the completed shape be similar to existing plausible shapes.

Hang Zhao, Yang Yu, Kai Xu*, "Learning Efficient Online 3D Bin Packing on Packing Configuration Trees". ICLR 2022. [Paper | Code]

We propose to enhance the practical applicability of online 3D-BPP via learning on a novel hierarchical representation - packing configuration tree (PCT). PCT is a full-fledged description of the state and action space of bin packing which can support packing policy learning based on deep reinforcement learning (DRL). In training, PCT expands based on heuristic rules. However, the DRL model learns a much more effective and robust packing policy than heuristics.

Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, Kai Xu*, "Geometric Transformer for Fast and Robust Point Cloud Registration", CVPR 2022, Oral presentation. [Paper | Code]

GeoTransformer encodes distance and angular information of superpoints sampled from point clouds, thus enabling the learning of rotation-invariant representation of global structures. The resultant features leads to high-quality point correspondences. This makes it possible that fast and accurate point cloud registration is achieved in a RANSAC-free manner. Our method attains 17%~31% performance boost on the challenging dataset of 3DLoMatch, with a 100x faster speed.

Chengjie Niu, Manyi Li, Kai Xu*, Hao Zhang, "RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures", CVPR 2022. [Paper | Code]

We introduce RIM-Net, a neural network which learns recursive implicit fields for unsupervised inference of hierarchical shape structures. Our network recursively decomposes an input 3D shape into two parts, resulting in a binary tree hierarchy. Each level of the tree corresponds to an assembly of shape parts, represented as implicit functions, to reconstruct the input shape.

Junhua Xi, Yifei Shi, Yijie Wang, Yulan Guo, Kai Xu*, "RayMVSNet: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereo", CVPR 2022. [Paper | Code]

Different from existing works on deep MVS dedicated to adaptive refinement of cost volumes, we opt to directly optimize the depth value along each camera ray, mimicking the range (depth) finding of a laser scanner. This reduces the MVS problem to ray-based depth optimization which is much more light-weight than full cost volume optimization.

Yao Duan, Chenyang Zhu, Yuqing Lan, Renjiao Yi, Xinwang Liu, Kai Xu*, "DisARM: Displacement Aware Relation Module for 3D Detection", CVPR 2022. [Paper | Code]

The core idea of DisARM is that contextual information is critical to tell the difference between different objects when the instance geometry is incomplete or featureless. We find that relations between proposals provide a good representation to describe the context. Rather than working with all relations, we find that training with relations only between the most representative ones, or anchors, can significantly boost the detection performance.

Kunhong Li, Longguang Wang, Li Liu, Qing Ran, Kai Xu, Yulan Guo, "Decoupling Makes Weakly Supervised Local Feature Better", CVPR 2022. [Paper | Code]

Weakly supervised learning can help local feature methods to overcome the obstacle of acquiring a large-scale dataset with densely labeled correspondences. However, since weak supervision cannot distinguish the losses caused by the detection and description steps, directly conducting weakly supervised learning within a joint training describethen- detect pipeline suffers limited performance. We propose a decoupled training describe-then-detect pipeline tailored for weakly supervised local feature learning, where the detection step is decoupled from the description step and postponed until discriminative and robust descriptors are learned.

Suyuan Liu, Siwei Wang, Pei Zhang, Xinwang Liu, Kai Xu, Changwang Zhang, Feng Gao, "Efficient One-pass Multi-view Subspace Clustering with Consensus Anchors", AAAI 2022. [Paper]

We propose a scalable and parameter-free multi-view subspace clustering method to directly output the clustering labels with optimal anchor graph.

Yi Zhang, Xinwang Liu, Jiyuan Liu, Sisi Dai, Changwang Zhang, Kai Xu, En Zhu, "Fusion Multiple Kernel K-means", AAAI 2022. [Paper]

It unifies base partition learning and late fusion clustering into one single objective function, and adopts early fusion technique to capture more sufficient information in kernel matrices.

Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, Kai Xu*, "ROSEFusion: Random Optimization for Online Dense Reconstruction under Fast Camera Motion", ACM Transactions on Graphics (SIGGRAPH 2021). [Paper | Project page | Code & data]

Online reconstruction based on RGB-D sequences has thus far been restrained to relatively slow camera motions (<1m /s). Under very fast camera motion (e.g., 3m/s), the reconstruction can easily crumble even for the state-of-the-art methods. Fast motion brings two challenges to depth fusion: 1) the high nonlinearity of camera pose optimization due to large inter-frame rotations and 2) the lack of reliably trackable features due to motion blur. We propose to tackle the difficulties of fast-motion camera tracking in the absence of inertial measurements using random optimization. Our method attains good quality pose tracking under fast camera motion (up to 4m/s) in a realtime framerate without including loop closure or global pose optimization.

Hang Zhao, Chenyang Zhu, Xin Xu, Hui Huang, Kai Xu*, "Learning Practically Feasible Policies for Online 3D Bin Packing", Science China (Information Sciences). [Paper] (NOTE: This method is patent protected. Contact me for commercial use.)

This is a follow-up of our AAAI 2021 work on online 3D BPP. In this work, we aim to learn more PRACTICALLY FEASIBLE policies with REAL ROBOT TESTING! To that end, we propose three critical designs: (1) an online analysis of packing stability based on a novel stacking tree which is highly accurate and computationally efficient and hence especially suited for RL training, (2) a decoupled packing policy learning for different dimensions of placement for high-res spatial discretization and hence high packing precision, and (3) a reward function dictating the robot to place items in a far-to-near order and therefore simplifying motion planning of the robotic arm.

Jian Liu, Shiqing Xin, Xifeng Gao, Kaihang Gao, Kai Xu, Baoquan Chen, Changhe Tu, "Computational Object-Wrapping Rope Nets", ACM Transactions on Graphics (TOG). 41(1). [Paper]

We propose to compute a rope net that can tightly wrap around various 3D shapes. Based on the key observation that if every knot of the net has four adjacent curve edges, then only a single rope is needed to construct the entire net. We reformulate the rope net computation problem into a constrained curve network optimization and propose a discrete-continuous optimization.

Pengdi Huang, Liqiang Lin, Fuyou Xue, Kai Xu, Danny Cohen-Or, Hui Huang, "Hausdorff Point Convolution with Geometric Priors", Science China (Information Sciences). [Paper | Project page]

We advocate the use of Hausdorff distance as a shape-aware distance measure for calculating point convolutional responses. We present Hausdorff Point Convolution which constitutes a powerful point feature learning with a rather compact set of only four types of geometric priors as kernels and outperforms strong point convolution baselines (e.g., KPConv).

Yifei Shi, Junwen Huang, Xin Xu, Yifan Zhang, Kai Xu*, "StablePose: Learning 6D Object Poses from Geometrically Stable Patches", CVPR 2021. [Paper | Code]

We introduce the concept of geometric stability to the problem of 6D object pose estimation and propose to learn pose inference based on geometrically stable patches extracted from observed 3D point clouds. According to the theory of geometric stability analysis, a minimal set of three planar/cylindrical patches are geometrically stable and determine the full 6DoFs of the object pose. We train a deep neural network to regress 6D object pose based on geometrically stable patch groups via learning both intra-patch geometric features and inter-patch contextual features. Working with patch groups makes our method generalize well for random occlusion and unseen instances.

Xiaogang Wang, Xun Sun, Xinyu Cao, Kai Xu*, Bin Zhou*, "Learning Fine-Grained Segmentation of 3D Shapes without Part Labels", CVPR 2021. [Paper | Code ]

Learning-based 3D shape segmentation is usually formulated as a semantic labeling problem, assuming that all parts of training shapes are annotated with a given set of labels. This assumption, however, is unrealistic for training fine-grained segmentation on large datasets since the annotation of fine-grained parts is extremely tedious. In this paper, we approach the problem with deep clustering, where the key idea is to learn part priors from a dataset with fine-grained segmentation but no part annotations. We model the clustering priors of points with a similarity matrix and achieve part-based segmentation through minimizing a novel low rank loss.

Qian Xie, Yu-Kun Lai, Jing Wu, Zhoutao Wang, Yiming Zhang, Kai Xu, Jun Wang, "Vote-Based 3D Object Detection with Context Modeling and SOB-3DNMS", International Journal of Computer Vision (IJCV), 129(6):1857-1874. [Paper | Code]

We propose a novel 3D object detection network, which is built on the VoteNet but takes into consideration of the contextual information at multiple levels for detection and recognition of 3D objects. To encode relationships between elements at different levels, we introduce three contextual sub-modules, capturing contextual information at patch, object, and scene levels respectively, and build them into the voting and classification stages of VoteNet.

Pengdi Huang, Liqiang Lin, Kai Xu, Hui Huang, "Autonomous Outdoor Scanning via Online Topological and Geometric Path Optimization", IEEE Transactions on Intelligent Transportation Systems (TITS). [Paper | Code]

Unlike for indoor scenes where the scanning effort is mainly devoted to the discovery of boundary surfaces, scanning an open and unbounded area requires actively delimiting the extent of scanning region and dynamically planning a traverse path within that region. We formulate the planning of outdoor scanning through a discrete-continuous optimization of scanning paths.

Qiaoyun Wu, Kai Xu, Jun Wang, Mingliang Xu, Xiaoxi Gong, Dinesh Manocha, "Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization", ICRA 2021 (The IEEE Robotics and Automation Letters). [Paper | Code]

To enhance the cross-target and cross-scene generalization of target-driven visual navigation based on deep reinforcement learning (RL), we introduce an information-theoretic regularization term into the RL objective. The regularization maximizes the mutual information between navigation actions and visual observation transforms of an agent.

Qiaoyun Wu, Xiaoxi Gong, Kai Xu, Dinesh Manocha, Jingxuan Dong, Jun Wang, "Towards Target-driven Visual Navigation in Indoor Scenes via Generative Imitation Learning", The IEEE Robotics and Automation Letters (RAL). [Paper| Code]

A target-driven, mapless visual navigation method. The agent conceives the next observation before making an action decision, achieved by learning a variational generative module from expert demonstrations. It also predicts static collision in advance, as an auxiliary task to improve safety during navigation.

Hang Zhao, Qijin She, Chenyang Zhu, Yin Yang, Kai Xu*, "Online 3D Bin Packing with Constrained Deep Reinforcement Learning", AAAI 2021. [Paper | Code] (Hang and Qijin are co-first authors) (NOTE: This method is patent protected. Contact me for commercial use.)

We solve the Online 3D Bin Packing problem, a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP). In Online 3D-BPP, the agent has limited information about the items to be packed into the bin, and an item must be packed immediately after its arrival without buffering or readjusting. The item's placement also subjects to the constraints of collision avoidance and physical stability. We formulate this online 3D-BPP as a constrained Markov decision process and solve it with Constrained Deep Reinforcement Learning. Our method handles well lookahead items and varying item orientations. A user study suggests that our method attains a HUMAN-LEVEL performance.

Yifei Shi, Junwen Huang, Hongjia Zhang, Xin Xu, Szymon Rusinkiewicz, Kai Xu*, "SymmetryNet: Learning to Predict Reflectional and Rotational Symmetries of 3D Shapes from Single-View RGB-D Images", ACM Transactions on Graphics (SIGGRAPH Asia 2020). [Paper | Project page | Code & data]

SymmeryNet is an end-to-end trainable deep neural network able to predict both reflectional and rotational symmetries of 3D objects present in an input RGB-D image. The key to the success of SymmeryNet is the multi-task learning for the prediction of not only symmetry parameters but also symmetry correspondences. This greatly alleviates overfitting.

Xiaogang Wang, Yuelang Xu, Kai Xu, Andrea Tagliasacchi, Bin Zhou, Ali Mahdavi-Amiri, and Hao Zhang, "PIE-NET: Parametric Inference of Point Cloud Edges", NeurIPS 2020. [Paper | Code]

The first deep model to extract parametric curves from point clouds, trained on the ABC dataset.

Abstract: We introduce an end-to-end learnable technique to robustly identify feature edges in 3D point cloud data. We represent these edges as a collection of parametric curves (i.e.,lines, circles, and B-splines). Accordingly, our deep neural network, coined PIE-NET, is trained for parametric inference of edges. The network relies on a "region proposal" architecture, where a first module proposes an over-complete collection of edge and corner points, and a second module ranks each proposal ...

Min Liu, Zherong Pan, Kai Xu*, Kanishka Ganguly, Dinesh Manocha, "Deep Differentiable Grasp Planner for High-DOF Grippers", Robotics: Science and Systems (RSS 2020). [Paper | Code]

A differentiable and generalized grasp quality metric for learning-based high-quality grasp planning.

Abstract: We present an end-to-end algorithm for training deep neural networks to grasp novel objects. Our algorithm builds all the essential components of a grasping system using a forward-backward automatic differentiation approach, including the forward kinematics of the gripper, the collision between the gripper and the target object, and the metric of grasp poses. In particular, we show that a generalized Q1 grasp metric is defined and differentiable for inexact grasps generated by a neural network ...

Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, Kai Xu*,Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation", CVPR 2020. [Paper | Code] (Jiazhao and Chenyang are co-first authors)

Online semantic scene segmentation with high speed (12 FPS) and SOTA accuracy (avg. IoU=0.72 measured w.r.t. per-frame ground-truth image labels). We have also submitted our results to the ScanNet benchmark, demonstrating an avg. IoU of 0.63 on the leaderboard. Note, however, the number was obtained by spatially transferring the point-wise labels of our online recontructed point clouds to the pre-reconstructed point clouds of the benchmark scenes. Such spatial transfer loses accuracy. Therefore, this is not a perfect way of evaluating online segmentation methods. Nevertheless, ours is still the most accurate among all the online methods on the list.

Dengsheng Chen, Jun Li, Zheng Wang, Kai Xu*, "Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation", CVPR 2020. [Paper | Code] (Dengsheng and Jun are co-first authors)

Estimating category-level 6D pose and size via learning a canonical shape embedding space with deep generative model.

Abstract: We present a novel approach to category-level 6D object pose and size estimation. To tackle intra-class shape variation, we learn canonical shape space (CASS), a unified representation for a large variety of instances of a certain object category. In particular, CASS is modeled as the latent space of a deep generative model of canonical 3D shapes with normalized pose and size. We train a VAE ...

Chenyang Zhu, Kai Xu*, Siddhartha Chaudhuri, Li Yi, Leonidas J. Guibas, Hao Zhang, "AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss", CVPR 2020, Oral presentation. [Paper | Code]

We achieve set-adaptive co-segmentation with weakly supervised online learning.

Abstract: We introduce AdaSeg, a deep neural network architecture for adaptive co-segmentation of a set of 3D shapes represented as point clouds. Differently from the familiar single-instance segmentation problem, co-segmentation is intrinsically contextual: how a shape is segmented can vary depending on the set it is in. Our network features an adaptive learning module to produce a consistent ...

Rundi Wu, Yixin Zhuang, Kai Xu, Hao Zhang, Baoquan Chen, "PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes", CVPR 2020. [Paper | Code]

A part-aware shape generation model based on sequence-to-sequence autoencoder.

Abstract: We introduce PQ-NET, a deep neural network which represents and generates 3D shapes via sequential part assembly. The input to our network is a 3D shape segmented into parts, where each part is first encoded into a feature representation using a part autoencoder. The core component of PQ-NET is a sequence-to-sequence or Seq2Seq autoencoder which encodes a sequence of part features into a latent vector of fixed size, and the decoder reconstructs the 3D shape, one part at a time, resulting in a sequential assembly. The latent space formed by the Seq2Seq encoder encodes both part structure and fine part geometry. The decoder can be adapted to perform several generative tasks ...

Qian Xie, Yu-Kun Lai, Jing Wu, Zhoutao Wang, Yiming Zhang, Kai Xu, Jun Wang, "MLCVNet: Multi-Level Context VoteNet for 3D Object Detection", CVPR 2020. [Paper | Code ]

Boosting object detection accuracy of VoteNet by encoding multi-level contextual inforamtion.

Abstract: ... We propose Multi-Level Context VoteNet (MLCVNet) to recognize 3D objects correlatively, building on the state-of-the-art VoteNet. We introduce three context modules into the voting and classifying stages of VoteNet to encode contextual information at different levels. Specifically, a Patch-to-Patch Context (PPC) module is employed to capture contextual information between the point patches. patches, before voting for their corresponding object centroid points ...

Siddhartha Chaudhuri, Daniel Ritchie, Jiajun Wu, Kai Xu*, Hao Zhang, "Learning Generative Models of 3D Structures", Computer Graphics Forum, Eurographics 2020 State-of-The-Art Report (EG STAR). [Paper]

Historical work and recent progress on learning structure-aware generative models of 3D shapes and scenes.

Abstract: ... To allow users to edit and manipulate the synthesized 3D content to achieve their goals, the generative model should also be structure-aware: it should express 3D shapes and scenes using abstractions that allow manipulation of their high-level structure ...

Chengjie Niu, Yang Yu, Zhenwei Bian, Jun Li, Kai Xu*, "Weakly Supervised Part‐wise 3D Shape Reconstruction from Single‐View RGB Images", Computer Graphics Forum, (PG 2020). [Paper]

Self-taught learning of a deep neural network for single-view reconstruction of 3D point cloud represented in parts.

Abstract: In order for the deep learning models to truly understand the 2D images for 3D geometry recovery, we argue that single‐view reconstruction should be learned in a part‐aware and weakly supervised manner. Such models lead to more profound interpretation of 2D images in which part‐based parsing and assembling are involved ...

Jun Li, Chengjie Niu, Kai Xu*, "Learning Part Generation and Assembly for Structure-aware Shape Synthesis", AAAI 2020. [Paper]

A part-aware generative model of 3D shapes composed of several part generators and one part assembler.

Abstract: Learning deep generative models for 3D shape synthesis is largely limited by the difficulty of generating plausible shapes with correct topology and reasonable geometry. Indeed, learning the distribution of plausible 3D shapes seems a daunting task for most existing holistic shape representation, given the significant topological variations of 3D objects even within the same shape category. Enlightened by the common view that 3D shape structure is characterized as part composition and placement, we propose to model 3D shape variations with a part-aware deep generative network which we call PAGENet. The network is composed of an array of per-part VAE-GANs ...

Qiaoyun Wu, Dinesh Manocha, Jun Wang, Kai Xu*, "NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations", AAAI 2020. [Paper | Code]

We show that predicting / imagining the next observations the agent expects to see improves the performance of the visual navigation significantly, leading to the state-of-the-art cross-target and cross-scene generalization.

Abstract: We propose improving the cross-target and cross-scene generalization of visual navigation through learning an agentthat is guided by conceiving the next observations it expects to see. This is achieved by learning a variational Bayesian model, called NeoNav, which generates the next expected observations (NEO) conditioned on the current observations ofthe agent and the target view ...

Min Liu, Zherong Pan, Kai Xu*, Dinesh Manocha, "New Formulation of Mixed-Integer Conic Programming for Globally Optimal Grasp Planning", IROS 2020 (The IEEE Robotics and Automation Letters (RAL)) [Paper]

A formulation of globally optimal gripper posing based on mixed-integer conic programming.

Abstract: We present a two-level branch-and-bound (BB) algorithm to compute the globally optimal gripper pose that maximizes a grasp metric. Our method can take the gripper's kinematics feasibility into consideration to ensure that a given gripper can reach the set of grasp points without collisions or predict infeasibility with finite-time termination when no pose exists for a given set of grasp points. Our main technical contribution is a novel mixed-integer conic programming (MICP) formulation for the inverse kinematics of the gripper that uses a small number of binary variables and tightened constraints ...

Siyan Dong, Kai Xu*, Qiang Zhou, Andrea Tagliasacchi, Shiqing Xin, Matthias Nießner, Baoquan Chen*, "Multi-Robot Collaborative Dense Scene Reconstruction," ACM Transactions on Graphics (SIGGRAPH 2019), 38(4). [Paper | Project page | ROS package]

Min Liu, Zherong Pan, Kai Xu*, Kanishka Ganguly, Dinesh Manocha, "Generating Grasp Poses for a High-DOF Gripper Using Neural Networks," IROS 2019. [Paper]

Lintao Zheng, Chenyang Zhu, Jiazhao Zhang, Hang Zhao, Hui Huang, Matthias Niessner, Kai Xu*, "Active Scene Understanding via Online Semantic Reconstruction," Computer Graphics Forum (Pacific Graphics 2019). [Paper]

Maciej Halber, Yifei Shi, Kai Xu, Thomas Funkhouser, "RESCAN: Inductive Instance Segmentation for Indoor RGBD Scans," ICCV 2019. [Paper]

Yifei Shi, Angel Chang, Manolis Savva, Zhelun Wu, Kai Xu*, "Hierarchy Denoising Recursive Autoencoders for 3D Scene Layout Prediction," CVPR 2019. [Paper | Project page | Code]

Xiaogang Wang, Yahao Shi, Bin Zhou, Xiaowu Chen, Qinping Zhao and Kai Xu*, "Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapes," CVPR 2019, Oral presentation. [Paper | Project page | Code | Benchmark]

Fenggen Yu, Kun Liu, Yan Zhang, Chenyang Zhu, Kai Xu*, "PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation," CVPR 2019. [Paper | Project page | Code | PartNet-Symh Dataset]

Manyi Li, Akshay Gadi Patil, Kai Xu*, Siddhartha Chaudhuri, Owais Khan, Ariel Shamir, Changhe Tu, Baoquan Chen, Daniel Cohen-Or, and Hao Zhang, "GRAINS: Generative Recursive Autoencoders for INdoor Scenes," ACM Transactions on Graphics (To be presented at SIGGRAPH 2019). [Paper | Project page | Code]

Min Liu, Yifei Shi, Lintao Zheng, Kai Xu*, Hui Huang and Dinesh Manocha, "Recurrent 3D Attentional Networks for End-to-End Active Object Recognition," CVM 2019. [Paper]

Chenyang Zhu, Kai Xu*, Siddhartha Chaudhuri, Renjiao Yi and Hao Zhang, "SCORES: Shape Composition with Recursive Substructure Priors," ACM Transactions on Graphics (SIGGRAPH Asia 2018), 37(6). (* corresponding author) [Paper | Project page | Code & data]

Xiaogang Wang, Bin Zhou, Haiyue Fang, Xiaowu Chen, Qinping Zhao and Kai Xu*, "Learning to Group and Label Fine-Grained Shape Components," ACM Transactions on Graphics (SIGGRAPH Asia 2018), 37(6). (* corresponding author) [Paper | Slides | Project page | Code & benchmark]

Yifei Shi, Kai Xu*, Matthias Niessner, Szymon Rusinkiewicz and Thomas Funkhouser, "PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction," ECCV 2018, Oral presentation (Acceptance rate: 1.9%). (* corresponding author) [Paper, 10M | Supplemental materia, 5M | Project page | Slides, 3M | Code & benchmark]

Chengjie Niu, Jun Li and Kai Xu*, "Im2Struct: Recovering 3D Shape Structure from a Single RGB Image," CVPR 2018. (* corresponding author) [Paper, 4.9M | Poster | Code ( NEW! Training code is now included)]

Fenggen Yu, Zhang Yan, Kai Xu*, Ali Mahdavi-Amiri and Hao Zhang, "Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines," ACM Transactions on Graphics (to be presented at SIGGRAPH 2018), 37(2). (* corresponding author) [Paper, 12M | Slides, 4M | Project page | Code & data | Online test website] Awarded the Graphics Replicability Stamp

Ligang Liu, Xi Xia, Han Sun, Hui Huang and Kai Xu*, "Object-Aware Guidance for Autonomous Scene Reconstruction," ACM Transactions on Graphics (SIGGRAPH 2018), 37(4). (* corresponding author) [Paper, 25M | Slides, 7M | Project page | Code | Benchmark]

Ke Xie, Hao Yang, Shengqiu Huang, Dani Lischinski, Marc Christie, Kai Xu, Minglun Gong, Daniel Cohen-Or and Hui Huang, "Creating and Chaining Camera Moves for Quadrotor Videography," ACM Transactions on Graphics (SIGGRAPH 2018), 37(4). [Paper, 40M | Project page]

Jian Liu, Shiqing Xin, Zengfu Gao, Kai Xu, Changhe Tu and Baoquan Chen, "Caging Loops in Shape Embedding Space: Theory and Computation," International Conference on Robotics and Automation (ICRA 2018). [Paper, 17M | Poster | Code]

Songle Chen, Lintao Zheng, Yan Zhang, Zhixin Sun and Kai Xu*, "VERAM: View-Enhanced Recurrent Attention Model for 3D Shape Classification," IEEE Transactions on Visualization and Computer Graphics. (* corresponding author) [Paper | Project | Code]

Biao Leng, Cheng Zhang, Xiaocheng Zhou, Cheng Xu, Kai Xu*, "Learning Discriminative 3D Shape Representations by View Discerning Networks," IEEE Transactions on Visualization and Computer Graphics. (* corresponding author) [Paper | Project | Code]

Qiaoyun Wu, Kai Xu and Jun Wang, "Constructing 3D CSG Models from 3D Raw Point Clouds," Computer Graphics Forum (SGP 2018). [Paper, 14.5M]

Yawei Zhao, Kai Xu, Xinwang Liu, En Zhu, Xinzhong Zhu and Jianping Yin, "Triangle Lasso for Simultaneous Clustering and Optimization in Graph Datasets," IEEE Transactions on Knowledge and Data Engineering. to appear [Paper]

Qi She, Yuan Gao, Kai Xu and Rosa H.M. Chan, "Reduced-Rank Linear Dynamical System," AAAI Conference on Artificial Intelligence (AAAI 2018). [Paper, 0.7M]

Kai Xu*, Lintao Zheng*, Zihao Yan, Guohang Yan, Eugene Zhang, Matthias Niessner, Oliver Deussen, Daniel Cohen-Or and Hui Huang, "Autonomous Reconstruction of Unknown Indoor Scenes Guided by Time-varying Tensor Fields," ACM Transactions on Graphics (SIGGRAPH Asia 2017), 36(6). (* co-first authors). [Paper, 16M | Slides, 4M | Project page | Code release on ROS]

Jun Li, Kai Xu*, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang and Leonidas Guibas, "GRASS: Generative Recursive Autoencoders for Shape Structures," ACM Transactions on Graphics (SIGGRAPH 2017), 36(4). (* corresponding author). [Paper, 10M | Slides, 3.9M | Project page | Poster | Code & data] Featured ACM SIGGRAPH Press Release

Chenyang Zhu, Renjiao Yi, Wallace Lira, Ibraheem Alhashim, Kai Xu and Hao Zhang, "Deformation-Driven Shape Correspondence via Shape Recognition," ACM Transactions on Graphics (SIGGRAPH 2017), 36(4). [Paper, 31M | Project page | Code & data]

Oussama Remil, Qian Xie, Xingyu Xie, Kai Xu and Jun Wang, "Data-Driven Sparse Priors of 3D Shapes," Computer Graphics Forum (Pacific Graphics 2017). 36(7):63-72. [PDF, 12.8M]

Oussama Remil, Qian Xie, Xingyu Xie, Kai Xu and Jun Wang, "Surface Reconstruction with Data-driven Exemplar Priors," Computer-Aided Design. 88(C): 31-41. [PDF, 6M]

Kai Xu, Vladimir G Kim, Qixing Huang, Niloy Mitra, Evangelos Kalogerakis, "Data-Driven Shape Analysis and Processing," SIGGRAPH Asia 2016 Course. [Course note, 12.5M]

Kai Xu, Yifei Shi, Lintao Zheng, Junyu Zhang, Min Liu, Hui Huang, Hao Su, Daniel Cohen-Or and Baoquan Chen, "3D Attention-Driven Depth Acquisition for Object Identification," ACM Transactions on Graphics (SIGGRAPH Asia 2016), 35(6). [PDF, 12.5M | PPT, 4.5M | Project page | Code]

Jun Wang and Kai Xu, "Shape Detection from Raw LiDAR Data with Subspace Modeling," IEEE Transactions on Visualization and Computer Graphics (TVCG). [PDF, 3.1M]

Xuekun Guo, Juncong Lin, Kai Xu, Siddhartha Chaudhuri and Xiaogang Jin, "CustomCut: On-demand Extraction of Customized 3D Parts with 2D Sketches," Computer Graphics Forum (SGP 2016), 35(5). [PDF, 11.3M]

Hao Li, Guowei Wan, Honghua Li, Andrei Sharf, Kai Xu and Baoquan Chen, "Mobility Fitting using 4D RANSAC," Computer Graphics Forum (SGP 2016), 35(5). [PDF, 11.4M | Project page]

Qing Yuan, Guiqing Li, Kai Xu, Xudong Chen and Hui Huang, "Space-Time Co-Segmentation of Articulated Point Cloud Sequences," Computer Graphics Forum (Eurographics 2016), 35(2). [PDF, 31M | Project page]

Yifei Shi, Pinxin Long, Kai Xu*, Hui Huang and Yueshan Xiong, "Data-Driven Contextual Modeling for 3D Scene Understanding," Computers and Graphics, 55: 55-67. [PDF, 4.9M]

Bo Wu, Kai Xu*, Yang Zhou, Yueshan Xiong, Hui Huang, "Skeleton-guided 3D shape distance field metamorphosis, Graphical Models, 85: 37-45. [PDF, 15M | Project page]

Yueqing Wang, Zhige Xie, Kai Xu, Yong Dou and Yuanwu Lei, "An Efficient and Effective Convolutional Auto-Encoder Extreme Learning Machine Network for 3D Feature Learning," Neurocomputing, 174: 988-998. [PDF, 2.7M]

Kai Xu, Hui Huang, Yifei Shi, Hao Li, Pinxin Long, Jianong Caichen, Wei Sun and Baoquan Chen, "Autoscanning for Coupled Scene Reconstruction and Proactive Object Analysis," ACM Transactions on Graphics (SIGGRAPH Asia 2015), 34(6). [PDF, 18.7M | PPT, 2.9M | Project page | Code]

Ibraheem Alhashim, Kai Xu, Yixin Zhuang, Junjie Cao, Patricio Simari and Hao Zhang, "Deformation-Driven Topology-Varying 3D Shape Correspondence," ACM Transactions on Graphics (SIGGRAPH Asia 2015), 34(6). [PDF | Project page | Code]

Kai Xu, Vladimir G. Kim, Qixing Huang, Evangelos Kalogerakis, "Data-Driven Shape Analysis and Processing," Computer Graphics Forum. [PDF, 12.5M | Wikipage]

Zhige Xie, Kai Xu*, Wen Shan, Ligang Liu, Yueshan Xiong and Hui Huang, "Projective Feature Learning for 3D Shapes with Multi-View Depth Images," Computer Graphics Forum (Pacific Graphics 2015). [PDF, 7M | Project page | Code]

Qian Zheng, Zhuming Hao, Hui Huang, Kai Xu, Hao Zhang, Daniel Cohen-Or and Baoquan Chen, "Skeleton-Intrinsic Symmetrization of Shapes," Computer Graphics Forum (Special Issue of Eurographics 2015), 37(4). [PDF, 49M | Project page]

Kai Xu, Rui Ma, Hao Zhang, Chenyang Zhu, Ariel Shamir,  Daniel Cohen-Or and Hui Huang, "Organizing Heterogeneous Scene Collections through Contextual Focal Points," ACM Transactions on Graphics (SIGGRAPH 2014), 33(4). [PDF, 13M | Project page | Code]

Ibraheem Alhashim, Honghua Li, Kai Xu, Junjie Cao, Rui Ma and Hao Zhang, "Topology-Varying 3D Shape Creation via Structural Blending," ACM Transactions on Graphics (SIGGRAPH 2014), 33(4). [PDF, 16.0M | Project page | Code]

Zhige Xie, Kai Xu*, Ligang Liu and Yueshan Xiong, "3D Shape Segmentation and Labeling via Extreme Learning Machine," Computer Graphics Forum (SGP 2014). [PDF, 3.3M | Code]

Xuekun Guo, Juncong Lin, Kai Xu and Xiaogang Jin, "Creature Grammar for Creative Modeling of 3D Monsters," Graphical Models (GMP 2014). [PDF, 8.1M]

Zhige Xie, Yueshan Xiong, Kai Xu*, "AB3D: Action-Based 3D Descriptor for Shape Analysis," The Visual Computer Journal (CGI 2014). [PDF, 3.7M | Erratum]

Jun Li, Weiwei Xu, Zhiquan Cheng, Kai Xu*, and Reinhard Klein, "Lightweight Wrinkle Synthesis for 3D Facial Modeling and Animation," Computer-Aided Design (SPM 2014). [PDF, 3.9M].

Kai Lu, Yi Zhang, Kai Xu, Yinghui Gao and Richard Wilson, "Approximate Maximum Common Sub-graph Isomorphism Based on Discrete-Time Quantum Walk," ICPR 2014. [PDF, 650K]

Jun Wang, Kai Xu, Ligang Liu, Junjie Cao, Shengjun Liu, Zeyun Yu, and Xianfeng Gu, "Consolidation of Low-quality Point Clouds from Outdoor Scenes," Computer Graphics Forum (SGP 2013). [PDF, 30M]

Xiaohua Xie, Kai Xu, Niloy Mitra, Daniel Cohen-Or, Wenyong Gong, Qi Su, Baoquan Chen, "Sketch-to-Design: Context-based Part Assembly," Computer Graphics Forum, 32(8): 233-245. [PDF, 9M | Project page]

Hao Zhang, Kai Xu*, Wei Jiang, Jinjie Lin, Daniel Cohen-Or and Baoquan Chen, "Layered Analysis of Irregular Facades via Symmetry Maximization," ACM Transactions on Graphics (SIGGRAPH 2013), 32(4). (* corresponding author) [PDF, 33M | MOV. 70M | Project page | Code | Data]

Oliver van Kaick, Kai Xu, Hao Zhang, Yanzhen Wang, Shuyang Sun, Ariel Shamir and Daniel Cohen-Or, "Co-Hierarchical Analysis of Shape Structures," ACM Transactions on Graphics (SIGGRAPH 2013), 32(4). [PDF, 17M | Project page]

Wei Jiang, Kai Xu*, Zhiquan Cheng, and Hao Zhang, "Skeleton-Based Intrinsic Symmetry Detection on Point Clouds," Graphical Models, 75(4):177-188. [PDF, 5.6M]

Wei Jiang, Kai Xu*, Zhiquan Cheng, Ralph Martin, and Gang Dang, "Curve Skeleton Extraction by Coupled Graph Contraction and Surface Clustering," Graphical Models, 75(3): 137-148. (A previous version appeared at CVM 2012) [PDF, 2.4M]

In this paper, we present a practical algorithm to extract a curve skeleton of a 3D shape. The core of our algorithm comprises coupled processes of graph contraction and surface clustering. Given a 3D shape represented by a triangular mesh, we first construct an initial skeleton graph by directly copying the connectivity and geometry information from the input mesh. Graph contraction and surface clustering are then performed iteratively. The former merges certain graph nodes based on computation of an approximate centroidal Voronoi diagram, seeded by subsampling the graph nodes from the previous iteration. Meanwhile, a coupled surface clustering process serves to regularize the graph contraction ... It can also handle point cloud data if we first build an initial skeleton graph based on k-nearest neighbors ...

Kai Xu, Hao Zhang, Wei Jiang, Ramsay Dyer, Zhiquan Cheng, Ligang Liu, and Baoquan Chen, "Multi-Scale Partial Intrinsic Symmetry Detection," ACM Transactions on Graphics (SIGGRAPH Asia 2012), 31(6). [PDF, 15.6M | PPTX, 16.0M | Project page | Data]

We present an algorithm for multi-scale partial intrinsic symmetry detection over 2D and 3D shapes, where the scale of a symmetric region is defined by intrinsic distances between symmetric points over the region. To identify prominent symmetric regions which overlap and vary in form and scale, we decouple scale extraction and symmetry extraction by performing two levels of clustering. First, significant symmetry scales are identified by clustering sample point pairs from an input shape. Since different point pairs can share a common point, shape regions covered by points in different scale ...

Kai Xu, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen, "Fit and Diverse: Set Evolution for Inspiring 3D Shape Galleries," ACM Transactions on Graphics (SIGGRAPH 2012), 31(4). [PDF, 15.8M | MOV, 51.7M | PPTX, 22.9M | Project page | Data]

We introduce set evolution as a means for creative 3D shape modeling, where an initial population of 3D models is evolved to produce generations of novel shapes. Part of the evolving set is presented to a user as a shape gallery to offer modeling suggestions. User preferences define the fitness for the evolution so that over time, the shape population will mainly consist of individuals with good fitness. However, to inspire the user's creativity, we must also keep the evolving set diverse. Hence the evolution is "fit and diverse", drawing motivation from evolution theory. We introduce a novel part crossover operator which works at the finer-level part structures of the shapes ...

Yanzhen Wang, Yueshan Xiong, Kai Xu, and Dong Liu, "vKASS: A Surgical Procedure Simulation System for Arthroscopic Anterior Cruciate Ligament Reconstruction" Computer Animation and Virtual World. 24(1): 25-41. [PDF, 2.2M]

Arthroscopic surgeries, which are widely used for anterior cruciate ligament (ACL) reconstruction, not only require advanced hand鈥揺ye coordination but also involve complicated surgical procedure, necessitating simulation-based training for surgeons. This paper describes a surgical procedure simulation system for the training of arthroscopic ACL reconstruction. Different from existing simulation-based training systems for basic surgical skills, this system provides a complete simulation for the entire procedure of arthroscopic ACL reconstruction, involving operations such as puncturing, probing, incision, and drilling. In this system, we employ a linear elastic finite element method and position-based dynamics for deformable modeling ...

Kai Xu, Hanlin Zheng, Hao Zhang, Daniel Cohen-Or, Ligang Liu, and Yueshan Xiong, "Photo-Inspired Model-Driven 3D Object Modeling," ACM Transactions on Graphics (SIGGRAPH 2011), 30(4). [PDF, 12.6M | MOV, 33.9M | PPTX, 14.3M | Project page]

Yanzhen Wang, Kai Xu, Jun Li, Hao Zhang, Ariel Shamir, Ligang Liu, Zhi-Quan Cheng, and Yueshan Xiong, "Symmetry Hierarchy of Man-Made Objects," Computer Graphics Forum (Special Issue of Eurographics 2011), 30(2): 287-296. [PDF, 12M | MOV, 28M | Project page]

We introduce symmetry hierarchy of man-made objects, a high-level structural representation of a 3D model providing a symmetry-induced, hierarchical organization of the model's constituent parts. We show that symmetry hierarchy naturally implies a hierarchical segmentation that is more meaningful than those produced by local geometric considerations. We also develop an application of symmetry hierarchies for structural shape editing.

Kai Xu, Honghua Li, Hao Zhang, Daniel Cohen-Or, Yueshan Xiong, and Zhi-Quan Cheng, "Style-Content Separation by Anisotropic Part Scales," ACM Transactions on Graphics (SIGGRAPH Aisa 2010), 29(5). [PDF, 9.8M | Project page]

We perform co-analysis of a set of man-made 3D objects to allow the creation of novel instances derived from the set. We analyze the objects at the part level and treat the anisotropic part scales as a shape style. The co-analysis then allows style transfer to synthesize new objects. The key to co-analysis is part correspondence, where a major challenge is the handling of large style variations and diverse geometric content in the shape set. We propose style-content separation as a means to address this challenge. Specifically, we define a correspondence-free style signature for style clustering. We show that confining analysis to within a style cluster facilitates tasks such as ... 

Z.-Q. Cheng, W. Jiang, G. Dang, R. Martin, J. Li, H. Li, Y. Chen, Y. Wang, B. Li, K. Xu, S. Jin, "Non-rigid Registration in 3D Implicit Vector Space," In Shape Modeling International 2010, Aix-en-Provence, France, 2010. [PDF, 4.3M]

We present an implicit approach for pair-wise non-rigid registration of moving and deforming objects. Shapes of interest are implicitly embedded in the 3D implicit vector space. In this implicit embedding space, registration is performed using a global-to-local framework. Firstly, a non-linear optimization functional defined on the vector distance function is used to find the global alignment between shapes. Secondly, an incremental cubic B-spline free form deformation is used to recover the non-rigid transformation parameters ...

Kai Xu, Hao Zhang, Andrea Tagliasacchi, Ligang Liu, Guo Li, Min Meng, and Yueshan Xiong, "Partial Intrinsic Reflectional Symmetry of 3D Shapes," ACM Transactions on Graphics (SIGGRAPH Aisa 2009), 28(5). [PDF, 15M | Video, 37M | Project page]

While many 3D objects around us exhibit various forms of global symmetries, prominent intrinsic symmetries which exist only on parts of an object are also well recognized. Such partial symmetries are often seen as more natural compared to a global one, especially on a composite shape. In this paper, we introduce algorithms to extract and utilize partial intrinsic reflectional symmetries (PIRS) of a 3D shape. Given a closed 2-manifold mesh, we develop a voting scheme to obtain an intrinsic reflectional symmetry axis (IRSA) transform ...

Kai Xu, Daniel Cohen-Or, Tao Ju, Ligang Liu, Hao Zhang, Shizhe Zhou, and Yueshan Xiong, "Feature-Aligned Shape Texturing," ACM Transactions on Graphics (SIGGRAPH Aisa 2009), 28(5). [PDF, 20.1M | Video, 31M | Project page | Code]

We present an implicit approach for pair-wise non-rigid registration of moving and deforming objects. Shapes of interest are implicitly embedded in the 3D implicit vector space. In this implicit embedding space, registration is performed using a global-to-local framework. Firstly, a non-linear optimization functional defined on the vector distance function is used to find the global alignment between shapes. Secondly, an incremental cubic B-spline free form deformation is used to recover the non-rigid transformation parameters ...

Kai Xu, Hao Zhang, Daniel Cohen-Or, and Yueshan Xiong, "Dynamic Harmonic Fields for Surface Processing," Computers and Graphics (Special Issue of Shape Modeling International 2009), 33(3): 391-398. [PDF, 0.6M | Video, 49.2M | Source code]

We propose a method for fast updating of harmonic fields defined on polygonal meshes, enabling real-time insertion and deletion of constraints. Our approach utilizes the penalty method to enforce constraints in harmonic field computation. It maintains the symmetry of the Laplacian system and takes advantage of fast multi-rank updating and downdating of Cholesky factorization, achieving both speed and numerical stability. We demonstrate how the interactivity induced by fast harmonic field update can be utilized in several applications ...

Kai Xu, Zhi-Quan Cheng, Yanzhen Wang, Yueshan Xiong, and Hao Zhang, "Quality Encoding for Tetrahedral Mesh Optimization," Computers and Graphics (Special Issue of Shape Modeling International 2009), 33(3): 250-261. [PDF, 1M]

We define quality differential coordinates (QDC) for per-vertex encoding of the quality of a tetrahedral mesh. QDC measures the deviation of a mesh vertex from a position which maximizes the combined quality of the tetrahedra incident at that vertex. Our formulation allows the incorporation of element quality metrics into QDC construction to penalize badly shaped and inverted tetrahedra. We develop an algorithm for tetrahedral mesh optimization through energy minimization driven by QDC ...

Yanzhen Wang, Kai Xu, Yueshan Xiong, and Zhi-Quan Cheng, "2D Shape Deformation Based on As-Rigid-As-Possible Squares Matching," Computer Animation and Virtual World (Special Issue of CASA 2008), 19(3-4): 411-420. [PDF, 5.8M]

In this paper, we propose a fast and stable method for 2D shape deformation based on rigid square matching. Our method utilizes uniform quadrangular control meshes for 2D shapes and tries to maintain the rigidity of each square in the control mesh during user mani-pulation. A rigid shape matching method is performed to find an optimal pure rotational transformation for each square in the control mesh. An iterative solver is proposed to com-pute the final deformation result for the entire control mesh by minimizing the difference between ...

Kai Xu, Yanzhen Wang, Yueshan Xiong, and Zhi-Quan Cheng, "Interactive Shape Manipulation Based on Space Deformation with Harmonic-Guided Clustering," In: Proc. of International Conference on Computer Animation and Social Agent, 2008. [PDF, 0.3M]

We present an efficient and effective deformation algorithm for interactive shape manipulation. To obtain the advantages of both surface and space-based deformation, we propose to maximally incorporate surface geometry information into space deformation framework while preventing the dependence on surface representation. Our deformation model significantly reduces the problem size through sampling the shape surface and ...

Zhi-Quan Cheng, Yanzhen Wang, Bao Li, Kai Xu, Gang Dang, and Shiyao Jin, "A Survey of Methods for Moving Least Squares Surfaces," In: Proc. of IEEE/Eurographics Symposium on Point Based Graphics 2008, Los Angeles, USA, 2008. [PDF, 2.2M]

Moving least squares (MLS) surfaces representation directly defines smooth surfaces from point cloud data, on which the differential geometric properties of point set can be conveniently estimated. Nowadays, the MLS surfaces have been widely applied in the processing and rendering of point-sampled models and increasingly adopted as the standard definition of point set surfaces. We classify the MLS surface algorithms into two types: projection MLS surfaces and implicit MLS surfaces, according to employing a stationary projection or a scalar field in their definitions ...