search

huangjh-pub / multibody-sync

[CVPR'21 Oral] MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
59 stars 5 forks source link

Custom dataset #2

Closed xuanzhezhao closed 3 years ago

xuanzhezhao commented 3 years ago

Hi, thanks for your work. I notice that the dataset you provided(like data/000000.npz) contains three parts: pc, segm and trans. Could you please explain more details about the generation of "trans"? And how to generate my custom dataset if I want to test multibody-sync on it? Thanks a lot!

Best, Xuanzhe

heiwang1997 commented 3 years ago

Hi @xuanzhezhao, thanks for your interest in our work!

In the case where you just want to test with our pre-trained weights, you don't need 'segm' or 'trans'. If you check out the 'Usage' part in the readme, you just need to pass in a (1, K, N, 3) tensor where K is the number of point clouds and N is the number of points in each point cloud.

If you want to train using your own data, then 'trans' should be a dictionary containing the following keys: 'cam', 1, 2, ..., NUM_PARTS. Each key should be a list of size K, so that trans[i][k] is the 4x4 matrix denoting the transformation of part i (or the camera) in view k. You can also checkout these lines for the actual usage of the transformation matrices.

Best wishes, Jiahui

xuanzhezhao commented 3 years ago

Thank you very much, It makes sense to me.