Numerical Misalignment between nyu_class and color_ply

[chester256]

Thanks for your work. I am trying to preprocess data like what you did in CVPR'18 (Pointwise Conv), but with different block sizes, strides. etc. However, there seem to be numerical misalignments between /scenenn/contrib/color_ply and /scenenn/contrib/nyu_class. Since I want to get points with raw RGB (raw object colors) and category-level semantic annotations, I have to combine scenes in color_ply and nyu_class. However, when I read them in PlyData, there seem to be numerical misalignments between their global xyz. Below is the code :

>> plydata = PlyData.read('scenenn_data/raw_data/005.ply') # nyu_class data
>> plydata.elements[0].properties
>> 
(PlyProperty('x', 'float'),
 PlyProperty('y', 'float'),
 PlyProperty('z', 'float'),
 PlyProperty('nx', 'float'),
 PlyProperty('ny', 'float'),
 PlyProperty('nz', 'float'),
 PlyProperty('label', 'uint'),
 PlyProperty('nyu_class', 'ushort'),
 PlyProperty('red', 'uchar'),
 PlyProperty('green', 'uchar'),
 PlyProperty('blue', 'uchar'))   # RGB here represents different colors for different classes
>> plydata = np.array(plydata.elements[0].data.tolist())
>> plydata[plydata[:, 0].argsort()] # First three columns represent xyz
>> 
array([[ -0.7512365 ,   2.58200002,   0.34838411, ..., 139.        ,
        177.        , 182.        ],
       [ -0.74452573,   2.58820009,   0.34803241, ..., 139.        ,
        177.        , 182.        ],
       [ -0.74421382,   2.58200002,   0.35398427, ..., 139.        ,
        177.        , 182.        ],
       ...,
       [  4.19533205,   3.95700002,  -0.56017441, ..., 139.        ,
        177.        , 182.        ],
       [  4.19554138,   3.94919991,  -0.55236465, ..., 139.        ,
        177.        , 182.        ],
       [  4.19653988,   3.95700002,  -0.55241698, ..., 139.        ,
        177.        , 182.        ]])
>> plydata2 = PlyData.read('005_color.ply')
>> plydata2.elements[0].properties
>>
(PlyProperty('x', 'float'),
 PlyProperty('y', 'float'),
 PlyProperty('z', 'float'),
 PlyProperty('nx', 'float'),
 PlyProperty('ny', 'float'),
 PlyProperty('nz', 'float'),
 PlyProperty('red', 'uchar'),
 PlyProperty('green', 'uchar'),
 PlyProperty('blue', 'uchar'),
 PlyProperty('alpha', 'uchar'))
>> plydata2 = np.array(plydata2.elements[0].data.tolist())
>> plydata2[plydata2[:, 0].argsort()]
array([[ -0.76844001,   2.58200002,   0.30858999, ..., 107.        ,
         73.        ,   0.        ],
       [ -0.76172   ,   2.58200002,   0.31455001, ..., 107.        ,
         73.        ,   0.        ],
       [ -0.76172   ,   2.59770012,   0.31626001, ..., 107.        ,
         73.        ,   0.        ],
       ...,
       [  4.22860003,   3.91799998,  -0.64453   , ..., 127.        ,
         81.        ,   0.        ],
       [  4.22870016,   3.94919991,  -0.54297   , ..., 121.        ,
         76.        ,   0.        ],
       [  4.22949982,   3.92580009,  -0.64453   , ..., 125.        ,
         84.        ,   0.        ]])
>> plydata[:, 0].max()
>>
4.196539878845215
>> plydata2[:, 0].max()
>>
4.229499816894531

It seems that positions (xyz) do not match between color_ply and nyu_class, in other words, there is no one-to-one correspondence relationship between points in color_ply and nyu_class, so that I cannot map the real object color in color_ply to nyu_class. Could you please help me with this? In addition, the numerical problem may caused by the python lib PlyData, if so, could you please provide the preprocess tools/scripts used in aforementioned CVPR'18 paper? Thank you very much.

[chester256]

Perhaps my statement was not clear. Let me re-state the problem. For instance, I have to crop the RGB info in 005_color.ply (/scenenn/contrib/color_ply), and concatenate the color info to 005.ply (/scenenn/contrib/nyu_class). Since the ply files do not store vertex indices info, so I have to match the points between 005_color.ply and 005.ply based on their xyz. They should share same xyz since they store same points with different attributes. However, the above code shows that the xyz is not shared between 005_color.ply and 005.ply. Could you please tell me how to map the color info in color_ply to nyu_class? Thank you very much.

BTW, nyu_class still lacks 20+ scenes. It will be great if you upload them. Thank you very much.

[songuke]

As I remember, the geometry mismatch between ply files are not an issue as the order of the vertices when converted from TSDF is not changed. It means, you can just read the geometry once, and get other data channels from other files if needed.

Please see https://github.com/hkust-vgd/scenenn/issues/9#issuecomment-528227631 for the 76 scenes data. Thanks.

[chester256]

As I remember, the geometry mismatch between ply files are not an issue as the order of the vertices when converted from TSDF is not changed. It means, you can just read the geometry once, and get other data channels from other files if needed.

Please see #9 (comment) for the 76 scenes data. Thanks.

Thanks for your reply. I just tried on 005.ply and it really works. Thank you very much.