THuman2 Labels seems to be wrong

[fatbao55]

Hi, thanks for the great work!

I could successfully visualise the labels for Close-Di and Dc datasets. However, all the the labels in THuman2.npz seems to be wrong. I have included the code for extracting the vertices of a selected category and visualisation of the extracted vertices for THuman2 samples.

My code for vis: labels = np.load('/mnt/nvme1n1/Datasets/CloSe/THumans2.npz') scan_path = '/mnt/nvme1n1/Datasets/thuman2/scans' fid = '0035' obj_path = f'{scan_path}/{fid}/{fid}.obj' scan_labels = labels[fid] mesh = trimesh.load(obj_path, process=False, maintain_order=True) verts = mesh.vertices sel_verts = verts[scan_labels == 8] point_cloud = trimesh.points.PointCloud(sel_verts) point_cloud.export('debug.ply') Extracted vertices:

Do you have any advice?

[Bozcomlekci]

Greetings,

You are not extracting and using vertex colors at all for your point cloud. Use something similar to:

import trimesh 
import numpy as np

labels = np.load('data/THuman/THumans2.npz') 
scan_path = 'data/THuman'
fid = '0006' 
obj_path = f'{scan_path}/{fid}/{fid}.obj'
scan_labels = labels[fid] 
mesh = trimesh.load(obj_path, process=False, maintain_order=True)
verts = mesh.vertices 
sel_verts = verts[scan_labels == 8]
colors = mesh.visual.to_color().vertex_colors[scan_labels == 8]
point_cloud = trimesh.points.PointCloud(sel_verts, colors=colors)
point_cloud.export('debug.ply') 

[fatbao55]

@Bozcomlekci Hi, I still got the wrong vertices after using your code. I have attached my visualisation below. My problem is not about the vertex colors but rather the clothing labels do not seem to correspond to the correct vertices.

Just to check, are you using THuman2.0 or THuman2.1? Just to be sure, i redownloaded the THuman2.0 dataset and still got the same error. Also, there seems to only be 72 labelled humans in the provided npz file (THumans2.npz) but THuman Dataset contains 500+ meshes. Is this correct?

Hope to get your advice on this, thanks

[garvita-tiwari]

@fatbao55 Yes we only provide labels for a subset of THuman2.0 scans.

[fatbao55]

@garvita-tiwari Thanks for the clarification.

Do you have any advice on why the clothing labels does not correspond to the vertices for the thuman2 samples? I did not have this problem for other samples i.e. Close-Di, Close-Dc and I was wondering if the labels are wrong, or if I'm using the wrong dataset i.e. THuman2.0/ THuman2.1?

[Bozcomlekci]

@fatbao55 Sorry for my misunderstanding of the issue. I think the labels inside THumans2.npz require a fix. We will let you know after the fix.

[garvita-tiwari]

@fatbao55 The problem is that the ordering of labels in THumans2.npz is not consistent with the vertex ordering in the original scan dataset. We have uploaded the correct ordered labels for 20 scans here: https://nextcloud.mpi-klsb.mpg.de/index.php/s/CgDFjgymkzeDrAM

For the remaining, we will update you soon. Thanks.

[fatbao55]

@garvita-tiwari The new labels are correct now. Thanks for the fix, and looking forward to the full set.

[zfonemore]

@garvita-tiwari I cannot find the correct ordered labels for remaining scans. Do you release the full set?

Thanks!

[garvita-tiwari]

@garvita-tiwari I cannot find the correct ordered labels for remaining scans. Do you release the full set?

Thanks!

We plan to release it soon, approximately in a month.

[garvita-tiwari]

@zfonemore , @fatbao55 We have updated labels for 500 scans from THuman2.0 here: https://nextcloud.mpi-klsb.mpg.de/index.php/s/CgDFjgymkzeDrAM Please check the file: THuman2.0_labels.npz

I have checked these labels with the orginal scans and this is how the segmentation quality should look like: