Training to Generate Point Cloud with More Features

[djFatNerd]

Hello, I really appreciate your great work and want to explore it more!

In the paper you mentioned that the each point can have more features than only its xyz coordinates, if I were to train this model to generate a point cloud for a more complex object, for example, an object with different colors for different parts, is it possible for me to just append the rgb values to each point's coordinates, so its now N x 6? Or should I just have multiple colors and treat them as class labels, then encode these class labels as features to append to each point? In this case do I need to re-design the loss? Would this potentially cause the training procedure to take much much longer?

Thank you so much!

[ZENGXH]

Hi! Sorry for the late reply: yes simply appending rgb values to the coordinates will work. I try this before can it can generate reasonable output with color. I use the float number for rgb (in range [0,1]). The training time is not much longer. I used the exact default hyper-parameters.

[djFatNerd]

Thank you so much! I will follow your advice and try!

[OswaldoBornemann]

Do we have to normalize the feature, such as RGB, or not? @ZENGXH

[OswaldoBornemann]

@ZENGXH If we add more features, should we change the extra_feature_channels in PointTransPVC object to the number of features? Thanks.

[OswaldoBornemann]

@ZENGXH Also, do we need to change the ddpm.input_dim to 4 ( 3 + 1 feature dim) ?

[ZENGXH]

• it will be better to normalize RGB to -1,1
• I think changing ddpm.input_dim 6 data.input_dim 6 (3 for xyz and 3 for rgb) may be enough. you can try and see if it can run.

[OswaldoBornemann]

@ZENGXH, I see. I have also noticed that you haven't normalized the dataset coordinate. But you recommend normalising the features, such as RGB. Am I right?

[OswaldoBornemann]

Do we need to change the value of extra_feature_channels in style_encoder and encoder? Or just leave it as 0.

[OswaldoBornemann]

@ZENGXH, I see. I have also noticed that you haven't normalized the dataset coordinate. But you recommend normalising the features, such as RGB. Am I right?

Oh, my apology. Actually, I saw you apply the normalization operation on the coordinate. https://github.com/nv-tlabs/LION/blob/ca8129d8c00bb314e30e51992c3abfe002c625d9/datasets/pointflow_datasets.py#L261

[ZENGXH]

Do we need to change the value of extra_feature_channels in style_encoder and encoder? Or just leave it as 0.

I think you dont need to change it. You could leave it as 0 and see if the running raise any error (like channel mismatched)

[OswaldoBornemann]

I see. I just leave it as 0, and everything goes well.