About unsupervised training of BAE-NET

[supriya-gdptl]

Hello @czq142857,

Thanks for sharing the code. I noticed that the commands you mentioned for unsupervised training of BAE-NET use all models in one category during the training step. Shouldn't it be using only train-set models instead of all models? If it is by design (and not a typo), how did you decide the number of iterations the model should be trained for? How do you know the model is not overfitting?

Another question I have related to unsupervised training is how does the model learn part segmentation with just the mean square loss, especially the parts which are not connected to each other directly e.g. arms of a chair or engines of an airplane? How come one branch learns implicit field for both arms of a chair rather than two separate branches each one learning one arm?

Thank you, Supriya

[czq142857]

Using all models for training is by design. We want to overfit the entire dataset (for reconstruction), in order to obtain accurate segmentations for each shape. You can consider the algorithm as one without training phase - it just takes a collection of shapes as input and then output the segmentations. If you have new shapes, you will have to re-run the algorithm (re-train the network).

I do not know exactly why the model performs part segmentation like that, and the paper does not explain it either. The unsupervised training is not exactly controllable. Please read the 5. Conclusion, limitations, and future work section of the paper for details.

Best, Zhiqin

[supriya-gdptl]

Thank you!

[supriya-gdptl]

Hello @czq142857,

Sorry to bother you again, but I am unable to reproduce the results by using pre-processing code that you have provided. I used the following command to create binvox files for the ShapeNetCore-v2 03001627_Chair dataset:

~/binvox -bb -0.5 -0.5 -0.5 0.5 0.5 0.5 -d 64 -e {model_name}/models/model_normalized.obj

Then, I used gather_vox_from_binvox.py to get voxels and point-value pairs from it. I trained BAE-NET with 4 branches in the L3 layer in an unsupervised way and I could get the following part segmentation results.

However, when I used the voxels and point-value pairs from the data that you have provided and trained the same BAE-NET in an unsupervised way, I could get better part segmentation.

Is there something that I am missing/doing wrong in the pre-processing step? Did you use any other pre-processing steps? Because I used the same model for training, so technically I should get similar part-segmentation results. Appreciate your help.

Thanks, Supriya

[czq142857]

Hi Supriya,

Please visualize the reconstructed shapes, and see if they align with the point clouds. If they do not align, then there should be a problem in the data pre-processing step.

Best, Zhiqin

[supriya-gdptl]

Thank you @czq142857. I visualized the reconstructed shapes and I noticed that they are shifted from the original position of the mesh. Please see below the example. If we compare it with actual mesh images below, especially 2nd one, we can see it has been shifted, by comparing chair position with respect to the origin shown by the small gray sphere

It looks like the mesh normalization code in utils.py/load_obj is causing the issue. What is the purpose of normalizing the mesh while loading it? Would it affect the subdivision code if we don't normalize the mesh?

Thank you, Supriya

[czq142857]

The mesh is normalized when being loaded, because in all my training data the shapes are normalized in this way. if your prepared training data has a different mesh normalization, you could remove the normalization code in "load_obj". But based on your figures, your normalization was not very successful when preparing voxels. Please use the normalization method in "load_obj" to normalize your obj files.

[supriya-gdptl]

Sure, I will normalize the models as done in "load_obj" for further experiments.

Btw, I could reproduce the results now after I removed lines 156, 157 and 158 from utils.py/load_obj, and then executed

python main.py --mesh --L1reg --iteration 200000 --dataset corev2_vox 
--data_dir "./mydata/corev2_03001627_chair/" --checkpoint_dir "checkpoint_unsup_corev2/chair" 
--supervision_list "corev2_vox.txt" --sample_dir "samples/corev2_03001627_chair"  
--real_size 32 --points_per_shape 8192

Thank you very much for all the help!