Open ssh98son opened 7 months ago
Hi, while looking into the code, I also found that there are Smooth L1 Loss which are applied to depth output head. It seems like the depth output head is actually not used for shapo inference. Can I ask the purpose of it?
Hi @ssh98son
Thanks for your interest in our work and trying out the training code we released.
A bit hacky solution, but you could append a random latent vector to generate your validation data and run training. This can be generated for all instances as follows: latent_vec = torch.rand(64)
. The intuition for this is two-folds, we don't have access to GT SDF field for the validation or test set either during training or testing so even if we optimize over a new latent code given a pretrained SDF network, the validation error is not a great metric for shape latent vector to determine early stopping. Hence, we completely ignore this metric during validation and only determine early stopping based on the rotation and translation errors, which is a more sound metric. So a random latent code here would mean that your training would run smoothly, and it would just produce a very high val error for shape which you can safely ignore. A better way to do it would be to still include random latent vectors during data generation so that the training runs fine but you could comment out the shape_emb_loss and appearance_emb_loss from here depending on the prefix value i.e. only in val mode so these losses don't appear in your total loss calculation oinly furing validation.
Your observation is correct. This is just an auxillary loss. Please see our previous work CenterSnap for a motivation behind it. Specially, Table 3 ablation where we showed it helps sim2real transfer as we trying to infer noise free depth from noisy depth which brings synthetic and real depth domains more closer.
Hope it helps!
Hi, I'm trying to train shapo with my own synthetic dataset. Beforehand, I wanted to reproduce your results.
I tried to generate a dataset file and succeeded in the train set. However, it seems like some files are missing to generate the validation set. In the attached![Screenshot from 2024-01-15 21-44-07](https://github.com/zubair-irshad/shapo/assets/50664072/ebd934f5-069f-4cf3-a0a9-bd9c4dae865b)
sdf_rgb_pretrained.tar.xz
, I cannot find "Reconstructions" directory.Because of it, the generation script says the following error. I ran the following command.
![Screenshot from 2024-01-15 21-49-03](https://github.com/zubair-irshad/shapo/assets/50664072/fa1e279c-1edd-4e2c-9941-b48dfa668898)
Here is my data structure. I followed the guidelines and was able to reproduce the training set, but not validation set.![Screenshot from 2024-01-15 21-50-08](https://github.com/zubair-irshad/shapo/assets/50664072/8946c011-c9c6-47e1-862b-9c1203182f79)
Thank you a lot in advance.