Poor results on own dataset

[ricshaw]

Hi, thank you for the code and great work. However, I am having issues using my own data. My result after training for 200 epochs is shown below:

Prediction:

Groundtruth:

I am using an edited version of the sport1 nhr dataloader and following the config file from: python train_net.py --config configs/nhr/sport1.py

Do you know why this is not working for my data? The dataset consists of 72 cameras, I am using 100 frames, and I have fit SMPL vertices to all views as shown below:

[pengsida]

Did you use the mask loss? The mask loss should remove the floating artifacts around the human.

[ricshaw]

Yes I used the same config file as NHR sport1. mask_weight is set to 0.1, maybe I should increase it? I also set the following: face_sample_ratio = 0.0 body_sample_ratio = 1.0 Because I don’t have the face segmentation. Is that okay?

[yunzhiy]

Hi. You can try to set body_sample_ratio = 0.7 as in the config file of NHR sport1 for body_sample_ratio = 1.0 means all sampled rays are on foreground objects. You can read this part of the code for more detail https://github.com/zju3dv/mlp_maps/blob/7fda4ac19e84692f1070614f669e44080541bf2b/lib/utils/if_nerf/if_nerf_data_utils.py#L172

[ricshaw]

Ah that might help, I will try that! I had another question about fixedcameras in the config file. I set fixedcameras to be 3 random training views… What does fixedcameras mean?

[pengsida]

It follows Neural Volumes, which takes some fixed subviews as the input of the encoder. Please read the paper of Neural Volumes.

[ricshaw]

Thank you!