Questions about quantitative results

[wcy199705]

Hello. Thanks for the great work and the released dataset.

I have a question about the results of your experiments on paper. I'm trying to reproduce the results of your paper, but the result of the algorithm for 300 frames of 'flame_salmon_1' does not show a good score unlike your paper (approximately PSNR: 26). Besides, the convergences of NeRF-T, NeRF-ISG and NeRF-IST are much slower than yours. I implemented the code based on https://github.com/yenchenlin/nerf-pytorch.

Could you please tell me the score (PSNR) for each scene? Can you give me some hints to fully implement it (eg: used library or base code, learning strategy (learning rate decay, weight initialization of MLPs, used batch sizes of each model(NeRF-T, NeRF-ISG/IST)) ?

thank you!

[MELANCHOLY828]

Hi, I heard that you are reproducing the results of this paper, I would like to ask a question about the reading of the dataset? The dataset has a high resolution and a large number of pictures, and I have no way to load it all at once. Can you discuss with me? Grateful.

[zhaoyang-lv]

Could you please tell me the score (PSNR) for each scene?

We did not do the full evaluations on other scenes. All PSNRs are reported in the paper.

The batch sizes, learning rate, network architecture, and other training parameters are also discussed in the paper. The training image resolution usually will impact the PSNR. Make sure you follow the same setting (1K resolution for training and evaluation).

the convergences of NeRF-T, NeRF-ISG and NeRF-IST re much slower than yours.

Not sure whether you count the dataloaders part in this process. You will need to good cache strategy to preload and precompute rays, which will affect the speed quite a bit.

[wcy199705]

@zhaoyang-lv Thanks for the quick answer to my question! And Sorry for the confusion. By "speed" in my question I mean "number of iterations". Could you tell me about libraries, baseline codes and weight initialization you used to effectively implement NeRF? (e.g. pytorch3d, jax)

[zhaoyang-lv]

We start the implementation similar as https://github.com/yenchenlin/nerf-pytorch but changed quite a bit in the end.

We did not change the weight initialization. No particular other library we used. The number of iterations is tied to the number of gpus and which gpu you use. For 8 16GB P-100 machine, I think you can fit about 1K rays per-gpu, and that will be about 8K rays per iterations.

[wcy199705]

@zhaoyang-lv Thank you for your meaningful reply! These replies would be of great help to me!

[wcy199705]

@13753925279 I precompute and store weights(ISG/IST) of all frames(300 frames). And I load the precomputed weight of each frame during training. This question will help. https://github.com/facebookresearch/Neural_3D_Video/issues/8