search

facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.85k stars 1.32k forks source link

Cannot reproduce NeRF results on LLFF Fern dataset #1292

Open sergeyprokudin opened 2 years ago

sergeyprokudin commented 2 years ago

Hi Pytorch3d team,

I was recently trying to reproduce the results of Pytroch3d NeRF implementation on the LLFF "fern" dataset. No changes on my side were made to the code, I've just run the provided command:

python ./train_nerf.py --config-name fern

However, the results I'm getting on the validation set are significantly lower than the ones reported in the readme (27.9 PSNR vs 17.8 I'm getting). I somehow feel this might be connected to the behavior described in the following open issue.

It would be great if you can look into this. Having reproducible NeRF baselines in a modular and extendable framework like Pytorch3d would be of great help for future research on neural rendering.

Thanks in advance,

~Sergey

gkioxari commented 2 years ago

It's my understanding that we are deprecating projects/nerf with the new implicitron library

sergeyprokudin commented 2 years ago

I see, thanks for the clarification! It would be great to have the code for reproducing NeRF LLFF and Lego within the implicitron framework then.

gkioxari commented 2 years ago

@bottler @davnov134 Can you point Sergey to the code that reproduces NeRF for LLFF and Lego?

sergeyprokudin commented 2 years ago

Hi folks! Any update on this?

weizhenFrank commented 1 year ago

@sergeyprokudin Bro, I also have the same problem as you. I run the code python ./train_nerf.py --config-name fern and python ./test_nerf.py --config-name fern. But I didn't fully reproduce the result. My result is as the screenshot shows.

image

However, the repo here said the PSNR should be 


| Implementation |  Lego: test PSNR |  Fern: test PSNR |  training speed |
+----------------+------------------+------------------+-----------------+
| TF (official)  |             31.0 |             27.5 |  0.24 sec/it    |
| PyTorch3D      |             32.7 |             27.9 |  0.18 sec/it    |
+----------------+------------------+------------------+-----------------+```
weizhenFrank commented 1 year ago

Update: Just as https://github.com/facebookresearch/pytorch3d/issues/868 said, this implementation has some problems for non-square images. But I successfully reproduce the lego result.