RefNeRF Real Dataset

[grgkopanas]

Hi,

Thank you all for the great work and the release of the code. I want to try the RefNeRF in real datasets, starting by reproducing the results in the scenes from the paper. I noticed that, first, the only config file for RefNeRF is for the shiny blender dataset, and second that the scene files for the real scenes from the RefNeRF website don't match the structure of the other scenes usually created for NeRF (transforms.json etc are missing).

Is there an easy way to run the real scenes and do I need special config parameters to reproduce the results of the paper?

Best, Georgios Kopanas

[StarsTesla]

Same issue here

[gkouros]

The codebase supports different dataset formats. The shiny dataset is of type blender and the real dataset is of type LLFF. To train on real scenes with RefNeRF, you can use _config/blenderrefnerf.gin but change _Config.datasetloader from "blender" to "llff". You probably need to copy some params from the other llff config files as well.

[gkouros]

Yeah, I had the same issue with one of my datasets. I think you need to disable Config.compute_normal_metrics.

On 30/08/2022 15:24, grgkopanas wrote:

@gkouros https://github.com/gkouros did it work for you? I am trying garden-spheres and I get an error related to normals:

|Traceback (most recent call last): File "/gpfsssd/scratch/rech/tqd/urx22mc/multinerf/train.py", line 288, in

app.run(main) File "/gpfslocalsup/pub/anaconda-py3/2021.05/envs/tensorflow-2.9.1+py3.10/lib/python3.10/site-packages/absl/app.py", line 308, in run _run_main(main, args) File "/gpfslocalsup/pub/anaconda-py3/2021.05/envs/tensorflow-2.9.1+py3.10/lib/python3.10/site-packages/absl/app.py", line 254, in _run_main sys.exit(main(argv)) File "/gpfsssd/scratch/rech/tqd/urx22mc/multinerf/train.py", line 55, in main dataset = datasets.load_dataset('train', config.data_dir, config) File "/gpfsssd/scratch/rech/tqd/urx22mc/multinerf/internal/datasets.py", line 52, in load_dataset return dataset_dict[config.dataset_loader](split, train_dir, config) File "/gpfsssd/scratch/rech/tqd/urx22mc/multinerf/internal/datasets.py", line 328, in __init__ self._queue.put(self._next_fn()) File "/gpfsssd/scratch/rech/tqd/urx22mc/multinerf/internal/datasets.py", line 483, in _next_train return self._make_ray_batch(pix_x_int, pix_y_int, cam_idx, File "/gpfsssd/scratch/rech/tqd/urx22mc/multinerf/internal/datasets.py", line 447, in _make_ray_batch batch['normals'] = self.normal_images[cam_idx, pix_y_int, pix_x_int] TypeError: 'NoneType' object is not subscriptable | It seems that even with the LLFF dataset, the code is trying to read the images for the normals that don't exist. — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned.Message ID: ***@***.***>

[prakashknaikade]

How the shiny blender dataset is created in blender from this blend files ?? @gkouros @grgkopanas I am new to blender.

[gkouros]

How the shiny blender dataset is created in blender from this blend files ?? @gkouros @grgkopanas I am new to blender.

The question is not relevant to this issue so create a new issue. In the meantime, you can look at similar issues in the original nerf repo e.g. https://github.com/bmild/nerf/issues/78.

[prakashknaikade]

https://github.com/google-research/multinerf/issues/136