search

chenhonghua / MVIP-NeRF

Other
17 stars 0 forks source link

Documents for running on llff and custom dataset #5

Open ericyangchen opened 4 months ago

ericyangchen commented 4 months ago

Hi, congratulations on the good paperwork! I'm having problems with running llff and custom datasets. The document seems to be missing some important parts regarding how to run with the spinnerf-dataset from scratch. Can you provide a clear example of the steps required to run experiments with the original spinnerf-dataset?

Another question is, in the paper, you claimed not to rely on direct 2D RGB inpainted results, but in DS_NeRF/load_llff.py, the RGB inpainted images are loaded. I'm a bit confused as to why the RGB inpainted images are loaded in the first place.

I would greatly appreciate the clarification and additional documents. Thank you for the good work!

chenhonghua commented 4 months ago

For the first question, I will update the readme later. For the second question, we just use the RGB inpainted images as an example. In our experiments, we did not use these inpainted results.

ericyangchen commented 4 months ago

@chenhonghua Hi, thank you for the reply. I'm looking forward to the documents so we can all enjoy your fantastic work!

For the second question, should the load_llff.py be modified to load the original images instead of the RGB_inpainted ones as it currently does?

Arcxml commented 4 months ago
image

Hello, I have one more question, should the load_llff.py be modified to load the original images instead of the RGB_inpainted, I think you use the RGB_inpainted images in your code

chenhonghua commented 4 months ago

Hi,

I put the repaired image in this folder so that we can also run SPInNeRF. As for my work, I put the masked original image in it, i.e. keep the masked area black. I will upload new data for reference.

ericyangchen commented 4 months ago

Hi, thanks for the explanation.

Could you please provide references on running other datasets (MipNeRF360, llff, etc)? I believe this would clear the confusion with the current code requiring RGB_inpainted to run.

Thank you again!

ericyangchen commented 3 months ago

Hi, thanks for following up on the documentation. Is there an expected date when the document will be ready? Really looking forward to it!

jingyushi commented 4 weeks ago

Hi, For a customized dataset, you can follow SPinNeRF to generate pseudo depth, get inpatined depth and RGB, and then organize the data as the given spinnerf dataset. Put 60 inpainted or original training images in the RGB_inpainted/Depth_inpainted folder (the object area is masked during training anyway). I was able to create a dataset this way and trained MVIP-NeRF on it.