threestudio-project / threestudio

A unified framework for 3D content generation.
Apache License 2.0
6.35k stars 480 forks source link

Error in in2n pre-training nerf #328

Open hrz2000 opened 1 year ago

hrz2000 commented 1 year ago
python launch.py --config configs/instructnerf2nerf.yaml --train --gpu 1 data.dataroot="data/bear" data.camera_layout="front" data.camera_distance=1 system.prompt_processor.prompt="Turn the bear into a grizzly bear" \
trainer.val_check_interval=100 \
data.eval_data_interval=15 \
system.start_editing_step=1000 \
trainer.max_steps=10000 \

image image I use in2n to edit nerf, but I can't train a complete scene. Where is the problem?thanks (it works in face datset)

DSaurus commented 1 year ago

Hi @hrz2000 , I think there are currently two problems. One is that our NeRF model does not support 360-degree scenes. Another issue is that some losses in NeRF, such as loss_opaque and loss_orient, are not suitable for representing the 3D background. So I am working on implementing Gaussian Splatting for these 3D scenes in the next few days.

Kirito-Ausna commented 2 months ago

@hrz2000 Hi, I am also using in2n to edit nerf but I can't find original scene data (multi-view images) with appropriate format (which can be processed with three studio). I did have the bear example you used from the original In2n, but it can't be processed with three studio. (the data of face case was provided by threestudio and it works). Would you like to share how to get the other benchmark cases data in the threestudio format?(like the bear case you used). Sorry for any trouble.