search

nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
9.41k stars 1.28k forks source link

process our own data（video with dynamic scene）and train with nerfstudio？ #2018

Open Kane-LAU opened 1 year ago

Kane-LAU commented 1 year ago

Question：

I noticed Nerf Studio provides options for download and train dnerf datasets (Thanks for @JulianKnodt ). meanwhile a plugin called kplanes-nerfstudio @Giodiro has been introduced( as @liruilong940607 mentioned ). Thanks for all your contribution.

It worked well with the existing Dnerf data, and I was quite impressed. However, I'm unsure about how to process my own video data to make it compatible for training with nerfstudio , I would greatly appreciate it if you could provide guidance !

Data description

device： shot with mobile Phone (without camera position) Content moving robot arm screenshot image

Originally posted by @JulianKnodt in https://github.com/nerfstudio-project/nerfstudio/issues/813#issuecomment-1569451803

JulianKnodt commented 1 year ago

eh, don't just copy this, but please fill in the details of your issue

Kane-LAU commented 1 year ago

eh, don't just copy this, but please fill in the details of your issue

Sorry, You can check it again, I have edited and filled it.

JulianKnodt commented 1 year ago

If your camera is fixed, I think you can just set the camera to be a fixed position, and set the configuration the same as an existing config file. Try looking at the existing d-nerf dataset, and mimicking the same format.

aleatorydialogue commented 1 year ago

Question：

I noticed Nerf Studio provides options for download and train dnerf datasets (Thanks for @JulianKnodt ). meanwhile a plugin called kplanes-nerfstudio @Giodiro has been introduced( as @liruilong940607 mentioned ). Thanks for all your contribution.

It worked well with the existing Dnerf data, and I was quite impressed. However, I'm unsure about how to process my own video data to make it compatible for training with nerfstudio , I would greatly appreciate it if you could provide guidance !

Data description

device： shot with mobile Phone (without camera position) Content moving robot arm screenshot image

Originally posted by @JulianKnodt in https://github.com/nerfstudio-project/nerfstudio/issues/813#issuecomment-1569451803

Any luck with this? I can get the correct data format for kplanes-dynamic to start training but it never converges into anything viewable at all.

xjli360 commented 1 year ago

If your camera is fixed, I think you can just set the camera to be a fixed position, and set the configuration the same as an existing config file. Try looking at the existing d-nerf dataset, and mimicking the same format.

how about the camera is not fixed? i know the camera of d-nerf dataset is fixed, but the camera in my dataset is not fixed.