search

bmild / nerf

Code release for NeRF (Neural Radiance Fields)
http://tancik.com/nerf
MIT License
9.6k stars 1.34k forks source link

Issue of traing self-made data #102

Closed hengfei-wang closed 3 years ago

hengfei-wang commented 3 years ago

Hi, thank you for sharing your impressive work.

Recently, I am trying to train nerf using my own data. Since colmap didn't work on the pictures I collected, I created my own poses_bounds.npy using calibration parameters of cameras according to this the following step which is from link:

  1. Make sure your poses are in camera-to-world format, not world-to-camera.
  2. Make sure your rotation matrices have the columns in the correct coordinate frame [down, right, backwards].
  3. Concatenate each pose with the [height, width, focal] intrinsics vector to get a 3x5 matrix.
  4. Flatten each of those into 15 elements and concatenate the close and far depths.
  5. Stack the 17-d vectors to get a Nx17 matrix and use np.save to store it as poses_bounds.npy in the scene's base directory (same level containing the images/ directory).

I also see the issue about the unit of data in poses_bounds.npy and set hwf in pixel and set translation and bounds in meter. But I get some confusing result like the following.

https://user-images.githubusercontent.com/39575336/112758030-c4cb2b80-901e-11eb-89d4-26f66a480471.mp4

The training picture is one person sitting in front of a green wall. It looks like that the training result mixes some colors from the person and the wall but makes no sense. The person is sitting at approximately one meter distance in front of the cameras so I set near and far bounds to 1 meter and 2 meters. Is there anything wrong I have done? Hope someone can kindly provide some help.

BioWar commented 3 years ago

I have the same problem with the green screen, if you solved the problem, please reply)

hengfei-wang commented 3 years ago

I did something wrong in the transformation of camera poses. Have corrected it.

closed.

Duncan1115 commented 2 years ago

@hengfei-wang Hi, i have the same situation. Could you tell me how to use calibration parameters of cameras to create poses_bound.npy? And would you like to share your code? Thank you so much!

luchaoqi commented 1 year ago

@Duncan1115 Have you figured out a good way to solve it?

rohin-k18 commented 1 week ago

can any one help me in training part of nerf existing code?