search

sxyu / pixel-nerf

PixelNeRF Official Repository
https://alexyu.net/pixelnerf
BSD 2-Clause "Simplified" License
1.39k stars 196 forks source link

Test with custom chair from synthetic Nerf dataset #30

Open emres8 opened 3 years ago

emres8 commented 3 years ago

I am trying to test the pre-trained srn-chair model for custom data. I am working on the synthetic chair model from Nerf paper since camera intrinsics and rotation matrix for each image is available.

Approach Downsampled images to 128x128, filled their transparent backgrounds with white. Updated some of the parameters with respect to the usage in Nerf code, explicitly: elevation,z_near,z_far.

Visual results from gen_video.py have three main problems:

  1. Some of the generated rays aren't connected with the main part of the object (probably related to focal length or depth)
  2. Although rotation matrices are given, in the results images are rotated in a wrong way.
  3. When using multiple images as input, the output becomes messier.

It seems the problems may occur because simply the chair model doesn't belong to the dataset. Still, do you have any suggestions?

emres8 commented 3 years ago

https://user-images.githubusercontent.com/74253593/124925375-09df0080-e005-11eb-803a-d815dfc0fbee.mp4

an example output generated using single image

william2ai commented 1 year ago

Did this question fixed? @emres8

yuliangguo commented 11 months ago

@emres8 I also realize such rendering artifact happens very often even the model is familier with the object category. Have you found any solution to those duplicated artifacts in backgroudn so far?