search

graphdeco-inria / gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
Other
14.57k stars 1.91k forks source link

Why the images I generated cannot be reconstructed by colmap (only two out of 21 images were recognized after running)? #732

Open qixuanwang-233 opened 7 months ago

qixuanwang-233 commented 7 months ago

Describe the bug The multi-view image I generated by myself cannot be reconstructed by colmap. The image pixels are 576*576, and the quality is ok.

My own picture display frame_0 frame_1 frame_2 frame_3 frame_4 frame_5 frame_6 frame_7 frame_8 frame_9 frame_10 frame_11 frame_12 frame_13frame_14 frame_15 frame_16 frame_17 frame_18 frame_19 frame_20 png…]()

Screenshots I'm using colmap version 3.9. When I finished running, the terminal looked like this： image Error accompanies the process:
=> Could not register, trying another image. Does anyone know the reason for this?

jaco001 commented 7 months ago

Colmap is little tricky. If it says that it cannot do things... is is usualy your fault. Datasets need to be prepared in specific ways that help colmap and don't disturb it. Not knowing how to take pictures is first sin. :)

This is not that hard.

About your dataset.

  1. Sharp images (picture size with full HD and above will be ok)
  2. Sharp images with great image depth
  3. Sharp images... ok ok :)
  4. Do not remove background - any surface with lots of details give more points to trace (also this help if your object is 'thin' at one axis)
  5. Do few more orbits with your objects with different heights and angle.
  6. 15-20 degree picture to picture is ok (as you take in your dataset)

Usualy 50 pic is at low end to make something, above 400 introduce noise and it can be diffictult cus GPU VRAM limitations.

Learn your lessons, be smart and GL.

qixuanwang-233 commented 7 months ago

Colmap is little tricky. If it says that it cannot do things... is is usualy your fault. Datasets need to be prepared in specific ways that help colmap and don't disturb it. Not knowing how to take pictures is first sin. :)

This is not that hard.

About your dataset.

  1. Sharp images (picture size with full HD and above will be ok)
  2. Sharp images with great image depth
  3. Sharp images... ok ok :)
  4. Do not remove background - any surface with lots of details give more points to trace (also this help if your object is 'thin' at one axis)
  5. Do few more orbits with your objects with different heights and angle.
  6. 15-20 degree picture to picture is ok (as you take in your dataset)

Usualy 50 pic is at low end to make something, above 400 introduce noise and it can be diffictult cus GPU VRAM limitations.

Learn your lessons, be smart and GL.

Thank you for your answer! I will try according to your suggestion. I also have a problem, that is, the back and side of the teddy bear I generated appear this error "=> Could not register, trying another image." Is it because there are too few feature points to collect on the back? It is difficult for me to increase the feature points on the back.Do you have a solution to this problem？

jaco001 commented 7 months ago

AI images have flaws. They don't have consistency in the tiny details and colmap need this for catching what is going there. So... bad luck for now. But if you know photoshop(something) and 3D software like Blender there can still be a way. Some work is included ;)

  1. Make alpha chanell on your images (at photosomething)
  2. Put simple scene in 3D software
    • at centre your image
    • at bottom any high detailed texture or reference points (this will be main horse for colmap)
    • set camera that corespond to you image
  3. Make render, change image, set camera and repeat ;) (you simulate turn table)

Now colmap should have a lot of solid data to proceed with your new renders.
GL

qixuanwang-233 commented 7 months ago

AI images have flaws. They don't have consistency in the tiny details and colmap need this for catching what is going there. So... bad luck for now. But if you know photoshop(something) and 3D software like Blender there can still be a way. Some work is included ;)

  1. Make alpha chanell on your images (at photosomething)
  2. Put simple scene in 3D software
  • at centre your image
  • at bottom any high detailed texture or reference points (this will be main horse for colmap)
  • set camera that corespond to you image
  1. Make render, change image, set camera and repeat ;) (you simulate turn table)

Now colmap should have a lot of solid data to proceed with your new renders. GL

This is the second time you have helped me. Thank you very much! You are really professional and enthusiastic. I will try according to what you said. :)

jaco001 commented 7 months ago

image

Furkathertaha commented 6 months ago

Hi, have you figured out the way to solve this problem? I did similar trial as rendering totoal 90 pictures around an object with the orbit being an entire sphere. For certain latitude, I have up to 16 images. But colmap only select 4 pictures from 90, and I believe they are all viewpoints optimized by 3dGS. As expected, the result is very poor not looking from these 4 directions. Have you found a feasible solution? or I guess I have to manually set camera parameters of 90 pics with this initial PCL from 4 pics.

MilesMena commented 2 months ago

Maybe I have a seperate issue, but COLMAP recreates this problem for me on Linux, but works as intended on Windows. Using the same video, COLMAP creates the correct dataset on windows but COLMAP on linux only creates two images and has this exact output.