convert.py results in two images being squashed

[malamutes]

Can someone explain why when i run convert.py on a dataset with 200+ images only two images is chosen for "images" folder for train.py to use. I've tried rendering the same dataset and it's much better in COLMAP than in gaussian splatting so I must be doing something wrong.

[malamutes]

Oh, and the two images are resized and squashed so my final splat is just a mess, I've tried putting the same dataset into polycam's gaussian splatter and theirs render beautifully, I'm not sure what I'm doing wrong

[jaco001]

If you don't mind - share your dataset.

[zhangwenniu]

I also encountered the same situation. Many images under the input folder were used for calibration in Colmap. However, in the end, only two images entered the images folder, and the images were distorted. The size of the output images is different from that of the original images.

[jaco001]

If your loops aren't connected/overlapped enough for colmap you end with 'best' loop in sparse/0. And this is not fault of colmap, but rather how you make your photo-session for photogrametry.

[malamutes]

jaco can i send u the dataset via google drive or something?

[malamutes]

If your loops aren't connected/overlapped enough for colmap you end with 'best' loop in sparse/0. And this is not fault of colmap, but rather how you make your photo-session for photogrametry.

Does this mean its best if the input video/ images are in a perfect loop? can i just reverse the original video and append that to my current video ?

[jaco001]

Video - is usualy blury for detailed photogrametry (compresion and not much you can do with light).

By loops I mean This example for one object to get ok - photogrammetry. Those loopa have very big overlaps and not much angle changes.

If you try scan room or multiple rooms you must reverse this to walls. Good tutorial (specialy foir gaussians) are https://medium.com/@heyulei/capture-images-for-gaussian-splatting-81d081bbc826

And here is good talk about photogrametry in general. https://youtu.be/5D-P2kRUuKA?si=yYAojtNd9UVo3CxE

[malamutes]

if u have time do u mind trying to reconstruct with the video i have? i want to see if the video is just bad or something wrong with my end

[jaco001]

Post here link for your dataset and I will try.

[malamutes]

can i share with google drive? the video is bigger than 25 mb

[malamutes]

https://drive.google.com/file/d/1O5xPtWvWz1J91K8dPXgRvf8wi2wivAQb/view?usp=drive_link here,

[jaco001]

If so use site like https://wetransfer.com/

[malamutes]

can u get the video from the link?

[jaco001]

I took your video. From ffmpeg conversion to png for every 5 frame (first 72) colmap don't have a problem. I used colmap 3.8 and old Inria scripts. So only methodology. I share my side - check email.

[malamutes]

do u mind sending me ur output trained with gaussian splatter or at least let me see what it should looke like?

[jaco001]

np

[malamutes]

also when u run it do u do external commands, so far ive just been following the tutorial for python convert.py + python train.p after ffmpeg to images

[jaco001]

I only prepare and select good images. Rest is standard scripts with default parameters.

[malamutes]

by standard scripts u also just do ffmped, convert.py and train.py?

[malamutes]

and aside from blurry images what would you consider a good image in this case?

[jaco001]

In short

1. mov video file -> ffmpeg every 5 frames
2. selection of images (you can check it in first zip) - rotation + rescaling with bit of sharpnes in Light room. You can use any spoftware to manage images (open source eg. Dark table). Choose sharp images! (I took first 72 pic after that)
3. convert and train standard parameters BTW I send you models too

[malamutes]

yea im looking at ur model rn, is there a way to have a good splat all around? since if i go to the other side the splats start to break down lol, and ill try to see if i can get one working too

[jaco001]

This is only from first 72 pics. Try put rest and train model. You can crop model too to remove background. Anyway it is working on your file.

[malamutes]

yea its working good, thanks for the help, ill see if i can get it to work, TY!

[malamutes]

oh also, in dark table, is it possible to simultaneously apply changes to ALL images at once?

[jaco001]

Should :) https://docs.darktable.org/usermanual/4.0/en/module-reference/utility-modules/lighttable/styles/ But simply rotation and export you can do without this.

[malamutes]

hmm lemme re clone the files frmo here, there might be some issue with mine

[malamutes]

wait there might be something wrong with my gaussian convert files, even with your images, its only giving me two images output

[jaco001]

At export tab - as I recall you set export size (eg. by long side) and you type size. But usualy upscale is only take your VRAM. Use it if you have many pic from 4K+ and lov VRAM.

[malamutes]

any ideas what could be causing this 2 output files issue?

[malamutes]

something here maybe?

[willei]

I think it is sparse/0 dir is not the final result for colmap mapper， when run convert.py, u may meet Could not register, trying another image , and the first Initializing with image pair #xx1 and #xx2 did not ok, then colmap continue find，as result, u can sparse/1 or sparse/2 dir , but in convert.py , when do image_undistorter set input_path to sparse/0 dir in source_path，so only find two pair image xx1 and xx2。I change the convert.py to set input_path to the final sparse dir when do image_undistorter。It works for me.

my change below:

sparse_sub_dirs = os.listdir(args.source_path + "/distorted/sparse") final_sparse_sub_dir = str(max([int(i) for i in sparse_sub_dirs]))

img_undist_cmd = (colmap_command + " image_undistorter \ --image_path " + args.source_path + "/input \ --input_path " + args.source_path + "/distorted/sparse/" + final_sparse_sub_dir + "\ --output_path " + args.source_path + "\ --output_type COLMAP")

[teym822]

I have also encountered the same problem. Is there any solution now？

[yoyo-000]

I've solved the problem:

1. Using colmap's GUI, the input image is reconstructed automatically instead of using a script. And you can see the generated points and camera trajectory inside the GUI.
2. need to change the model of the camera, i have seen some solutions to change the database, but the code inside reads the contents of camera.bin, you can change the python script so that it accepts the default SIMPLE_RADIAL. Then the training will work fine.

[teym822]

I've solved the problem:

1. Using colmap's GUI, the input image is reconstructed automatically instead of using a script. And you can see the generated points and camera trajectory inside the GUI.
2. need to change the model of the camera, i have seen some solutions to change the database, but the code inside reads the contents of camera.bin, you can change the python script so that it accepts the default SIMPLE_RADIAL. Then the training will work fine.

sorry, i do not understand. do you mean to use colmap GUI directly to convert the data instead of using the convert.exe file to convert the data. And what script should be changed?

[yoyo-000]

I've solved the problem:

1. Using colmap's GUI, the input image is reconstructed automatically instead of using a script. And you can see the generated points and camera trajectory inside the GUI.
2. need to change the model of the camera, i have seen some solutions to change the database, but the code inside reads the contents of camera.bin, you can change the python script so that it accepts the default SIMPLE_RADIAL. Then the training will work fine.

sorry, i do not understand. do you mean to use colmap GUI directly to convert the data instead of using the convert.exe file to convert the data. And what script should be changed?

1. YES.
2. If you use the GUI directly like above and are guaranteed to get the desired results, you can get the images and sparse folders directly. At this point the script that loads the data needs to be changed where it recognizes the camera and then it's ready to run.

In dataset_readers.py:

if intr.model=="SIMPLE_PINHOLE":
    focal_length_x = intr.params[0]
    FovY = focal2fov(focal_length_x, height)
    FovX = focal2fov(focal_length_x, width)
elif intr.model=="SIMPLE_RADIAL":
    focal_length_x = intr.params[0]
    FovY = focal2fov(focal_length_x, height)
    FovX = focal2fov(focal_length_x, width)

This may cause other problems, but it's currently being successfully trained on my end, so you can talk to me if you see potential problems.

[teym822]

I've solved the problem:

1. Using colmap's GUI, the input image is reconstructed automatically instead of using a script. And you can see the generated points and camera trajectory inside the GUI.
2. need to change the model of the camera, i have seen some solutions to change the database, but the code inside reads the contents of camera.bin, you can change the python script so that it accepts the default SIMPLE_RADIAL. Then the training will work fine.

sorry, i do not understand. do you mean to use colmap GUI directly to convert the data instead of using the convert.exe file to convert the data. And what script should be changed?

1. YES.
2. If you use the GUI directly like above and are guaranteed to get the desired results, you can get the images and sparse folders directly. At this point the script that loads the data needs to be changed where it recognizes the camera and then it's ready to run.

In dataset_readers.py:

if intr.model=="SIMPLE_PINHOLE":
    focal_length_x = intr.params[0]
    FovY = focal2fov(focal_length_x, height)
    FovX = focal2fov(focal_length_x, width)
elif intr.model=="SIMPLE_RADIAL":
    focal_length_x = intr.params[0]
    FovY = focal2fov(focal_length_x, height)
    FovX = focal2fov(focal_length_x, width)

This may cause other problems, but it's currently being successfully trained on my end, so you can talk to me if you see potential problems.

thanks a lot. it solve my problem in some situations. but i also found that dataset also effect the results.