How can i get transforms_train.json from my data

yungommi commented 2 months ago

i have 250 photos and colmap(sparse or dense) output. how can i get transform_train.json from mine?

piotr310100 commented 2 months ago

Hi, you should be able to obtain it by using for example this script from instant-ngp repository https://github.com/NVlabs/instant-ngp/blob/master/scripts/colmap2nerf.py. However if you have just photos and no mesh you should most likely run the code with the gs_flat/gs_points flag instead of gs_mesh.

ManOfStory commented 2 weeks ago

Hello, I run the colmap2nerf.py as your mentioned above with hotdog's images. However, The result No good initial image pair found appears, only 3 frames are valid, the error code I list below, how can I generate the transforms_train.json with the hogdog's images? Thank you!

...
I0620 22:35:09.762295 11476 incremental_mapper.cc:375] Registering image #53 (4)
I0620 22:35:09.762300 11476 incremental_mapper.cc:378] => Image sees 62 / 630 points
I0620 22:35:09.762358 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:09.762364 11476 incremental_mapper.cc:375] Registering image #21 (4)
I0620 22:35:09.762368 11476 incremental_mapper.cc:378] => Image sees 530 / 971 points
I0620 22:35:09.762490 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:09.762495 11476 incremental_mapper.cc:375] Registering image #48 (4)
I0620 22:35:09.762499 11476 incremental_mapper.cc:378] => Image sees 136 / 1095 points
I0620 22:35:09.762598 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:09.762604 11476 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0620 22:35:10.696174 11476 incremental_mapper.cc:375] Registering image #29 (4)
I0620 22:35:10.696195 11476 incremental_mapper.cc:378] => Image sees 721 / 1233 points
I0620 22:35:10.696313 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:10.696318 11476 incremental_mapper.cc:375] Registering image #21 (4)
I0620 22:35:10.696323 11476 incremental_mapper.cc:378] => Image sees 530 / 971 points
I0620 22:35:10.696447 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:10.696452 11476 incremental_mapper.cc:375] Registering image #20 (4)
I0620 22:35:10.696457 11476 incremental_mapper.cc:378] => Image sees 527 / 917 points
I0620 22:35:10.696573 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:10.696578 11476 incremental_mapper.cc:375] Registering image #36 (4)
I0620 22:35:10.696583 11476 incremental_mapper.cc:378] => Image sees 460 / 1103 points
I0620 22:35:10.696710 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:10.696715 11476 incremental_mapper.cc:375] Registering image #40 (4)
I0620 22:35:10.696719 11476 incremental_mapper.cc:378] => Image sees 319 / 1050 points
I0620 22:35:10.696817 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:10.696823 11476 incremental_mapper.cc:375] Registering image #48 (4)
I0620 22:35:10.696827 11476 incremental_mapper.cc:378] => Image sees 136 / 1095 points
I0620 22:35:10.696925 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:10.696930 11476 incremental_mapper.cc:375] Registering image #53 (4)
I0620 22:35:10.696935 11476 incremental_mapper.cc:378] => Image sees 62 / 630 points
I0620 22:35:10.696992 11476 incremental_mapper.cc:389] => Could not register, trying another image.
I0620 22:35:10.698244 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:10.752538 11476 incremental_mapper.cc:289] Initializing with image pair #55 and #47
I0620 22:35:10.752990 11476 incremental_mapper.cc:294] Global bundle adjustment
I0620 22:35:10.861254 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:10.895501 11476 incremental_mapper.cc:289] Initializing with image pair #92 and #73
I0620 22:35:10.895983 11476 incremental_mapper.cc:294] Global bundle adjustment
I0620 22:35:10.999040 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.001511 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.002315 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.004791 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.005489 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.007964 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.008641 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.011111 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.011775 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.014236 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.014909 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.017369 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.018029 11476 incremental_mapper.cc:265] Finding good initial image pair
...

I0620 22:35:11.469830 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.472297 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.472959 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.475425 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.476089 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.478544 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.479228 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.481621 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.482295 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.484755 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.485415 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.487890 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.488565 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.491008 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.491672 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.494063 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.494745 11476 incremental_mapper.cc:265] Finding good initial image pair
I0620 22:35:11.497138 11476 incremental_mapper.cc:269] => No good initial image pair found.
I0620 22:35:11.497220 11476 timer.cc:91] Elapsed time: 0.666 [minutes]
==== running: colmap bundle_adjuster --input_path colmap_sparse/0 --output_path colmap_sparse/0 --BundleAdjustment.refine_principal_point 1
I0620 22:35:11.540465 13472 misc.cc:198]
==============================================================================
Global bundle adjustment
==============================================================================
I0620 22:35:11.973769 13472 bundle_adjustment.cc:866]
Bundle adjustment report:
    Residuals : 4968
   Parameters : 3058
   Iterations : 45
         Time : 0.430327 [s]
 Initial cost : 0.23405 [px]
   Final cost : 0.23319 [px]
  Termination : Convergence

I0620 22:35:11.973799 13472 timer.cc:91] Elapsed time: 0.007 [minutes]
==== running: mkdir colmap_text
==== running: colmap model_converter --input_path colmap_sparse/0 --output_path colmap_text --output_type TXT
outputting to transforms.json...
camera 1:
        res=(800.0, 800.0)
        center=(411.8908334149666, 371.52371570672915)
        focal=(1346.186917631821, 1423.572476985837)
        fov=(33.09711307373078, 31.388945489320005)
        k=(0.4526649231735813, -14.888869859638497) p=(-0.05545366318381546, 0.03252210165494483)
./hotdog_train/r_20.png sharpness= 349.563809375
./hotdog_train/r_12.png sharpness= 310.815190625
./hotdog_train/r_18.png sharpness= 319.477234375
up vector was [-0.98776828 -0.02606481 -0.15373498]
computing center of attention...
[-25.85897426  -3.3130837   -0.13101653]
avg camera distance from origin 26.48482140722281
3 frames
writing transforms.json

waczjoan commented 2 weeks ago

Hi, To help you more precisely, let me ask few questions:

did you run code with flag ---gs_type gs_flat and than run render with --gs_type gs_flat/gs_points?
why/for what do you need transformation_dataset.json file?

Tansformation_dataset.json is file dedicated mostly for blender type dataset, there is possible use mentioned script colmap2nerf to create this file, but it is not necessary in our method (you should train flat GS and then parameterized gaussians using gs_points). But for more information about transformation.jskn please take a look also here:

For hotdog we had already this tranfsformation.json from benchmark open dataset: https://drive.google.com/drive/folders/1JDdLGDruGNXWnM1eqY1FNL9PlStjaKWi

waczjoan / gaussian-mesh-splatting

How can i get transforms_train.json from my data #14