search

3DOM-FBK / deep-image-matching

Multiview matching with deep-learning and hand-crafted local features for COLMAP and other SfM software. Supports high-resolution formats and images with rotations. Both CLI and GUI are supported.
https://3dom-fbk.github.io/deep-image-matching/
BSD 3-Clause "New" or "Revised" License
338 stars 40 forks source link

How to change the camera model from "SIMPLE_RADIAL" to "PINHOLE"? #28

Closed takafire2222 closed 7 months ago

takafire2222 commented 9 months ago

How can I reconstruct one sfm model by Pinpole in the deep-image-matching?

lcmrl commented 9 months ago

Hi, you can go in main.py and change

    if use_pycolmap:
        from src.deep_image_matching import reconstruction

        # Define database path and camera mode
        database = output_dir / "database_pycolmap.db"

in

    if use_pycolmap:
        from src.deep_image_matching import reconstruction

        # Define database path and camera mode
        database = output_dir / "database_pycolmap.db"
        camera_mode: pycolmap.CameraMode = pycolmap.CameraMode.SINGLE # pycolmap.CameraMode.AUTO

        # Define cameras
        cam0 = pycolmap.Camera(
            model="PINHOLE",
            width=1500,
            height=1000,
            params=[
                1500,
                1500,
                750,
                500,
            ],
        )

Finally change cameras = None in cameras = [cam0] params are fx, fy, cx, cy Let me know if you encounter any problem

takafire2222 commented 9 months ago

Sorry for asking again. I rewrote as described and successfully wrote the files (database.db, database_pycolmap.db, features.h5, matches.h5, pairs.txt). During conversion, the following error syntax appears as shown below. I'm sorry if this is a basic question, but if you have any ideas, could you please give me some advice?

2024-01-04 18:13:28 | [INFO ] Creating an empty database... 2024-01-04 18:13:28 | [INFO ] Importing images into the database... Traceback (most recent call last): File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 416, in main() File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 362, in main model = reconstruction.main( File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/reconstruction.py", line 145, in main update_cameras(database, cameras) File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/reconstruction.py", line 53, in update_cameras if not all([isinstance(cam, pycolmap.Camera) for cam in cameras]): TypeError: 'pycolmap.Camera' object is not iterable**

My corrections

if use_pycolmap: from src.deep_image_matching import reconstruction

    # Define database path and camera mode
    database = output_dir / "database_pycolmap.db"
    camera_mode: pycolmap.CameraMode = pycolmap.CameraMode.AUTO # pycolmap.CameraMode.AUTO

    # Define cameras
    cam0 = pycolmap.Camera(
        model="PINHOLE",
        width=1500,
        height=1000,
        params=[
            1500,
            1500,
            750,
            500,
        ],
    )

)

    #cameras = [cam0] # cameras = [cam1, cam2]
    cameras = cam0
lcmrl commented 9 months ago

Hi, try change cameras = cam0 to cameras = [cam0] should do the work

takafire2222 commented 9 months ago

Thank you for giving me the answer. I was able to export the "model" without any problems and no errors.

I have about 3 questions include bug reports, so please give me some advice.

  1. When I loaded it into Colmap and checked it with database management, the model name remained as SINPLE_RADIAL. (When I checked camera.txt, it was PINHOLE, so I think it was a bug that occurred when loading into Colmap. After that, when I performed 3D Gaussian Splatting, it was PINHOLE recognized, so I was able to perform Splatting.) However, the image was corrupted, failed. I think that This is probably the result of Splatting with SINPLE_RADIAL.)

  2. When I calculated the same number of sheets as above using 4K data, there was no problem with SIMPLE_RADIAL, but when setting it to PINHOLE, the following error occurred and stopped. , unfortunately I could not output it.

100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 154.27it/s] 54%|████████████████████▉ | 188/351 [00:00<00:00, 2584.70it/s] 2024-01-04 21:26:05 | [INFO ] Creating an empty database... 2024-01-04 21:26:05 | [INFO ] Importing images into the database... 2024-01-04 21:26:10 | [INFO ] Importing features into the database... 100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 734.88it/s] 2024-01-04 21:26:10 | [INFO ] Importing matches into the database... 54%|████████████████████▉ | 188/351 [00:00<00:00, 1942.57it/s] 2024-01-04 21:26:10 | [INFO ] Running 3D reconstruction... WARNING: Logging before InitGoogleLogging() is written to STDERR W20240104 21:26:52.518604 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.524106 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.601553 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.607028 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.612496 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.643158 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.743846 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.749533 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W20240104 21:26:52.754989 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. continue--------------------------------------------

3.When I tried to process 900 HD size movies, the following error occurred. No problem occurred with 100 sheets. Are there any restrictions on the number of sheets that can be processed?

2024-01-04 21:32:53 | [INFO ] Number of pairs: 430128 2024-01-04 21:32:53 | [INFO ] Extracting features with superpoint... 2024-01-04 21:32:53 | [INFO ] superpoint configuration: {'keypoint_threshold': 0.0001, 'max_keypoints': 4096, 'name': 'superpoint'} 100%|█████████████████████████████████████████| 928/928 [01:51<00:00, 8.34it/s] 2024-01-04 21:34:44 | [INFO ] Features extracted! 2024-01-04 21:34:44 | [INFO ] Matching features with lightglue... 2024-01-04 21:34:44 | [INFO ] lightglue configuration: {'depth_confidence': -1, 'filter_threshold': 0.5, 'flash': True, 'n_layers': 9, 'name': 'lightglue', 'width_confidence': -1} 2024-01-04 21:34:44 | [INFO ] Matching features... 2024-01-04 21:34:44 | [INFO ] 0%| | 0/430128 [00:00<?, ?it/s] Traceback (most recent call last): File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 416, in main() File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 237, in main match_path = img_matching.match_pairs(feature_path) File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/image_matching.py", line 311, in match_pairs matches = self._matcher.match( File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 279, in match self._matches = self._match_by_tile( File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 363, in _match_by_tile tile_pairs = self._tile_selection(img0, img1, method) File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 513, in _tile_selection res = self._preselction_matcher({"image0": feats0, "image1": feats1}) File "/home/denpozi/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, *kwargs) File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/thirdparty/LightGlue/lightglue/lightglue.py", line 463, in forward return self._forward(data) File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/thirdparty/LightGlue/lightglue/lightglue.py", line 474, in _forward kpts0 = normalize_keypoints(kpts0, size0).clone() File "/home/denpozi/.local/lib/python3.10/site-packages/torch/cuda/amp/autocast_mode.py", line 118, in decorate_fwd return fwd(args, **kwargs) File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/thirdparty/LightGlue/lightglue/lightglue.py", line 29, in normalize_keypoints size = 1 + kpts.max(-2).values - kpts.min(-2).values IndexError: max(): Expected reduction dim 1 to have non-zero size.

lcmrl commented 9 months ago
  1. Thanks for reporting I check. There are two databases in general, database.db and database_pycolmap.db. Are you using the second one? Can I suggest in the meantime to simply import database.db in COLMAP, change the camera model in the database with the GUI and run the reconstruction with the GUI. This will give you complete control on the process
  2. Have you accordenly modified the internals values for SIMPLE_RADIAL (f,cx,cy,k) and PINHOLE (fx,fy,cx,cy)?
takafire2222 commented 9 months ago

Thank you very much for your detailed and detailed explanation. The two problems were successfully resolved. Next time I would like to try 900 sequence data.

lcmrl commented 7 months ago

The remaining problem of processing large image sets (>1000 images) still needs to be solved. I close here, and keep open issue #Issue 29. Thanks for the feedback, feel free to open other issues or contribute to the repository.