search

nianticlabs / ace

[CVPR 2023 - Highlight] Accelerated Coordinate Encoding (ACE): Learning to Relocalize in Minutes using RGB and Poses
https://nianticlabs.github.io/ace
Other
359 stars 34 forks source link

Support principle point intrinsic parameters in training #15

Closed daniel-sudz closed 1 year ago

daniel-sudz commented 1 year ago 
    # Create the intrinsics matrix.
    intrinsics = torch.eye(3)
    intrinsics[0, 0] = focal_length
    intrinsics[1, 1] = focal_length
    # Hardcode the principal point to the centre of the image.
    intrinsics[0, 2] = image.shape[2] / 2
    intrinsics[1, 2] = image.shape[1] / 2

https://github.com/nianticlabs/ace/blob/2507cdb01146898b011e3e91e716d7399a533c5c/dataset.py#L489

Assuming I'm understanding this correctly, would be happy to make a PR for this. We are looking to train from Arkit IOS data where the optical center can vary from frame-to-frame as a result of some distortion correction that apple seems to be doing.

tcavallari commented 1 year ago

Hi, thanks for bringing it up. We'd be happy to accept a PR that introduced support for reading the full intrinsics (fx,fy, cx, cy) from the calibration files.

I think this line would need to be changed to read either a single value (to continue supporting the paper datasets that only provide a single focal length), or 4 values.

After that, the changes to the rest of the code should be reasonably straightforward.

tcavallari commented 1 year ago

Thanks for the contribution.

Mingyuan1997 commented 1 year ago

Thanks for the contribution. Could you share some suggetions how we can get a posed RGB video frame from IOS and merge with this repo?

Best