Intrinsic format?

[antithing]

Hi, and thanks for making this code available. What format are intrinsics calculated in? Fx, fy, cx,cy and distortion?

Thanks!

[sarlinpe]

https://github.com/cvg/GeoCalib/blob/917ebb94543aa25291a07d768566a86ff2d38614/geocalib/camera.py#L27-L31

The principal point is not estimated and instead assumed to be at the center of the image, but this can be overwritten if different. The distortion parameters depend on the selected camera model.

[antithing]

Thanks! Is there a way to calculate the principal point as well?

Does camera_model="simple_radial" give a 5 parameter matrix for distortion?

[sarlinpe]

Is there a way to calculate the principal point as well?

We always assume that the principal point is at the center of the image but it should be possible to set a different value if needed: https://github.com/cvg/GeoCalib/blob/89182c053841d6463efda6c93e5b8532d519624e/geocalib/lm_optimizer.py#L46-L49 https://github.com/cvg/GeoCalib/blob/917ebb94543aa25291a07d768566a86ff2d38614/geocalib/camera.py#L58 We haven't tried to optimize the principal point and I would bet that it is currently not sufficiently constrained.

Does camera_model="simple_radial" give a 5 parameter matrix for distortion?

https://github.com/cvg/GeoCalib/blob/917ebb94543aa25291a07d768566a86ff2d38614/geocalib/camera.py#L542-L546

[antithing]

Thanks! I am running this code:

from geocalib import GeoCalib
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = GeoCalib(weights="distorted")  # default is "pinhole"

# load image as tensor in range [0, 1] with shape [C, H, W]
img = model.load_image("assets/query.jpg").to(device)
result = model.calibrate(img, camera_model="simple_radial")  # or pinhole, simple_divisional

print("camera:", result["camera"])
print("gravity:", result["gravity"])

and I get the error:

Traceback (most recent call last):
  File "D:\Calibration\GeoCalib-main\run.py", line 9, in <module>
    result = model.calibrate(img, camera_model="simple_radial")  # or pinhole, simple_divisional
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\Calibration\GeoCalib-main\geocalib\extractor.py", line 109, in calibrate
    out = self.model(img_data | prior_values)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Calibration\GeoCalib-main\geocalib\geocalib.py", line 110, in forward
    features = {"hl": self.backbone(data)["features"], "ll": self.ll_enc(data)["features"]}
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Calibration\GeoCalib-main\geocalib\modules.py", line 568, in forward
    x, H, W = patch_embed(x)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Calibration\GeoCalib-main\geocalib\modules.py", line 368, in forward
    x = self.proj(x)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\container.py", line 215, in forward
    input = module(input)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\conv.py", line 460, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "C:\Users\B\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\conv.py", line 456, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

[veichta]

It seems that the model is not moved to the GPU. Make sure that you are also moving the model to the correct device:

model = GeoCalib(weights="distorted")

should be

model = GeoCalib(weights="distorted").to(device)

[antithing]

That was it. Thank you!