Feature Request: Additional Statistics for the Reprojection Error

[DanielSimsTOA]

Hi!

First, I want to express my appreciation for mrcal—it's been an invaluable tool for my projects.

I have a feature request concerning the reprojection error statistics provided by mrcal. Currently, I'm trying to obtain detailed statistics like the minimum, maximum, mean, and standard deviation of the reprojection errors directly through the Python API. While mrcal-calibrate-cameras provides some of this data, I've noticed discrepancies when attempting to calculate these statistics manually using the API. I'm pretty sure I am doing something wrong.

Here is the code snippet I've been working with:

import mrcal
import numpy as np

# Load the camera model and obtain optimization inputs
model = mrcal.cameramodel("camera-0.cameramodel")
optimization_inputs = model.optimization_inputs()

# Calculate reprojection error statistics from existing camera model. 
x_chessboard = mrcal.residuals_chessboard(optimization_inputs=optimization_inputs).ravel()

print("Standard deviation of reprojection errors:", np.std(x_chessboard))
print("Mean of reprojection errors:", np.mean(x_chessboard))
print("Root mean square of reprojection errors:", np.sqrt(np.mean(x_chessboard**2)))
print("Maximum reprojection error:", np.max(x_chessboard))
print("Minimum reprojection error:", np.min(x_chessboard))

Is it possible to enhance mrcal to natively support these statistical computations? Am I missing something with the Python API?

Having these directly available would be beneficial to my project.

Thanks for considering this enhancement. Please let me know if you need any more details or if there's a workaround I might have missed.

Best Regards, Daniel Sims

[dkogan]

Hello. I'm glad you're finding mrcal useful.

There are two nuances in the measurement vector you should be aware of. These are documented: https://mrcal.secretsauce.net/docs-2.4/formulation.html#orgb634d5f

• The bulk of the measurement vector is composed of WEIGHTED reprojection errors

• The measurement vector also contains regularization terms, unrelated to each individual pixel observation

If you're getting numbers that don't match up with what you expect, you should double-check how you're handling both of these.

Furthermore, your question pointed out to me that a number of user-facing functions and tools (for instance mrcal.residuals_board() and mrcal-show-residuals) are misleadingly-named: these use MEASUREMENTS, not RESIDUALS (i.e. the results are weighted). Most observations have weight==1, so these are the same, but less-certain observations might have lower weights. I'll clarify the naming in the next release.

To visualize the distribution of the measurements (NOT the residuals) you can run

mrcal-show-residuals --histogram

So I don't know if you actually want residuals or measurements (usually measurements are what you want). And I don't know if you do or do not want to take regularization terms into account.

Does the above solve your problem? If not, can you clarify what kind of extra interface you'd like to see? You want new API functions? Or more things returned by mrcal.optimize()?

Thanks