3-D Processing frame error?

[Definitely-A-Person]

Describe the bug In previous versions, when in the 3-D processing mode, the processed image view would display a box over the AprilTag. This is no longer happening for my team (see attached images). Is this a new system, and if so, is this how it is supposed to look? We think there is a bug that is causing the center of the coordinate frame to be something other than the center of the AprilTag. Additionally, distance measurements become less reliable with range.

To Reproduce Steps to reproduce the behavior:

1. Set camera resolution to 600*800, and calibrate using chessboard.
2. Set pipeline to AprilTag, 3-D processing mode, and show processed camera view.
3. Make sure AprilTag family is 36h11, but make no other adjustments to tuning. 4.Look at processed camera view in display while targeting an AprilTag to see error.

Screenshots / Videos

Platform:

• Hardware Platform: Orange Pi 5, Linux AARCH 64
• Network Configuration: Radio-> Network Switch-> Orange Pi
• PhotonVision Version: v2024.2.0
• Browser (with Version) (Chrome, Edge, Firefox, etc.): Chrome (120.0.6099.224)
• Camera(s) Used: Arducam OV9281(?)

[mcm001]

That's usually caused by poor camera calibration. Send your settings zip and also a screenshot of the calibration details pop-up for 800x600? (Click on it in the table in the calibration card)

[mcm001]

Or actually instead of settings zip just pull up that detail card and send me the cal JSON?

https://docs.photonvision.org/en/latest/docs/calibration/calibration.html under accessing calibration images

[Definitely-A-Person]

calibration JSON: photon_calibration_Arducam_OV9281_USB_Camera_1280x720.json Settings (in case needed): photonvision-settings-export.zip

[mcm001]

You said this was 600*800 but the json you attached is 1280x720?

Anyways clicking into the calibration details (see https://docs.photonvision.org/en/latest/docs/calibration/calibration.html#accessing-calibration-images ) I see very funny numbers. do these make any physical sense to you? they fail vibe check for me hard

[mcm001]

You could try again with mrcal turned off?

[mcm001]

Other notes:

• your camera is quite blurry -- try adjusting the lens, or taking pictures with the object further if its properly focused at infinity
• your object isnt flat, i can see it bubbling. this is not something that mrcal can model. try adhering it more flat. this wont fix this just more a general note.

[mcm001]

Final spam. your images are not very varied. review our calibration tips to more completely cover the imager and take pictures from more angles. dont be scared to take a -lot- of (still good quality and not blurry) photos -- up to 50 is fine

[mcm001]

i exported your calibration to images and ran it back through mrcal in python and got out much saner numbers. might be a bug in our code, maybe in the initial guess? great to have this dataset so we can keep digging offline.

# generated on 2024-01-20 12:20:34 with   /usr/bin/mrcal-calibrate-cameras --corners-cache corners.vnl --lensmodel LENSMODEL_OPENCV8 --focal 1200 --object-spacing 0.0254 --object-width-n 7 '*.png'
<snip>
    # intrinsics are fx,fy,cx,cy,distortion0,distortion1,....
    'intrinsics': [ 909.6867196, 903.2488663, 577.5179279, 340.1756753, -0.01338670833, -0.08025219906, 0.003925534879, -0.02027513374, 0.1065605664, -0.04406133662, -0.007898860056, -0.01494201441,],

[mcm001]

what happens if you edit the calibration json you sent me to have these numbers intead? does stuff look saner?

• intrinsics: [909.6867196, 903.2488663, 577.5179279, 340.1756753]
• distortion: [-0.01338670833, -0.08025219906, 0.003925534879, -0.02027513374, 0.1065605664, -0.04406133662, -0.007898860056, -0.01494201441]

[mcm001]

I just recalibrated using opencv and get these intrinsics, which still seem bad. At this point i suggest trying again, taking care to collect good calibration images.