search

strawlab / nextgen-camera-users

next generation camera software from the Straw Lab
1 stars 0 forks source link

unable to do intrinsic calibration with high-res cameras #11

Closed elhananby closed 3 years ago

elhananby commented 3 years ago

Hi everyone, As I mentioned I recently got a couple of new cameras, and whenever I try to do the intrinsic calibration I'm getting a flood of errors: "Channel full sending frame to process thread. Dropping frame data." This stops once I remove the checkerboard from the view of the camera. I assume this has something to do with the high-resolution of the cameras? I guess another option would be to reduce the resolution by using the binning option in Pylon, but then I would also have to do the extrinsic calibration with the same resolution (and the recording of course)?

Thanks

astraw commented 3 years ago

The problem is that the image processing is not running as fast as the frames are coming in. Typically when doing the checkerboard calibration this is no problem, but the code that prints that error does not know that. You can limit the maximum frame rate in the camera settings (near exposure duration and gain). If you slow down the incoming frames to no more than the rate at which they can be processed, the error messages should stop.

elhananby commented 3 years ago

The framerate I use for checkerboard calibration is already pretty low (5-10fps), but the errors still continue even when setting it to 1fps. By the way, can I ask what the Apriltag detection will be used for?

thanks

astraw commented 3 years ago

I just tested and I am also able to reproduce this with a a2A1920-160umBAS camera. However, it seems I can ignore the errors (and repeatedly dismiss the dialog that pops in the UI telling me about them) and successfully perform the actual calibration step. So, while this is certainly something I would like to fix (and thanks for the report), is this a blocker for you?

I think it is a useful, general-purpose feature for Strand-Cam and Braid to have April Tag detection built-in. We are starting to experiment with using april tags for camera calibration. This is working well so far. The data could also be used for a lot of different things beyond camera calibration.

elhananby commented 3 years ago

Yeah, I also started just ignoring the errors - then the calibration is really slow (~0.2fps), but it generally works. I did find that doing the calibration vis ROS camera_calibration is very quick, so perhaps it would be possible to perform the calibration externally and then save it in the same location as the Braid calibration?

astraw commented 3 years ago

Yes, that should be no problem at all. If I remember correctly, there is a ROS node (I forget which) that lets you replay a saved movie, so you could save an .mkv and then run the calibration script on it. The format of the saved file should be identical.

It is useful to know that with ROS is it so fast - I tried to reimplement exactly the ROS algorithm. So I can check where the difference lies. (Both Strand-Cam and ROS mainly call into OpenCV, which does most of the heavy lifting, and so it is surprising there is such a large difference.)

astraw commented 3 years ago

I discovered that my implementation was not complete and missing a key flag (to run fast) and a key step (to run the sub-pixel refinement) compared to ROS. I must have inadvertently left these out originally. But now I fixed this in https://github.com/strawlab/strand-braid/commit/a02b44329d98554a7a3fccea3c2fbb55472bf3d5. I have made a new release (0.9.0) which includes these fixes. The new release is in the newly-public strand-braid repo here.