Fisheye Calibration

[hautruong36]

@sabmah @Kekahuna

I have got an error with cv::initUndistortRectifyMap. The function definition is found on here:

http://docs.opencv.org/3.1.0/da/d54/group__imgproc__transform.html#ga7dfb72c9cf9780a347fbe3d1c47e5d5a&gsc.tab=0

This is how I define the variables

//New Camera Matrix. This is from output XML file cv::Mat cam(3, 3, cv::DataType::type); cam.at(0, 0) = 3.5506343877920330e+02f; cam.at(0, 1) = 0.0f; cam.at(0, 2) = 0.0f;

cam.at<float>(1, 0) = 0.0f;
cam.at<float>(1, 1) = 3.5506343877920330e+02f;
cam.at<float>(1, 2) = 0.0f;

cam.at<float>(2, 0) = 3.1950000000000000e+02f;
cam.at<float>(2, 1) = 2.3950000000000000e+02f;
cam.at<float>(2, 2) = 1.0f;

Mat R, map1, map2;

cv::initUndistortRectifyMap(cameraMatrix, distCoeffs, R, cam, Size(640, 480), CV_16SC2, map1, map2);

//cameraMatrix and disCoeffs are predefined. Size(640, 480) from output XML.

I cannot get pass this function due to an error. What am I doing wrong? Code is uploaded on github.

[bhunt2]

What does the error say? Or better yet, you can upload the compile/run output for it. It would be good to have all of this documented in the Wiki rather here in the issues. This is a place where we can discuss the issues, but the primary location for all documentation of design, procedures, and issue resolution should be in the wiki. Since this is camera related, you can add another page in the wiki linked to in the Camera page or something like that. It's up to you guys how you organize the documentation, but think of the Wiki as your final report.

[ghost]

I am still putting together opencv on my new system at home, but in the mean time I notice in the comment you reference (CameraCalib.cpp:459, commit https://github.com/bhunt2/QC1.0/commit/80757c2d75062bc8493e33e082204a5995529f29):

Mat R, map1, map2;

cv::initUndistortRectifyMap(cameraMatrix, distCoeffs, R, cam, Size(640, 480), CV_16SC2, map1, map2);

but the code has:

Mat R, map1, map2;
cv::Mat src;
src = (argv[1], 1);
//map1 = cvCreateImage(cvGetSize(src), IPL_DEPTH_16S, 1);

cv::initUndistortRectifyMap(cameraMatrix, distCoeffs, Matx33d::eye(), cam, Size(640, 480), CV_16SC2, map1, map2);

where the matrix parameter R is replaced by Matx33d::eye() in the call to cv::initUndistortRectifyMap().

I didn't know if this was relevant, but I figured I would verify which you are trying to get to work while I am gettng my build environment on this new laptop ready.

[ghost]

For documentation sake the error I am seeing thrown by the call to initUndisortRectifyMap() from opencv is:

OpenCV Error: Assertion failed (A.size() == Size(3,3) && A.size() == R.size()) in initUndistortRectifyMap, file /home/vexille/Builds/opencv-3.1.0/modules/imgproc/src/undistort.cpp, line 102
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/vexille/Builds/opencv-3.1.0/modules/imgproc/src/undistort.cpp:102: error: (-215) A.size() == Size(3,3) && A.size() == R.size() in function initUndistortRectifyMap

I finally got opencv 3.1 up and running on my distribution (had 2.4 previously); I'm trying to look in to why this is the case in the code, but as the assertion suggests there may be an issue with the dimensions of the matrix parameters being used.

[hautruong36]

I have calibrated fisheye camera successfully. I had to redefine the captured frames to be able to calibrate without error, and also if there are many frames, it will take a long time to calibrate. I set it to be 30 frames.

Here is the link to the demo: https://tree.taiga.io/project/sabmah-object-following-drone/us/86?milestone=63318

[ghost]

Cool good job!

[bhunt2]

@hautruong36 @sabmah @Kekahuna @spesialstyrker

This is really great! I am sure you had a fun time figuring out that resolution. That is one I found useful myself when working on feature tracking of humans and robots. Please update the project in Taiga and add appropriate documentation to the wiki.

You have added the good information about the cameras and the design choice of using the USB camera, but nothing about your work with camera calibration. Please add documentation about the algorithm/code used to accomplish the calibration and about the issues you encountered.

One thing I noticed during the calibration is that the camera was not completely stable. Think about how this may affect the calibration process. Do you need to have different calibration settings for different environments? Do the calibration settings change dependent on the lighting when calibration is accomplished? How many frames per second do you get before and after the calibration process? This tells you how cumbersome the continuous flattening of the image is. Also, think about how you might store the calibration settings into the system as you will not want to calibrate the camera every time you start up the drone.

I hope these questions help you start thinking about other things you might want to consider. Again, great work on this @hautruong36.

[hautruong36]

The image wasn't stable after calibrated. I think it's due the ambient light. I have had a better output when I had a brighter environment. I has been cloudy here in Oregon and the calibration was done in my room. I haven't done any research on how to get camera FPS in Visual Studio. I have used an online resource to test the camera FPS with almost the same ambient light. It varied from 20 fps to 30 fps.

https://www.onlinemictest.com/webcam-test-in-adobe-flash

I have thought about storing the calibration settings, so there is no need to calibrate everytime we use it. I might have to reopen this issue if it's going to be related to remap functions that I had problem with.

I haven't done much documentation for camera, but I am planning to do them after the finals. I have tried to work on all of my tasks early last week, so I can focus on finals. I will try to update camera documentation camera in wiki section as soon as I can.