search

EgalYue / Mobile_Marker_Based_Navigation

2 stars 3 forks source link

cam distrbution using colormap; R_error;t_error #7

Open EgalYue opened 6 years ago

EgalYue commented 6 years ago

cam distribution on YZ plane using matplotlib colormap:

cam_distrbution_colormap

raultron commented 6 years ago

I am wondering why the condition number is asymmetric, it is better on Y+. Rotate the camera around its Z axis and repeat the test to see if yaw angle has an influence.

EgalYue commented 6 years ago

I think, the reason for condition number asymmetric is: the image points of symmetrical cam position are different. And i tested for 2 cam postions. cam position: [ 0., 0.40957603, 0.28678823] image points: [[ 512.6557029 127.3442971 638.19502511 1.80497489] [ 129.49722282 129.49722282 422.5091646 422.5091646 ] [ 1. 1. 1. 1. ]] cond num : 6562.11683517 image_points-2

cam position: [ 0. , -0.40957603 , 0.28678823] image points: [[ 638.19502511 1.80497489 512.6557029 127.3442971 ] [ 57.4908354 57.4908354 350.50277718 350.50277718] [ 1. 1. 1. 1. ]] cond num: 4368.18626759 image_points-1

EgalYue commented 6 years ago

And cam distribution after fixing the lookat method, we find that compared to Y axis,in X axis only a few cam positions are valid. cam_distribution

xy_circle_cam_dis_valid

raultron commented 6 years ago

"in X axis only a few cam positions are valid." What do you mean by "valid"?. I think that you have an error, when you move a REAL camera in front of a marker you have way more valid positions around the marker, the field of view in X is lower than in Y but still is not like what you have in the figures.

EgalYue commented 6 years ago

I also considered this issue once time. I used 4 cam positions(2 cams on Y axis, 2 cams on X axis) to test. I found that for the same height, after using Lookat method we got different R, and t for each cam position. The cam position which on X axis after projecting the object points, the image points are out of the bound of image: https://github.com/EgalYue/Mobile_Marker_Based_Navigation/blob/9b3222ffedd7a6619a34a0c430cd77b61bbe1a9e/python/vision/camera_distribution.py#L395 https://github.com/EgalYue/Mobile_Marker_Based_Navigation/blob/9b3222ffedd7a6619a34a0c430cd77b61bbe1a9e/python/vision/camera_distribution.py#L396 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 .camera on X axis, This cam position is not valid: cam position : [ 2.12132025 0. 2.12132025] cam R: [[-0.70710678 0. -0.70710678 0.] [ 0. 1. 0. 0. ] [ 0.70710678 0. -0.70710678 0.] [ 0. 0. 0. 1. ]] image points are out of bound: [[ -2.31074160e+04 2.21474160e+04 -2.31074160e+04 2.21474160e+04] [ 1.37137085e+03 -8.91370850e+02 -8.91370850e+02 1.37137085e+03] [ 1.00000000e+00 1.00000000e+00 1.00000000e+00 1.00000000e+00]]

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2.camera on X axis, This cam position is not valid: cam position : [ -2.12132025e+00 2.59786805e-16 2.12132025e+00] cam R: [[ -7.07106781e-01 6.12323399e-17 7.07106781e-01 0.00000000e+00] [ 0.00000000e+00 1.00000000e+00 -8.65956055e-17 0.00000000e+00] [ -7.07106781e-01 -6.12323399e-17 -7.07106781e-01 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]] image points are out of bound: [[ -2.15074160e+04 2.37474160e+04 -2.15074160e+04 2.37474160e+04] [ -8.91370850e+02 1.37137085e+03 1.37137085e+03 -8.91370850e+02] [ 1.00000000e+00 1.00000000e+00 1.00000000e+00 1.00000000e+00]]

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3.camera on Y: cam position : [1.29893402e-16 2.12132025e+00 2.12132025e+00] cam R: [[ -1.00000000e+00 -4.32978028e-17 -4.32978028e-17 0.00000000e+00] [ 0.00000000e+00 7.07106781e-01 -7.07106781e-01 0.00000000e+00] [ 6.12323399e-17 -7.07106781e-01 -7.07106781e-01 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]] [ 1.29893402e-16 2.12132025e+00 2.12132025e+00 1.00000000e+00]

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.camera on Y: cam position : [ -3.89680234e-16 -2.12132025e+00 2.12132025e+00] cam R: [[ -1.00000000e+00 -1.29893417e-16 1.29893417e-16 0.00000000e+00] [ 0.00000000e+00 7.07106781e-01 7.07106781e-01 0.00000000e+00] [ -1.83697032e-16 7.07106781e-01 -7.07106781e-01 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

EgalYue commented 6 years ago

Display R_Error and tError of cam distribution on YZ plane for cv2.SOLVEPNP_DLS, cv2.SOLVEPNP_EPNP,cv2.SOLVEPNP_ITERATIVE_ Using 2 ways to display(1. using colormap; 2. using 3D): ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ cv2.SOLVEPNP_DLS: r_error_and_t_error r_error_and_t_error_3d ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ cv2.SOLVEPNP_EPNP: r_error_and_t_error r_error_and_t_error_3d ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ cv2.SOLVEPNP_ITERATIVE r_error_and_t_error r_error_and_t_error_3d

raultron commented 6 years ago

Something is wrong Yue. It is not possible that you have such a big restriction in the field of view, that simply doesn't happen with a real camera. Please check, again, if it is necessary you would have to compare against real results from a real camera to compare.

raultron commented 6 years ago

I have an idea, you can test what is wrong by creating a camera that has an image with the same width and height (a square camera), then you should have valid camera poses completely symmetric, if you don't, then there is a problem somewhere.

EgalYue commented 6 years ago

Finally finally finally!!! I found the error, i make a really big mistake: The R rotation matrix in camera.py is from camera coordinate to world coordinate, not from world to camera coordinate! So your ""Lookat" method is totally right! inv(R) = R.T

cam_dis

So all the results above are wrong! tomorrow i will run the simulations again,it will take some time

EgalYue commented 6 years ago

Now I find out, why the condition number is asymmetric:

According to the paper, c(A) = s1 / s8 and A is composed of 4 object points and 4 image points, we know that the object points always be origin(constant), so the only affected factor is 4 image points. So we can see what is happened by image points: Different cam position(Symmetrical cam position) on YZ plane has different R. And we can get different image points according to R.

The Symmetrical cam positions, which are closer to Z-axis (theta = 90) and Y-axis (theta = 0), get almost similar image points -> so similar condition number (smaller difference). At somewhere theta between 0 - 90, we get the biggest difference of condition number on Symmetrical cam positions. use

theta = 0, 180, image points exactly same 1 19

theta = close to 90, image points exactly very similar 9 11

theta = between 0 - 90, image points big difference 5 15

EgalYue commented 6 years ago

The new simulation results: cam distribution: figure_1 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ DLS: r_error_and_t_error r_error_and_t_error-1 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ EPNP: r_error_and_t_error r_error_and_t_error-1 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ITERATIVE: r_error_and_t_error r_error_and_t_error-1

EgalYue commented 6 years ago

normalize points: condition number cam distribution figure_1

EgalYue commented 6 years ago

I just find a problem, when i focused on the both following images: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ without normalization: it seems not exactly right when the Angle changes, cam_dis ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ with normalization: Do not work for the Height figure_1

And now im thinking to use the Taubin estimator method instead of normalization. Im not sure about this, maybe need to discuss this problem with you.

EgalYue commented 6 years ago

normalised Homography without scale:

https://github.com/EgalYue/Mobile_Marker_Based_Navigation/blob/795f95504467dbac0daec1cab555f6be6d67ca1c/python/CondNum_TheoryValidation/hT_gradient.py#L305

the scale should be set as 1 : scale = 1 BUT in this case the condition number distribution seems not right! After reading Multiple View Geometry in Computer Vision Second Edition Page 107,108,there is a explain for Why is normalization essential? I think if we do not use scale(just set scale = 1), we can not get the purpose of the normalization.

nor_withoutscale ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ And in a alternative way we already know that , when the cam height increases, the cond num can get bigger! So i set the *_scale = radiusnp.sqrt(2) / meandist_**, which means for different height(distance from camera to marker) we set different scale. But im not sure whether we can do like this. And following is the distribution

https://github.com/EgalYue/Mobile_Marker_Based_Navigation/blob/795f95504467dbac0daec1cab555f6be6d67ca1c/python/CondNum_TheoryValidation/hT_gradient.py#L305

nor_radius

raultron commented 6 years ago

Hi Yue, are you applying the same normalization function both to Image Coordinates and Object Coordinates?. Try to apply the modified normalization (with scale=1) only to image coordinates and use the regular normalization for object coordinates.

EgalYue commented 6 years ago

I make simulation for both cases again, for both case i apply image points with modified normalization (with scale=1) :

  1. object points with regular normalization(mean distance from the origin is sqrt(2)) https://github.com/EgalYue/Mobile_Marker_Based_Navigation/blob/9c89a0c34b19a6d5452f2ea63fe02c82b807c9dd/python/CondNum_TheoryValidation/hT_gradient.py#L71 object_nor

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  1. object points without normalization (also called modified normalization scale = 1)

object_without_nor