PR2 eye to base calibration - Single Sensor

[manuelgitgomes]

Hello @miguelriemoliveira!

I am trying to do an eye to base calibration on the PR2.

For that I recorded a bag file with the pattern on the hand of the PR2 and with only a single camera and moved it around. It generated this ATOM dataset: train_20231114.tar.gz

I thought everything was fine with this calibration, but unfortunately I did not fix the pattern, which it is (w.r.t. the end-effector).

Some things in ATOM seem to not align that much with eye to base calibration.

One example is this one: https://github.com/lardemua/atom/blob/30aba658f3a8c9f184e2a7fb2987b2ffeb5df528/atom_calibration/src/atom_calibration/calibration/visualization.py#L479-L491

In this block of code, we can see that the pattern visualization is not correctly done, as only one is rendered.

Another thing that does not seem right to me is the calibration results. The final residual table is:

+------------+-----------------------+                                                                                                                                             
| Collection | azure_rgb_camera [px] |                                                                                                                                             
+------------+-----------------------+                                                                                                                                             
|    000     |    15.1885 (0.5702)   |                                                                                                                                             
|    001     |    3.2207 (0.1209)    |                                                                                                                                             
|    002     |    10.9150 (0.4098)   |                                                                                                                                             
|    003     |    21.7006 (0.8147)   |                                                                                                                                             
|    004     |    30.3252 (1.1385)   |                                                                                                                                             
|    005     |    7.8314 (0.2940)    |                                                                                                                                             
|    006     |    9.1158 (0.3422)    |                                                                                                                                             
|    007     |   182.9993 (6.8703)   |                                                                                                                                             
|    008     |    9.6382 (0.3618)    |                                                                                                                                             
|    009     |    9.9364 (0.3730)    |                                                                                                                                             
|    010     |    16.1682 (0.6070)   |                                                                                                                                             
|    011     |    15.5045 (0.5821)   |                                                                                                                                             
|    013     |    10.7652 (0.4042)   |                                                                                                                                             
|    014     |    11.4116 (0.4284)   |                                                                                                                                             
|    015     |    11.8892 (0.4464)   |                                                                                                                                             
|    016     |    12.9904 (0.4877)   |                                                                                                                                             
|    017     |    7.4318 (0.2790)    |                                                                                                                                             
|    018     |    6.3122 (0.2370)    |                                                                                                                                             
|    019     |    10.9426 (0.4108)   |                                                                                                                                             
|    020     |    4.0670 (0.1527)    |                                                                                                                                             
|    021     |    6.8273 (0.2563)    |                                                                                                                                             
|    022     |    8.6981 (0.3265)    |                                                                                                                                             
|    023     |    10.1374 (0.3806)   |                                                                                                                                             
|    024     |    7.0010 (0.2628)    |                                                                                                                                             
|    025     |    8.4502 (0.3172)    |                                                                                                                                             
|    026     |    9.6708 (0.3631)    |                                                                                                                                             
|    027     |    6.7554 (0.2536)    |                                                                                                                                             
|    028     |    90.5383 (3.3990)   |                                                                                                                                             
|    029     |   148.0577 (5.5585)   |                                                                                                                                             
|    030     |    15.2118 (0.5711)   |                                                                                                                                             
|    031     |    17.2025 (0.6458)   |                                                                                                                                      
|    032     |    9.9321 (0.3729)    |
|    033     |   117.7812 (4.4218)   |
|    034     |    7.8145 (0.2934)    |
|    035     |    7.2236 (0.2712)    |
|    036     |    19.6845 (0.7390)   |
|    037     |    9.5812 (0.3597)    |
|    038     |    12.5422 (0.4709)   |
|    040     |    6.1946 (0.2326)    |
|    041     |    4.4140 (0.1657)    |
|    042     |    6.0199 (0.2260)    |
|    043     |    6.1293 (0.2301)    |
|    044     |    21.7106 (0.8151)   |
|    045     |    16.9495 (0.6363)   |
|    047     |    17.5260 (0.6580)   |
|    049     |    15.4382 (0.5796)   |
|    050     |    4.6917 (0.1761)    |
|    051     |    6.8912 (0.2587)    |
|  Averages  |    21.4048 (0.8036)   |
+------------+-----------------------+

So, they do not look good. However, these values do not seem to correlate with the images seen. As an example, collection 7:

And collection 29:

I think it would be better to "dumb down" the system and create a fully simulated system to test eye-to-base calibration. Do you have any suggestions?

(tagging @v4hn for visibility)

[miguelriemoliveira]

Hi @manuelgitgomes ,

a few comments:

Which are way better than the previous ones with the head moving and even comparable to the ones Pradeep achieved (0.5 pixels in 480p camera, while we have a 2K camera).

I see your point, we can talk about the error as a percentage of the image' resolution. Still I wonder why we cannot get better results. I still fell like there is a bug somewhere ...

Are you correcting the joint_bias of the head joints as well? What are the magnitudes of the estimated joint bias parameters? Are they 10 degrees, 0.1 degrees? Can you put the output of --print_parameters here?

When running the same evaluation with the calibrated dataset as test dataset as well:

I think the evaluation was not updated to function with joint calibration. So the good news is that these evaluation results are to be discarded. I created #816 for this.

In the objective function we do this:

https://github.com/lardemua/atom/blob/6220346162039f19c432c72b10422397ab44d698/atom_calibration/src/atom_calibration/calibration/objective_function.py#L389-L405

to get the new transformations obtained from the calibrated joint parameters. I think that the evaluation is not doing that, is it? Perhaps we can create an issue for this.

My suggestion is to address this after removing the joint_position_bias parameter, since that will probably make a lot of changes ... (done in #809)

[manuelgitgomes]

Hello @miguelriemoliveira

Still I wonder why we cannot get better results. I still fell like there is a bug somewhere ...

If it makes you more confident, I can run the error propagation study I created for these specific joint states and verify if the errors are bellow the maximum error, which would suffice, right?

Are you correcting the joint_bias of the head joints as well? Yes

What are the magnitudes of the estimated joint bias parameters? Are they 10 degrees, 0.1 degrees? Can you put the output of --print_parameters here?

joint-head_pan_joint-origin_yaw                                       joint-head_pan_joint-origin_yaw  0.000000 -0.002940 -0.002940 -inf  inf
joint-head_tilt_joint-origin_pitch                                 joint-head_tilt_joint-origin_pitch  0.000000  0.003444  0.003444 -inf  inf
joint-l_elbow_flex_joint-origin_pitch                           joint-l_elbow_flex_joint-origin_pitch  0.000000  0.013452  0.013452 -inf  inf
joint-l_forearm_roll_joint-origin_roll                         joint-l_forearm_roll_joint-origin_roll  0.000000 -0.039106 -0.039106 -inf  inf
joint-l_shoulder_lift_joint-origin_pitch                     joint-l_shoulder_lift_joint-origin_pitch  0.000000 -0.034454 -0.034454 -inf  inf
joint-l_shoulder_pan_joint-origin_yaw                           joint-l_shoulder_pan_joint-origin_yaw  0.000000 -0.002633 -0.002633 -inf  inf
joint-l_upper_arm_roll_joint-origin_roll                     joint-l_upper_arm_roll_joint-origin_roll  0.000000 -0.007963 -0.007963 -inf  inf
joint-l_wrist_flex_joint-origin_pitch                           joint-l_wrist_flex_joint-origin_pitch  0.000000  0.016567  0.016567 -inf  inf
joint-l_wrist_roll_joint-origin_roll                             joint-l_wrist_roll_joint-origin_roll  0.000000 -0.047547 -0.047547 -inf  inf

So the errors go from very small (-0.002940 [~-0.17º] radians for the head pan) to considerably larger (-0.047547 [~-2.73º] radians for the wrist roll)

I think the evaluation was not updated to function with joint calibration. So the good news is that these evaluation results are to be discarded. I created #816 for this.

Will work on that.

My suggestion is to address this after removing the joint_position_bias parameter, since that will probably make a lot of changes ... (done in #809)

I have done that already!

[miguelriemoliveira]

Hi @manuelgitgomes ,

If it makes you more confident, I can run the error propagation study I created for these specific joint states and verify if the errors are bellow the maximum error, which would suffice, right?

It would help, yes. But my feeling is more empirical, we calibrated a lot of systems already, why did we never encounter these problems?

[miguelriemoliveira]

So the errors go from very small (-0.002940 [-0.17º] radians for the head pan) to considerably larger (-0.047547 [-2.73º] radians for the wrist roll)

which is consistent with what you told me your impression would be, that the head joints should be much more accurate.

Do you know the resolution of the encoders in the joints?

[manuelgitgomes]

It would help, yes.

Will work on that.

But my feeling is more empirical, we calibrated a lot of systems already, why did we never encounter these problems?

We never done this with this high of a resolution...

Do you know the resolution of the encoders in the joints?

No idea, honestly... When I turn on the robot again I can tell you.