automated robot moving to collect data for camera in hand configuration and ROS based api

[Ruthrash]

usage: python3 ROSCameraRobotCalibration.py --move_robot_automatically True --use_delta_poses False/True ....other args...

[n-yoshikawa]

I tested the calibration script.

What I did

1. Start bash script

   cd /root/git/frankapy
   bash ./bash_scripts/start_control_pc.sh -i 172.16.0.4 -u sf3202inuc -d /root/git/franka-interface -a 172.16.0.2 -w 172.16.0.3

2. Start camera based on a previous comment at https://github.com/pairlab/franka_arm_infra/issues/22#issuecomment-1498034105

   roslaunch realsense2_camera rs_camera.launch filters:=pointcloud enable_infra1:=false enable_infra2:=false

3. Start end-effctor tf (for customized gripper)

   cd ~/git/frankapy/catkin_ws/src/franka_control
   python3 ee_robotiq_transformation.py

4. Start calibration

   python3 ROSCameraRobotCalibration.py --move_robot_automatically True --robot_base_frame_name /panda_link0 --ee_frame_name /virtual_eef_link --rgb_image_topic /camera/color/image_raw --rgb_camera_info_topic /camera/color/camera_info

Result

• The robot moved automatically
• The camera looked working because reprojection error was calculated
• The automatic pose was not suitable for our custom gripper that is parallel to table
• Problem 1: reprojection error was too high at some poses (maybe because I could not take enough distance between gripper and markers)

  reprojection error 0.4223259860582057
  #### Very high reprojection error ####
  #### Ignoring this sample ####

• Problem 2: ArUco tag was not detected in many poses because camera was not oriented to the markers.

  No aruco tag detected in this sample
  accepted pairs of pose, no. of frames processed 0 4

• As a result, I could not get a calibration value and evaluate its correctness.

Setup

[n-yoshikawa]

after truying the autoamtic calibration from the reset_arm confgituration, we had the following observations:

• some times the robot ee was not pointing toward the borad, i.e., maybe those intermediate waypoints should be updated.
• We tried two times and in both times the robot reached a joint limit and therefore stopped moving. here is a photo of the robot in joint limit:
• it would be nice to print the following as a user feedback while running automatic trajectory: waypoint X/Y so that the user knows when to expect to finish the data collection.

---

images of two scenarios robot reacing joint limits:

[n-yoshikawa]

as we discussed we started the robot from another configuration, as shown below, and right after starting the robot goes to initial reset configuration and starts moving. So, regardless of initial configuration set by user, the robot always goes to reset configuration. However this is different from what we have discussed in the meeting and we were expecting, but we think it is safer to be this way, since the robot motion is quite wide. So, for different robot, maybe it is safer the user moves the robot ee.

---

initial configuration:

[n-yoshikawa]

There should be a photo with inidication of distances from the robot for the user, so that they know where to place the apriltag board.