Data collector crashes at wating for message

[miguelriemoliveira]

Using rgb_rgb_system with command

roslaunch rgb_rgb_system_calibration collect_data.launch output_folder:=$ATOM_DATASETS/rgb_rgb_system/dataset1 overwrite:=true

We get an error like this:

left_camera:
{'_name': 'left_camera', 'modality': 'rgb', 'parent': 'rgb_left_optical_frame', 'calibration_parent': 'tripod_left_support', 'calibration_child': 'rgb_left_link', 'topic': '/rgb_left/image_raw', 'msg_type': 'Image', 'camera_info_topic': '/rgb_left/camera_info', 'camera_info': {'header': {'seq': 38, 'stamp': {'secs': 2559, 'nsecs': 699000000}, 'frame_id': 'rgb_left_optical_frame'}, 'height': 800, 'width': 1200, 'distortion_model': 'plumb_bob', 'D': [0.0, 0.0, 0.0, 0.0, 0.0], 'K': [1039.2275459834757, 0.0, 600.5, 0.0, 1039.2275459834757, 400.5, 0.0, 0.0, 1.0], 'R': [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0], 'P': [1039.2275459834757, 0.0, 600.5, -0.0, 0.0, 1039.2275459834757, 400.5, 0.0, 0.0, 0.0, 1.0, 0.0], 'binning_x': 0, 'binning_y': 0, 'roi': {'x_offset': 0, 'y_offset': 0, 'height': 0, 'width': 0, 'do_rectify': False}}, 'chain': [{'key': 'world-tripod', 'parent': 'world', 'child': 'tripod'}, {'key': 'tripod-tripod_plate', 'parent': 'tripod', 'child': 'tripod_plate'}, {'key': 'tripod_plate-tripod_left_support', 'parent': 'tripod_plate', 'child': 'tripod_left_support'}, {'key': 'tripod_left_support-rgb_left_link', 'parent': 'tripod_left_support', 'child': 'rgb_left_link'}, {'key': 'rgb_left_link-rgb_left_optical_frame', 'parent': 'rgb_left_link', 'child': 'rgb_left_optical_frame'}]}
Waiting for message /rgb_right/image_raw ...
1
================================================================================REQUIRED process [collect_data-8] has died!
process has died [pid 27145, exit code -11, cmd /home/mike/workspaces/catkin_ws/devel/lib/atom_calibration/collect_data -s 0.5 -o /home/mike/datasets/rgb_rgb_system/dataset1 -c /home/mike/workspaces/catkin_ws/src/calibration/atom/atom_examples/rgb_rgb_system/rgb_rgb_system_calibration/calibration/config.yml -ssl lambda sensor_name: False --overwrite __name:=collect_data __log:=/home/mike/.ros/log/010b3084-6233-11ee-aa6d-8fc72bb7338e/collect_data-8.log].
log file: /home/mike/.ros/log/010b3084-6233-11ee-aa6d-8fc72bb7338e/collect_data-8*.log
Initiating shutdown!
================================================================================

We traced the error here:

https://github.com/lardemua/atom/blob/ebde5c8bb7e76983760f91381f52cdceb547ff2a/atom_calibration/src/atom_calibration/collect/data_collector_and_labeler.py#L100-L102

but that does not make sense because its a ros function ...

[miguelriemoliveira]

Taging @Kazadhum , @brunofavs , @manuelgitgomes for visibility.

[miguelriemoliveira]

We traced the problem into the aruco detector, in particular here:

https://github.com/lardemua/atom/blob/ebde5c8bb7e76983760f91381f52cdceb547ff2a/atom_calibration/src/atom_calibration/collect/patterns.py#L100-L106

Its possible to create the params, but the editing such as:

params.adaptiveThreshConstant = 2

fails with:

[1]    12512 segmentation fault (core dumped)  /home/mike/workspaces/catkin_ws/devel/lib/atom_calibration/collect_data -s 0.

[miguelriemoliveira]

This seems to be a reported issue:

https://github.com/opencv/opencv-python/issues/735

Will try to adjust as suggested ...

[miguelriemoliveira]

I was able to make it work. The new version of opencv used a slightly different name for the functions, in particular

cv2.aruco.CharucoBoard() -> cv2.aruco.CharucoBoard_create()

and

cv2.aruco.DetectorParameters() -> cv2.aruco.DetectorParameters_create()

amonst other minor details. Now its working from my side.

One thing that puzzles me is how @manuelgitgomes and @danifpdra are using this without having the same problems.

Because I am not sure what is happenging here, its better is @manuelgitgomes or @danifpdra first test this branch just to make sure it works, before I merge it into the main branch.

@manuelgitgomes , @danifpdra can you try please?

Instructions for testing:

1. Switch to the atom branch called issue629 here

https://github.com/lardemua/atom/tree/miguelriemoliveira/issue629

rgb_rgb_system should already be installed in the repo.

2. Copy this bagfile to the folder:

ROS_BAGS/rgb_rgb_system/bag1.bag

3. See if the collector runs without crashing:

   roslaunch rgb_rgb_system_calibration collect_data.launch output_folder:=$ATOM_DATASETS/rgb_rgb_system/dataset1 overwrite:=true

[danifpdra]

I can test but I don't have this system, where is it?

[miguelriemoliveira]

It is included in the atom repo ...

On Fri, Oct 6, 2023, 11:10 AM Daniela Rato @.***> wrote:

I can test but I don't have this system, where is it?

— Reply to this email directly, view it on GitHub https://github.com/lardemua/atom/issues/629#issuecomment-1750337303, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWTHVTU4DUTEWKG4T6KFTDX57KHRAVCNFSM6AAAAAA5RU7B7GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJQGMZTOMZQGM . You are receiving this because you authored the thread.Message ID: @.***>

[danifpdra]

I don't know why but your branch doesn't show here

[miguelriemoliveira]

Where is here? Pycharm?

Perhaps a pull or a fecth are needed.

It shows on the github site

[danifpdra]

Hi @miguelriemoliveira ,

I managed to run this and it doesn't crash

[danifpdra]

But I don't see the interactive marker to save a collection. Was I supposed to?

[danifpdra]

Also, the labelers don't appear?

[manuelgitgomes]

Hello @miguelriemoliveira!

I am running to the exact same error as @danifpdra!

[miguelriemoliveira]

Hi @danifpdra and @manuelgitgomes ,

My bad. I was trying out the labeler and to make it easier I deleted from the collect_data.launch the collector node, and ran it in a different terminal.

So to test you should:

1. Run the collect_data.launch just to bringup rviz

   roslaunch rgb_rgb_system_calibration collect_data.launch output_folder:=$ATOM_DATASETS/rgb_rgb_system/dataset1 overwrite:=true

2. Run the collect node in a different terminal

   rosrun atom_calibration collect_data -s 0.5 -o /home/mike/datasets/rgb_rgb_system/dataset1 -c /home/mike/workspaces/catkin_ws/src/calibration/atom/atom_examples/rgb_rgb_system/rgb_rgb_system_calibration/calibration/config.yml --overwrite

Let me know if it works.

Thanks to both

[manuelgitgomes]

Hello @miguelriemoliveira

Everything runs smoothly from my side:

[danifpdra]

When running rosrun atom_calibration collect_data -s 0.5 -o /home/daniela/datasets/rgb_rgb_system/dataset1 -c /home/daniela/catkin_ws/src/calibration/atom/atom_examples/rgb_rgb_system/rgb_rgb_system_calibration/calibration/config.yml --overwrite I get the error

Creating an InteractiveDataLabeler for sensor left_camera
Getting aruco dict
Traceback (most recent call last):
  File "/home/daniela/catkin_ws/devel/lib/atom_calibration/collect_data", line 15, in <module>
    exec(compile(fh.read(), python_script, 'exec'), context)
  File "/home/daniela/catkin_ws/src/calibration/atom/atom_calibration/scripts/collect_data", line 145, in <module>
    data_collector = DataCollectorAndLabeler(args, server, menu_handler)
  File "/home/daniela/catkin_ws/src/calibration/atom/atom_calibration/src/atom_calibration/collect/data_collector_and_labeler.py", line 144, in __init__
    sensor_labeler = InteractiveDataLabeler(self.server, self.menu_handler, sensor_dict,
  File "/home/daniela/catkin_ws/src/calibration/atom/atom_calibration/src/atom_calibration/collect/interactive_data_labeler.py", line 137, in __init__
    self.pattern = patterns.CharucoPattern(calib_pattern['dimension'], calib_pattern['size'],
  File "/home/daniela/catkin_ws/src/calibration/atom/atom_calibration/src/atom_calibration/collect/patterns.py", line 85, in __init__
    self.dictionary = cv2.aruco.Dictionary_get(cdictionary)
AttributeError: module 'cv2.aruco' has no attribute 'Dictionary_get'

[danifpdra]

What's your cv2 version? I think this is the problem

I have 4.7.0

[miguelriemoliveira]

Hi @danifpdra ,

I was working on that during the weekend.

Lets do a zoom call this afternoon to fix it once and for all ok?

I will call you when available, to see if its ok for you.

[Kazadhum]

Hello @miguelriemoliveira! It's running correctly for me as well!

[miguelriemoliveira]

Hi @Kazadhum and @manuelgitgomes ,

thanks. Please confirm that your opencv version is 4.6.0.

We thing this is the problem.

[manuelgitgomes]

Yes!

[manuelgitgomes]

Do you have opencv-contrib-python instead of only opencv @danifpdra?

[manuelgitgomes]

This ties with issue #541, where similar issues were presented.

Should we all update to 4.8.0 and adapt the code?

[miguelriemoliveira]

Hi @manuelgitgomes ,

right, I forgot we bumped into this problem before. I think we should stick to opencv 4.6.0 because its the default for Ubuntu 20.04, ros focal.

In any case, I am not sure why, @danifpdra has Ubuntu 20.04 and opencv 4.7.0.

We will try to do some ifs to make it work with both opencv versions.

[manuelgitgomes]

In any case, I am not sure why, @danifpdra has Ubuntu 20.04 and opencv 4.7.0.

I believe if it's installed from pip it installs the must updated version, if no specific version is defined. If it is installed via the package manager apt, it goes to that default.

We will try to do some ifs to make it work with both opencv versions.

We can do the both versions but it seems to me like extra work, and I am not certain it will work. I think python verifies syntax before running, so running deprecated functions will result in errors.

If we define it on our requirements.txt that the user should have 4.7.0 or 4.8.0, when running our setup steps, the user will have the must updated opencv, guaranteeing the same version for everyone.

[miguelriemoliveira]

Hi @manuelgitgomes ,

@danifpdra already fixed it. I think its best not to enforce one specific version of opencv, as even inside our group we have several ... I will close for now as it seems to be working, if we run into problems again we'll use your method.

Thanks for the input @manuelgitgomes

[miguelriemoliveira]

Just merged PR. Should be fixed in the main branch now.