search

ifm / ifm3d-ros

ifm pmd-based 3D ToF Camera ROS Package
Apache License 2.0
40 stars 40 forks source link

ifm3d-ros camera node dies or times out (Ubuntu 14.04) #1

Closed achalarvind closed 7 years ago

achalarvind commented 7 years ago

Camera: O3X Engineering Sample (M03457) Operating system: 14.04 (Kernel: 4.4.0-66-generic) GCC: 6.2.0 20160901 cmake: version 3.7.2

Boost: 15.4 Gtest: 1.6.0-1ubuntu6 Glog: 0.3.3-1 libxmlrpc: 1.33.06-0ubuntu1

libovencv-dev: 2.4.8 ros-indigo-opencv3: 3.1.0 libpcl: 1.7.1-3

Network config: ip: 192.168.0.10 bitmask: 255.255.0.0 gateway: 192.168.0.201

Output of $ifm3d hz FrameGrabber running at: 5.00277 Hz 10 frames captured, over 1 runs

When launching camera.launch, one of the following happens: 1) The node dies 2) The camera times out (Even though I am able to ping the camera/ see the output of ifm3d hz)

Below is the terminal output for the above issues: Case 1:

... logging to /home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/roslaunch-achal-P300-660.log Checking log directory for disk usage. This may take awhile. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://achal-P300:42604/

SUMMARY

PARAMETERS

  • /ifm3d/camera/assume_sw_triggered: False
  • /ifm3d/camera/frame_id_base: ifm3d/camera
  • /ifm3d/camera/ip: 192.168.0.69
  • /ifm3d/camera/password:
  • /ifm3d/camera/schema_mask: 15
  • /ifm3d/camera/timeout_millis: 500
  • /ifm3d/camera/timeout_tolerance_secs: 5.0
  • /ifm3d/camera/xmlrpc_port: 80
  • /rosdistro: indigo
  • /rosversion: 1.11.21

NODES /ifm3d/ camera (ifm3d/ifm3d_node) camera_tf (tf2_ros/static_transform_publisher)

ROS_MASTER_URI=http://localhost:11311

core service [/rosout] found process[ifm3d/camera-1]: started with pid [679] process[ifm3d/camera_tf-2]: started with pid [742] [ INFO] [1491228069.969574906]: Running dtors... [ INFO] [1491228069.969639418]: Initializing camera... [ INFO] [1491228070.983571543]: Initializing framegrabber... [ INFO] [1491228071.584100176]: Initializing image buffer... [ WARN] [1491228072.084373584]: Timeout waiting for camera! [ INFO] [1491228072.099069227]: Got unit vectors, restarting framegrabber with mask: 15 [ INFO] [1491228072.099105938]: Running dtors... [ INFO] [1491228072.099437837]: Initializing camera... [ INFO] [1491228073.113847426]: Initializing framegrabber... [ INFO] [1491228074.083522718]: Initializing image buffer... Error in `/home/achal/catkin_workspace/devel/lib/ifm3d/ifm3d_node': double free or corruption (out): 0x0000000001c9ac90 [ifm3d/camera-1] process has died [pid 679, exit code -6, cmd /home/achal/catkin_workspace/devel/lib/ifm3d/ifm3d_node cloud:=camera/cloud distance:=camera/distance amplitude:=camera/amplitude raw_amplitude:=camera/raw_amplitude confidence:=camera/confidence unit_vectors:=camera/unit_vectors good_bad_pixels:=camera/good_bad_pixels xyz_image:=camera/xyz_image extrinsics:=camera/extrinsics name:=camera log:=/home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/ifm3d-camera-1.log]. log file: /home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/ifm3d-camera-1*.log ^C[ifm3d/camera_tf-2] killing on exit shutting down processing monitor... ... shutting down processing monitor complete

Case 2:

... logging to /home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/roslaunch-achal-P300-32539.log Checking log directory for disk usage. This may take awhile. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://achal-P300:44368/

SUMMARY

PARAMETERS

  • /ifm3d/camera/assume_sw_triggered: False
  • /ifm3d/camera/frame_id_base: ifm3d/camera
  • /ifm3d/camera/ip: 192.168.0.69
  • /ifm3d/camera/password:
  • /ifm3d/camera/schema_mask: 15
  • /ifm3d/camera/timeout_millis: 500
  • /ifm3d/camera/timeout_tolerance_secs: 5.0
  • /ifm3d/camera/xmlrpc_port: 80
  • /rosdistro: indigo
  • /rosversion: 1.11.21

NODES /ifm3d/ camera (ifm3d/ifm3d_node) camera_tf (tf2_ros/static_transform_publisher)

ROS_MASTER_URI=http://localhost:11311

core service [/rosout] found process[ifm3d/camera-1]: started with pid [32557] process[ifm3d/camera_tf-2]: started with pid [32621] [ INFO] [1491228040.265584910]: Running dtors... [ INFO] [1491228040.265643347]: Initializing camera... [ INFO] [1491228041.279113250]: Initializing framegrabber... [ INFO] [1491228042.288070097]: Initializing image buffer... [ WARN] [1491228042.788325553]: Timeout waiting for camera! [ WARN] [1491228043.288560058]: Timeout waiting for camera! [ WARN] [1491228043.788789724]: Timeout waiting for camera! [ WARN] [1491228044.289020681]: Timeout waiting for camera! [ WARN] [1491228044.789195214]: Timeout waiting for camera! [ WARN] [1491228045.289395551]: Timeout waiting for camera! [ WARN] [1491228045.789626672]: Timeout waiting for camera! [ WARN] [1491228046.289863943]: Timeout waiting for camera! [ WARN] [1491228046.790096079]: Timeout waiting for camera! [ WARN] [1491228047.290328433]: Timeout waiting for camera! [ WARN] [1491228047.290381979]: Attempting to restart framegrabber... [ INFO] [1491228047.290395738]: Running dtors... [ INFO] [1491228047.290650687]: Initializing camera... [ INFO] [1491228048.305343223]: Initializing framegrabber...

tpanzarella commented 7 years ago

I'm currently away at the ProMat show. However I notice you are using 14.04 and Indigo --- likely an older compiler as well. I have not yet tested on these. Are you able to work on 16.04, Kinetic, g++ 5.4 until we have a chance to look into these olde versions?

Tom Panzarella Founder & CEO Love Park Robotics, LLC 2025 Washington Ave, Suite 217 Philadelphia, PA 19146 USA

mobile: +1 215 341 1754 tom@loveparkrobotics.com http://loveparkrobotics.com

On Apr 3, 2017, at 9:10 AM, Achal notifications@github.com wrote:

Camera: O3X Engineering Sample (M03457) Operating system: 14.04 (Kernel: 4.4.0-66-generic) GCC: 6.2.0 20160901 cmake: version 3.7.2

Network config: ip: 192.168.0.10 bitmask: 255.255.0.0 gateway: 192.168.0.201

Output of $ifm3d hz FrameGrabber running at: 5.00277 Hz 10 frames captured, over 1 runs

When launching camera.launch, one of the following happens:

The node dies The camera times out (Even though I am able to ping the camera/ see the output of ifm3d hz) Below is the terminal output for the above issues: Case 1:

... logging to /home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/roslaunch-achal-P300-660.log Checking log directory for disk usage. This may take awhile. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://achal-P300:42604/

SUMMARY

PARAMETERS

/ifm3d/camera/assume_sw_triggered: False /ifm3d/camera/frame_id_base: ifm3d/camera /ifm3d/camera/ip: 192.168.0.69 /ifm3d/camera/password: /ifm3d/camera/schema_mask: 15 /ifm3d/camera/timeout_millis: 500 /ifm3d/camera/timeout_tolerance_secs: 5.0 /ifm3d/camera/xmlrpc_port: 80 /rosdistro: indigo /rosversion: 1.11.21 NODES /ifm3d/ camera (ifm3d/ifm3d_node) camera_tf (tf2_ros/static_transform_publisher)

ROS_MASTER_URI=http://localhost:11311

core service [/rosout] found process[ifm3d/camera-1]: started with pid [679] process[ifm3d/camera_tf-2]: started with pid [742] [ INFO] [1491228069.969574906]: Running dtors... [ INFO] [1491228069.969639418]: Initializing camera... [ INFO] [1491228070.983571543]: Initializing framegrabber... [ INFO] [1491228071.584100176]: Initializing image buffer... [ WARN] [1491228072.084373584]: Timeout waiting for camera! [ INFO] [1491228072.099069227]: Got unit vectors, restarting framegrabber with mask: 15 [ INFO] [1491228072.099105938]: Running dtors... [ INFO] [1491228072.099437837]: Initializing camera... [ INFO] [1491228073.113847426]: Initializing framegrabber... [ INFO] [1491228074.083522718]: Initializing image buffer... Error in `/home/achal/catkin_workspace/devel/lib/ifm3d/ifm3d_node': double free or corruption (out): 0x0000000001c9ac90 [ifm3d/camera-1] process has died [pid 679, exit code -6, cmd /home/achal/catkin_workspace/devel/lib/ifm3d/ifm3d_node cloud:=camera/cloud distance:=camera/distance amplitude:=camera/amplitude raw_amplitude:=camera/raw_amplitude confidence:=camera/confidence unit_vectors:=camera/unit_vectors good_bad_pixels:=camera/good_bad_pixels xyz_image:=camera/xyz_image extrinsics:=camera/extrinsics name:=camera log:=/home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/ifm3d-camera-1.log]. log file: /home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/ifm3d-camera-1*.log ^C[ifm3d/camera_tf-2] killing on exit shutting down processing monitor... ... shutting down processing monitor complete

Case 2:

... logging to /home/achal/.ros/log/e470cb40-1875-11e7-97a1-6c0b8498511f/roslaunch-achal-P300-32539.log Checking log directory for disk usage. This may take awhile. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://achal-P300:44368/

SUMMARY

PARAMETERS

/ifm3d/camera/assume_sw_triggered: False /ifm3d/camera/frame_id_base: ifm3d/camera /ifm3d/camera/ip: 192.168.0.69 /ifm3d/camera/password: /ifm3d/camera/schema_mask: 15 /ifm3d/camera/timeout_millis: 500 /ifm3d/camera/timeout_tolerance_secs: 5.0 /ifm3d/camera/xmlrpc_port: 80 /rosdistro: indigo /rosversion: 1.11.21 NODES /ifm3d/ camera (ifm3d/ifm3d_node) camera_tf (tf2_ros/static_transform_publisher)

ROS_MASTER_URI=http://localhost:11311

core service [/rosout] found process[ifm3d/camera-1]: started with pid [32557] process[ifm3d/camera_tf-2]: started with pid [32621] [ INFO] [1491228040.265584910]: Running dtors... [ INFO] [1491228040.265643347]: Initializing camera... [ INFO] [1491228041.279113250]: Initializing framegrabber... [ INFO] [1491228042.288070097]: Initializing image buffer... [ WARN] [1491228042.788325553]: Timeout waiting for camera! [ WARN] [1491228043.288560058]: Timeout waiting for camera! [ WARN] [1491228043.788789724]: Timeout waiting for camera! [ WARN] [1491228044.289020681]: Timeout waiting for camera! [ WARN] [1491228044.789195214]: Timeout waiting for camera! [ WARN] [1491228045.289395551]: Timeout waiting for camera! [ WARN] [1491228045.789626672]: Timeout waiting for camera! [ WARN] [1491228046.289863943]: Timeout waiting for camera! [ WARN] [1491228046.790096079]: Timeout waiting for camera! [ WARN] [1491228047.290328433]: Timeout waiting for camera! [ WARN] [1491228047.290381979]: Attempting to restart framegrabber... [ INFO] [1491228047.290395738]: Running dtors... [ INFO] [1491228047.290650687]: Initializing camera... [ INFO] [1491228048.305343223]: Initializing framegrabber...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

achalarvind commented 7 years ago

Thanks for the quick response. I am able to get data from the camera on kinetic using the binaries provided. I have not tried to compile from source on 16.04

We should be able to run some initial tests using 16.04 but given our robot currently runs 14.04 we will not be able to properly evaluate the sensor on our robot until we can get it to run on 14.04. Also, I'm currently using g++ 6.2.0. I was not able to get the code to compile using g++ 4.9.

Do you have a rough estimate of when you will be able to work on this? Thanks!

tpanzarella commented 7 years ago

I will be looking into this later this week and next. We have another early user on 14.04/Indigo so we will ensure compatibility. Thanks for being patient.

Tom Panzarella Founder & CEO Love Park Robotics, LLC 2025 Washington Ave, Suite 217 Philadelphia, PA 19146 USA

mobile: +1 215 341 1754 tom@loveparkrobotics.com http://loveparkrobotics.com

On Apr 3, 2017, at 9:58 AM, Achal notifications@github.com wrote:

Thanks for the quick response. I am able to get data from the camera on kinetic using the binaries provided. I have not tried to compile from source on 16.04

We should be able to run some initial tests using 16.04 but given our robot currently runs 14.04 we will not be able to properly evaluate the sensor on our robot until we can get it to run on 14.04. Also, I'm currently using g++ 6.2.0. I was not able to get the code to compile using g++ 4.9.

Do you have a rough estimate of when you will be able to work on this? Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

achalarvind commented 7 years ago

Thanks. Please let me know if you need any more information.

tpanzarella commented 7 years ago

@achalarvind I have not gotten to a formal fix here yet (still trying to find the time) and my O3X stayed out in Chicago for the ProMat show after I left. I get it back next week and can then deal with this. However, I wanted to provide some insights to help you along here. Starting backward:

For "case 2" above, there seems to be an issue in the hardware where, under certain conditions (I'm working through this with ifm) that this imaging process will lock up. At which point, our only option is to restart the device. Once simple check for you is to look at the camera itself. If the illumination source is not pulsing, and you are in the "restarting framegrabber" loop you see above, we are never going to get it. At this point, you need to restart your device. As I said above, I am working with ifm on getting this resolved.

For "case 1" I believe this is as a result of linking to two different versions of OpenCV in the same binary. Likely, when you are building the ROS node from source against Indigo, you are linking to 2.4. If you are using the ifm3d binaries, or even building from source, without modification to the cmake build files, you are going to link to OpenCV 3. I will formalize the way to handle this in the next release, but, for now, I think what you can try is to build ifm3d from source and explicitly link to OpenCV 2.4. To do this, you can edit the file ifm3d/modules/image/CMakeLists.txt. Change the line: find_package(OpenCV 3 REQUIRED) to find_package(OpenCV 2.4 REQUIRED).

You should then check that you are linking consistently in both the ROS node and in libifm3d_image.so. One way to check this is as follows. Below, first I am checking the ROS node, then the library. You will note, in my case, I am consistently linking against OpenCV 3. In your case, you want to make sure you are consistently linking against 2.4:

[ ~/ros/ifm3d/lib/ifm3d ]
tpanzarella@tuna: $ readelf -d ifm3d_node | grep -i 'NEEDED'
 0x0000000000000001 (NEEDED)             Shared library: [libimage_transport.so]
 0x0000000000000001 (NEEDED)             Shared library: [libcv_bridge.so]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core3.so.3.2]
 0x0000000000000001 (NEEDED)             Shared library: [libroscpp.so]
 0x0000000000000001 (NEEDED)             Shared library: [librosconsole.so]
 0x0000000000000001 (NEEDED)             Shared library: [libroscpp_serialization.so]
 0x0000000000000001 (NEEDED)             Shared library: [librostime.so]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_system.so.1.58.0]
 0x0000000000000001 (NEEDED)             Shared library: [libifm3d_camera.so.0.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libifm3d_framegrabber.so.0.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libifm3d_image.so.0.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

[ ~/ros/ifm3d/lib/ifm3d ]
tpanzarella@tuna: $ readelf -d /usr/lib/libifm3d_image.so.0.1.0 | grep -i 'NEEDED'
 0x0000000000000001 (NEEDED)             Shared library: [libifm3d_framegrabber.so.0.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libifm3d_camera.so.0.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libglog.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core3.so.3.2]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
achalarvind commented 7 years ago

@tpanzarella Thank you. The proposed solution seems to have solved the problem. Recompiling the library with OpenCV 2.4 allows me to stream data from the device.

tpanzarella commented 7 years ago

Great. Again, a formal solution will be put in place prior to production units shipping. Thanks for the patience during this beta period.

tpanzarella commented 7 years ago

@achalarvind I wanted to revisit our "case 2" above. I believe this is fixed in the latest firmware 0.1.16. I'd suggest you upgrade to it. But before doing so, you will also need to install the calibration file which I am going to have to defer you to @cfreundl or @graugans to walk you through doing that. The calibration file is also available at the link I sent.

In my testing of the new firmware today, the device is much more stable and with the ROS node running you should be able to plug/unplug the Ethernet cable willy nilly and both the device should remain stable and the ROS node will self-recover. If you experience something different please report back here.

I'll leave this issue open for now, but, if you see the same stability improvements that I see, feel free to close it.