magazino / pylon_camera

ROS-Driver for Basler Cameras
BSD 3-Clause "New" or "Revised" License
78 stars 108 forks source link

Grab was not successful / Error: 3774873620 The buffer was incompletely grabbed. #20

Closed fehlfarbe closed 7 years ago

fehlfarbe commented 7 years ago

I want to capture images from a Basler acA1300-60gm and the ROS pylon_camera node. I can open and control the camera with Pylon's PylonViewerApp. But when I'm starting the ROS node with roslaunch pylon_camera pylon_camera_node.launch I get error messages that the frames can't be grabbed:

started roslaunch server http://192.168.1.2:45055/

SUMMARY
========

PARAMETERS
 * /pylon_camera_node/binning_x: 2
 * /pylon_camera_node/binning_y: 2
 * /pylon_camera_node/camera_frame: pylon_camera_node
 * /pylon_camera_node/camera_info_url: 
 * /pylon_camera_node/device_user_id: 1
 * /pylon_camera_node/image_encoding: mono8
 * /pylon_camera_node/inter_pkg_delay: 11772
 * /rosdistro: kinetic
 * /rosversion: 1.12.7

NODES
  /
    pylon_camera_node (pylon_camera/pylon_camera_node)

auto-starting new master
process[master]: started with pid [10507]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to ba1098f6-28e9-11e7-84cf-f44d30679130
process[rosout-1]: started with pid [10520]
started core service [/rosout]
process[pylon_camera_node-2]: started with pid [10529]
binning x is given and has value 2
binning y is given and has value 2
[ INFO] [1493036997.880801762]: Trying to open the following camera: 1
[ INFO] [1493036998.138585721]: Found the desired camera with DeviceUserID 1: acA1300-60gm
[ INFO] [1493036998.407568803]: Cam has binning range: x(hz) = [1 - 4], y(vt) = [1 - 4].
[ INFO] [1493036998.408075961]: Cam has exposure time range: [10 - 916000] measured in microseconds.
[ INFO] [1493036998.408277173]: Cam has gain range: [0 - 3] measured in device specific units.
[ INFO] [1493036998.411048207]: Cam has gammma range: [0 - 3.99998].
[ INFO] [1493036998.412463342]: Cam has pylon auto brightness range: [50 - 205] which is the average pixel intensity.
[ INFO] [1493036998.429420227]: Cam supports the following [GenAPI|ROS] image encodings: ['Mono8'|'mono8'] ['Mono12'|'NO_ROS_EQUIVALENT'] ['Mono12Packed'|'NO_ROS_EQUIVALENT'] ['YUV422Packed'|'NO_ROS_EQUIVALENT'] ['YUV422_YUYV_Packed'|'NO_ROS_EQUIVALENT'] 
[ERROR] [1493036998.676944253]: Error: 3774873620 The buffer was incompletely grabbed. This can be caused by performance problems of the network hardware used, i.e. network adapter, switch, or ethernet cable. To fix this, try increasing the camera's Inter-Packet Delay in the Transport Layer category to reduce the required bandwidth, and adjust the camera's Packet Size setting to the highest supported frame size.
[ INFO] [1493036998.729242950]: CameraInfoURL needed for rectification! ROS-Param: '/pylon_camera_node/camera_info_url' = '' is invalid!
[ WARN] [1493036998.729288645]: Will only provide distorted /image_raw images!
[ INFO] [1493036998.777814074]: Setting horizontal binning_x to 2
[ WARN] [1493036998.777900512]: The image width of the camera_info-msg will be adapted, so that the binning_x value in this msg remains 1
[ INFO] [1493036998.834390106]: Setting vertical binning_y to 2
[ WARN] [1493036998.834634538]: The image height of the camera_info-msg will be adapted, so that the binning_y value in this msg remains 1
[ INFO] [1493036998.839134313]: Startup settings: encoding = 'mono8', binning = [2, 2], exposure = 5000, gain = 0, gamma = 1, shutter mode = default_shutter_mode
[ INFO] [1493036998.840824354]: Start image grabbing if node connects to topic with a frame_rate of: 5 Hz
[ INFO] [1493036998.841150046]: Camera not calibrated

When I start the ROS image viewer rosrun image_view image_view image:=/pylon_camera_node/image_raw I dont't get images but more error messages from the pylon_camera_node:

[ WARN] [1493037429.575879168]: Pylon camera returned invalid image! Skipping
[ERROR] [1493037429.784726148]: Error: 3774873620 The buffer was incompletely grabbed. This can be caused by performance problems of the network hardware used, i.e. network adapter, switch, or ethernet cable. To fix this, try increasing the camera's Inter-Packet Delay in the Transport Layer category to reduce the required bandwidth, and adjust the camera's Packet Size setting to the highest supported frame size.
[ERROR] [1493037429.784979476]: Error: Grab was not successful
[ WARN] [1493037429.785294487]: Pylon camera returned invalid image! Skipping
[ERROR] [1493037430.034649674]: Error: 3774873620 The buffer was incompletely grabbed. This can be caused by performance problems of the network hardware used, i.e. network adapter, switch, or ethernet cable. To fix this, try increasing the camera's Inter-Packet Delay in the Transport Layer category to reduce the required bandwidth, and adjust the camera's Packet Size setting to the highest supported frame size.
[ERROR] [1493037430.034807885]: Error: Grab was not successful

I tried different settings for packet delay, binning, framerate and PoE switch/PoE inserter but it's alway the same problem. After i started and closed the roslauncher I can't open the camera with PylonViewerApp until I replugged the device. The app prints these error messages to console:

Failed to get the node 'AcquisitionFrameRate' from the device's nodemap
Error while reading properties
onUpdateNode :  Exception occurred:
 Node is not readable. 
 /home/builder/jenkins_root/workspace/PylonLinux_ReleaseBuild/linux-build-tools/build/lsb-gcc-x86_64-release/build/genicam/Genicam/library/CPP/include/GenApi/impl/EnumerationT.h : 131 

Error while reading properties
onUpdateNode :  Exception occurred:
 Node is not readable. 
 /home/builder/jenkins_root/workspace/PylonLinux_ReleaseBuild/linux-build-tools/build/lsb-gcc-x86_64-release/build/genicam/Genicam/library/CPP/include/GenApi/impl/EnumerationT.h : 131

Any idea what's wrong?

Grillen80 commented 7 years ago

Dear Fehlfarbe, I'm running into the same problem while running the basler camera. A bunch of error messages appear while starting the roslaunch application. Did you get it running or are you still stucking in the same problem. I hope that someone could help me out.

marc-up commented 7 years ago

I think it's a problem of your network configuration. @fehlfarbe are you using a RaspberryPI? Why is your inter-pkg-delay set to 11772? When using GigE cameras you first have to set the MTU size of your NIC to the maximum supported (9000 is a good guess, validate e.g. via ifconfig). The corresponding parameter in the config file of the node should be set as well. Here is the description taken out of the default.yaml in the config folder: The MTU size. Only used for GigE cameras. To prevent lost frames configure the camera has to be configured with the MTU size the network card supports. A value greater 3000 should be good (~1500 for RaspberryPI) Next you have to figure out what would be the minimum required inter-pkg-delay value to prevent these Error: 3774873620 The buffer was incompletely grabbed. errors. I tried to explain how this can be done In the last post of issue #16 So what you can do is to run the PylonViewerApp, set exposure_time to min and MTU size so that you get the maximum possible frame-rate, run the acquisition in the free-running mode and check the logging-console. In the transport-layer-selection, you then have to increase the inter-pkg-delay and check the statistics till you are satisfied with the result.

fehlfarbe commented 7 years ago

Thanks! I looked at the inter-pkg-delay and MTU in PylonViewerApp and it was set to inter-pkg-delay: 0 and MTU: 1500 which works with ~85 fps and no errors. The pylon_camera driver doesn't work with these settings but when I increase the MTU of my network device from 1500 to 3000 it's working.

AustinDeric commented 6 years ago

I am also having this issue when trying to run the pylon_camera_node:

roslaunch frame_finder frame_finder.launch

SUMMARY
========

PARAMETERS
 * /pylon_camera_node/camera_frame: pylon_camera_node
 * /pylon_camera_node/camera_info_url: f
 * /pylon_camera_node/device_user_id: 
 * /pylon_camera_node/frame_rate: 5.0
 * /pylon_camera_node/image_encoding: bayer_bggr8
 * /pylon_camera_node/inter_pkg_delay: 1000
 * /pylon_camera_node/mtu_size: 3000
 * /rosdistro: kinetic
 * /rosversion: 1.12.7

NODES
  /
    pylon_camera_node (pylon_camera/pylon_camera_node)

auto-starting new master
process[master]: started with pid [11221]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to fe0e75f4-76c4-11e7-af08-54ee753af24d
process[rosout-1]: started with pid [11234]
started core service [/rosout]
process[pylon_camera_node-2]: started with pid [11245]
[ INFO] [1501597411.282207032]: No Device User ID set -> Will open the camera device found first
[ INFO] [1501597411.541916380]: Found camera with DeviceUserID Basler21135423: acA1600-20gc
[ INFO] [1501597411.693350871]: Cam does not support binning.
[ INFO] [1501597411.694335945]: Cam has exposure time range: [25 - 1e+06] measured in microseconds.
[ INFO] [1501597411.694379570]: Cam has gain range: [230 - 850] measured in device specific units.
[ INFO] [1501597411.695333398]: Cam has gammma range: [0 - 65536].
[ INFO] [1501597411.696272587]: Cam has pylon auto brightness range: [50 - 205] which is the average pixel intensity.
[ INFO] [1501597411.706390826]: Cam supports the following [GenAPI|ROS] image encodings: ['Mono8'|'mono8'] ['BayerBG8'|'bayer_bggr8'] ['BayerBG12'|'NO_ROS_EQUIVALENT'] ['YUV422Packed'|'NO_ROS_EQUIVALENT'] ['YUV422_YUYV_Packed'|'NO_ROS_EQUIVALENT'] ['BayerBG12Packed'|'NO_ROS_EQUIVALENT'] 
[ERROR] [1501597412.037458239]: Error: 3774873620 The buffer was incompletely grabbed. This can be caused by performance problems of the network hardware used, i.e. network adapter, switch, or ethernet cable. To fix this, try increasing the camera's Inter-Packet Delay in the Transport Layer category to reduce the required bandwidth, and adjust the camera's Packet Size setting to the highest supported frame size.
[ INFO] [1501597412.062628500]: CameraInfoURL needed for rectification! ROS-Param: '/pylon_camera_node/camera_info_url' = 'f' is invalid!
[ WARN] [1501597412.062692889]: Will only provide distorted /image_raw images!
[ INFO] [1501597412.066251535]: Startup settings: encoding = 'bayer_bggr8', binning = [1, 1], exposure = 10000, gain = 0.225806, gamma = 1, shutter mode = default_shutter_mode
[ INFO] [1501597412.066848119]: Start image grabbing if node connects to topic with a frame_rate of: 5 Hz
[ INFO] [1501597412.067197611]: Camera not calibrated
void-robotics commented 6 years ago

I'm also stuck like @AustinDeric; any more help?

AustinDeric commented 6 years ago

@nathangeorge1 What i ended up doing was setting my network adapter mtu to 9000 using the command (substitute your network adapter for eth0): ifconfig eth0 mtu 9000 then in the config set your mtu also to 9000. My inter_pkg_delay was 10000 but i am not sure the delay parameter had any impact. this should work for you.

It would be great if the ros pylon script worked as well as the pylon viewer app. This Issue should not be closed.

void-robotics commented 6 years ago

hello @AustinDeric I just set ifconfig enp0s3 mtu 9000 and also set within pylon_camera/config/default.yaml the mtu_size: 9000 and even inter_pkg_delay: 10000 just in case; but I'm still stuck

My error output:

roslaunch pylon_camera pylon_camera_node.launch
... logging to /home/nathan/.ros/log/515c97ae-7eae-11e7-baa9-080027eb2e78/roslaunch-nathan-bloom-2610.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://nathan-bloom:33043/

SUMMARY
========

PARAMETERS
 * /pylon_camera_node/camera_frame: pylon_camera_node
 * /pylon_camera_node/camera_info_url: 
 * /pylon_camera_node/device_user_id: 
 * /pylon_camera_node/frame_rate: 5.0
 * /pylon_camera_node/inter_pkg_delay: 10000
 * /pylon_camera_node/mtu_size: 9000
 * /rosdistro: kinetic
 * /rosversion: 1.12.7

NODES
  /
    pylon_camera_node (pylon_camera/pylon_camera_node)

auto-starting new master
process[master]: started with pid [2621]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 515c97ae-7eae-11e7-baa9-080027eb2e78
process[rosout-1]: started with pid [2634]
started core service [/rosout]
process[pylon_camera_node-2]: started with pid [2642]
[ INFO] [1502467282.125749617]: No Device User ID set -> Will open the camera device found first
[ INFO] [1502467282.382652598]: Found camera with DeviceUserID : acA1920-40gm
[ INFO] [1502467282.672967684]: Cam has binning range: x(hz) = [1 - 4], y(vt) = [1 - 4].
[ INFO] [1502467282.673051216]: Cam has exposure time range: [76 - 1e+07] measured in microseconds.
[ INFO] [1502467282.673072759]: Cam has gain range: [0 - 360] measured in device specific units.
[ INFO] [1502467282.678157004]: Cam has gammma range: [0 - 3.99998].
[ INFO] [1502467282.680596096]: Cam has pylon auto brightness range: [50 - 205] which is the average pixel intensity.
[ INFO] [1502467282.697544524]: Cam supports the following [GenAPI|ROS] image encodings: ['Mono8'|'mono8'] ['Mono12'|'NO_ROS_EQUIVALENT'] ['Mono12Packed'|'NO_ROS_EQUIVALENT'] 
[ WARN] [1502467282.697629581]: No image encoding provided. Will use 'mono8' or 'rgb8' as fallback!
[ERROR] [1502467282.976288487]: Error: 3774873620 The buffer was incompletely grabbed. This can be caused by performance problems of the network hardware used, i.e. network adapter, switch, or ethernet cable. To fix this, try increasing the camera's Inter-Packet Delay in the Transport Layer category to reduce the required bandwidth, and adjust the camera's Packet Size setting to the highest supported frame size.
[ WARN] [1502467282.982311260]: [] name not valid for camera_info_manger
[ INFO] [1502467283.031941094]: CameraInfoURL needed for rectification! ROS-Param: '/pylon_camera_node/camera_info_url' = '' is invalid!
[ WARN] [1502467283.032043894]: Will only provide distorted /image_raw images!
[ INFO] [1502467283.039714413]: Startup settings: encoding = 'mono8', binning = [1, 1], exposure = 3000, gain = 0, gamma = 1, shutter mode = default_shutter_mode
[ INFO] [1502467283.042175607]: Start image grabbing if node connects to topic with a frame_rate of: 5 Hz
[ INFO] [1502467283.042642946]: Camera not calibrated
martomi commented 6 years ago

I had the same problem and solved it the following way:

1) My network adapter didn't support MTU size larger than 1500, so increasing that was not an option. If you can increase it to at least 3000, it should work.

2) The pylon camera node is running with default parameter for mtu_size at 3000 => In order to decrease it you need to remove the comment in the config file and set it to 1500. But there's a catch!

The parameter file is designed in a way that is extremely error-prone to configure:

None of those would work:

Wrong example 1:

# gige:
mtu_size: 1500

# gige:
inter_pkg_delay: 0

Wrong example 2:

gige:
 mtu_size: 1500

gige:
 inter_pkg_delay: 0

Here's the configuration that worked for me:

gige:
 mtu_size: 1500
 inter_pkg_delay: 0

Verify you have configured it correctly! When you start the node, you should get this in the parameters summary:

 * /pylon_camera_node/gige/inter_pkg_delay: 0
 * /pylon_camera_node/gige/mtu_size: 1500

Note the /gige/ part that everyone in the comments above seems to be missing!

Hope that helps.

suho0515 commented 5 years ago

@martomi that was correct! i solved it with your mention! thanks!

eyildiz-ugoe commented 5 years ago

@AustinDeric This suggestion worked for me. Thanks.

Triocrossing commented 5 years ago

@martomi
Thanks for saving me out ; )

eyildiz-ugoe commented 4 years ago

Even though I set it to 3000 or 9000 in the config, when I launched the node it printed 1500. All other config settings were loaded but not this. So, in order to solve this, you must use the /opt/pylon5/bin/PylonViewerApp and follow this.