berndpfrommer / flir_spinnaker_ros2

ros2 driver for flir spinnaker SDK
26 stars 27 forks source link

Image incomplete: Image has missing packets #44

Closed gitissues79 closed 1 year ago

gitissues79 commented 1 year ago

After starting the multiple camera launch file, I am facing the following issues, I have also changed the MTU size to 9000. even after it didnt worked for me.

[camera_driver_node-2] [WARN] [1681064516.400810174] [cam1]: TransportLayerControl/GigEVision/GevSCPSPacketSize set to: -1 instead of: 9000 [camera_driver_node-1] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-1] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-1] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/

berndpfrommer commented 1 year ago

Can you get the camera to work correctly using spinview?

On Sun, Apr 9, 2023, 10:06 PM gitissues79 @.***> wrote:

After starting the multiple camera launch file, I am facing the following issues, I have also changed the MTU size to 9000. even after it didnt worked for me.

[camera_driver_node-2] [WARN] [1681064516.400810174] [cam1]: TransportLayerControl/GigEVision/GevSCPSPacketSize set to: -1 instead of: 9000 [camera_driver_node-1] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-1] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-1] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ [camera_driver_node-2] Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/

— Reply to this email directly, view it on GitHub https://github.com/berndpfrommer/flir_spinnaker_ros2/issues/44, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPLK2ROFIU4B5FRPVAA6K3XAMJDPANCNFSM6AAAAAAWYJSH5E . You are receiving this because you are subscribed to this thread.Message ID: @.***>

gitissues79 commented 1 year ago

Yes, it is working properly with spinview, able to run both the cameras

berndpfrommer commented 1 year ago

Does the Ros driver work if you launch only for one camera? Can you post your launch file? The warning about not being able to set the packet size could be a problem.

On Mon, Apr 10, 2023, 9:44 AM gitissues79 @.***> wrote:

Yes, it is working properly with spinview

— Reply to this email directly, view it on GitHub https://github.com/berndpfrommer/flir_spinnaker_ros2/issues/44#issuecomment-1501510517, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPLK2QIJD5MA2R3FE7H6YTXAO27NANCNFSM6AAAAAAWYJSH5E . You are receiving this because you commented.Message ID: @.***>

gitissues79 commented 1 year ago

Yes, When i am launching with one camera, it is working. I am using gige cameras.

the second camera has the issue, it is not setting mtu size to 9000. but again, i tried running it with blackfly_s_gige.launch file it is working fine. able to visulize through rqt.

This is the launch file for multi-camera

from launch_ros.actions import Node from launch.substitutions import LaunchConfiguration as LaunchConfig from launch.actions import DeclareLaunchArgument as LaunchArg from launch import LaunchDescription from ament_index_python.packages import get_package_share_directory

def generate_launch_description(): """Launch blackfly_s camera node.""" flir_dir = get_package_share_directory('flir_spinnaker_ros2') config_dir = flir_dir + '/config/'

# print([LaunchConfig('serial'), '_'])
ld = LaunchDescription()
cam_0 = Node(package='flir_spinnaker_ros2',
            executable='camera_driver_node',
            output='screen',
            name='cam_0',

            parameters=[

                {'parameter_file': config_dir + 'cam0.cfg',
                 'serial_number': '16260582',
                  'debug': False,
'compute_brightness': False,
'dump_node_map': False,
# set parameters defined in blackfly_s_gige.cfg
'gain_auto': 'Continuous',
'exposure_auto': 'Continuous',
'frame_rate_auto': 'Off',
'frame_rate': 20.0,
'frame_rate_enable': True,
'trigger_mode': 'Off',
'chunk_mode_active': True,
'chunk_selector_frame_id': 'FrameID',
'chunk_enable_frame_id': True,
'chunk_selector_exposure_time': 'ExposureTime',
'chunk_enable_exposure_time': True,
'chunk_selector_gain': 'Gain',
'chunk_enable_gain': True,
'chunk_selector_timestamp': 'Timestamp',
'chunk_enable_timestamp': True,
'gev_scps_packet_size': 9000, }],)
cam_1 = Node(package='flir_spinnaker_ros2',
            executable='camera_driver_node',
            output='screen',
            name='cam_1',
            parameters=[

                {'parameter_file': config_dir + 'cam1.cfg',
                 'serial_number': '17810590',
                 'debug': False,
                 'compute_brightness': False,
                 'dump_node_map': False,
                 # set parameters defined in blackfly_s_gige.cfg
                 'gain_auto': 'Continuous',
                 'exposure_auto': 'Continuous',
                 'frame_rate_auto': 'Off',
                 'frame_rate': 20.0,
                 'frame_rate_enable': True,
                 'trigger_mode': 'Off',
                 'chunk_mode_active': True,
                 'chunk_selector_frame_id': 'FrameID',
'chunk_enable_frame_id': True,
'chunk_selector_exposure_time': 'ExposureTime',
'chunk_enable_exposure_time': True,
'chunk_selector_gain': 'Gain',
'chunk_enable_gain': True,
'chunk_selector_timestamp': 'Timestamp',
'chunk_enable_timestamp': True,
'gev_scps_packet_size': 9000,}],)

ld.add_action(cam_0)
ld.add_action(cam_1)

return ld
berndpfrommer commented 1 year ago

Hmm. Some weird interaction then between the nodes. Can you run the cameras as composable nodes? See the stereo driver launch file as example.

On Mon, Apr 10, 2023, 12:13 PM gitissues79 @.***> wrote:

Yes, When i am launching with one camera, it is working.

This is the launch file for multi-camera

from launch_ros.actions import Node from launch.substitutions import LaunchConfiguration as LaunchConfig from launch.actions import DeclareLaunchArgument as LaunchArg from launch import LaunchDescription from ament_index_python.packages import get_package_share_directory camera_params = { 'debug': False, 'compute_brightness': False, 'dump_node_map': False, # set parameters defined in blackfly_s_gige.cfg 'gain_auto': 'Continuous', 'exposure_auto': 'Continuous', 'frame_rate_auto': 'Off', 'frame_rate': 20.0, 'frame_rate_enable': True, 'trigger_mode': 'Off', 'chunk_mode_active': True, 'chunk_selector_frame_id': 'FrameID', 'chunk_enable_frame_id': True, 'chunk_selector_exposure_time': 'ExposureTime', 'chunk_enable_exposure_time': True, 'chunk_selector_gain': 'Gain', 'chunk_enable_gain': True, 'chunk_selector_timestamp': 'Timestamp', 'chunk_enable_timestamp': True, 'gev_scps_packet_size': 9000, }

def generate_launch_description(): """Launch blackfly_s camera node.""" flir_dir = get_package_share_directory('flir_spinnaker_ros2') config_dir = flir_dir + '/config/'

print([LaunchConfig('serial'), '_'])

ld = LaunchDescription() cam_0 = Node(package='flir_spinnaker_ros2', executable='camera_driver_node', output='screen', name='cam_0',

        parameters=[

            {'parameter_file': config_dir + 'cam0.cfg',
             'serial_number': '16260582',
              'debug': False,

'compute_brightness': False, 'dump_node_map': False,

set parameters defined in blackfly_s_gige.cfg

'gain_auto': 'Continuous', 'exposure_auto': 'Continuous', 'frame_rate_auto': 'Off', 'frame_rate': 20.0, 'frame_rate_enable': True, 'trigger_mode': 'Off', 'chunk_mode_active': True, 'chunk_selector_frame_id': 'FrameID', 'chunk_enable_frame_id': True, 'chunk_selector_exposure_time': 'ExposureTime', 'chunk_enable_exposure_time': True, 'chunk_selector_gain': 'Gain', 'chunk_enable_gain': True, 'chunk_selector_timestamp': 'Timestamp', 'chunk_enable_timestamp': True, 'gev_scps_packet_size': 9000, }],) cam_1 = Node(package='flir_spinnaker_ros2', executable='camera_driver_node', output='screen', name='cam_1', parameters=[

            {'parameter_file': config_dir + 'cam1.cfg',
             'serial_number': '17810590',
             'debug': False,
             'compute_brightness': False,
             'dump_node_map': False,
             # set parameters defined in blackfly_s_gige.cfg
             'gain_auto': 'Continuous',
             'exposure_auto': 'Continuous',
             'frame_rate_auto': 'Off',
             'frame_rate': 20.0,
             'frame_rate_enable': True,
             'trigger_mode': 'Off',
             'chunk_mode_active': True,
             'chunk_selector_frame_id': 'FrameID',

'chunk_enable_frame_id': True, 'chunk_selector_exposure_time': 'ExposureTime', 'chunk_enable_exposure_time': True, 'chunk_selector_gain': 'Gain', 'chunk_enable_gain': True, 'chunk_selector_timestamp': 'Timestamp', 'chunk_enable_timestamp': True, 'gev_scps_packet_size': 9000,}],)

ld.add_action(cam_0) ld.add_action(cam_1)

return ld

— Reply to this email directly, view it on GitHub https://github.com/berndpfrommer/flir_spinnaker_ros2/issues/44#issuecomment-1501644175, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPLK2RO4RG3Q2VWGN2GWPTXAPMNBANCNFSM6AAAAAAWYJSH5E . You are receiving this because you commented.Message ID: @.***>

gitissues79 commented 1 year ago

I have tried the composable nodes, it is working but able to visualize only 2nd camera, after a min, it is terminating and throwing error.

when i am setting output=screen, it is throwing the same error[ Image incomplete: Image has missing packets]

here multi-camera launch file:

import os from launch_ros.actions import Node from launch.substitutions import LaunchConfiguration as LaunchConfig from launch.actions import DeclareLaunchArgument as LaunchArg from launch import LaunchDescription from launch_ros.actions import ComposableNodeContainer from ament_index_python.packages import get_package_share_directory from launch_ros.descriptions import ComposableNode

camera_params = { 'debug': False, 'compute_brightness': False, 'dump_node_map': False,

set parameters defined in blackfly_s_gige.cfg

'gain_auto': 'Continuous',
'exposure_auto': 'Continuous',
'frame_rate_auto': 'Off',
'frame_rate': 20.0,
'frame_rate_enable': True,
'trigger_mode': 'Off',
'chunk_mode_active': True,
'chunk_selector_frame_id': 'FrameID',
'chunk_enable_frame_id': True,
'chunk_selector_exposure_time': 'ExposureTime',
'chunk_enable_exposure_time': True,
'chunk_selector_gain': 'Gain',
'chunk_enable_gain': True,
'chunk_selector_timestamp': 'Timestamp',
'chunk_enable_timestamp': True,
'gev_scps_packet_size': 9000,
}

def generate_launch_description(): """Launch blackfly_s camera node.""" flir_dir = get_package_share_directory('flir_spinnaker_ros2') config_dir = flir_dir + '/config/'

container = ComposableNodeContainer(
    namespace='',
    name="camera_container",
    package = "rclcpp_components",
    executable = "component_container",
    composable_node_descriptions = [
        ComposableNode(
                package = 'flir_spinnaker_ros2',
                plugin = 'flir_spinnaker_ros2::CameraDriver',
                name = 'cam_0',
                parameters = [camera_params,
                {'parameter_file': config_dir + 'cam0.cfg',
                 'serial_number': '16260582',}],
                ),
        ComposableNode(
                package = 'flir_spinnaker_ros2',
                plugin = 'flir_spinnaker_ros2::CameraDriver',
                name = 'cam_1',
                parameters = [camera_params,
                {'parameter_file': config_dir + 'cam1.cfg',
                 'serial_number': '17810590',}],
                ),
    ], )

ld = LaunchDescription()    
ld.add_action(container)

return ld

error: [component_container-1] [INFO] [1681137380.109436004] [cam_1]: setting ChunkDataControl/ChunkSelector to: ExposureTime [component_container-1] [INFO] [1681137380.110498495] [cam_1]: setting ChunkDataControl/ChunkEnable to: 1 [component_container-1] [INFO] [1681137380.112822678] [cam_1]: setting ChunkDataControl/ChunkSelector to: Gain [component_container-1] [INFO] [1681137380.113777019] [cam_1]: setting ChunkDataControl/ChunkEnable to: 1 [component_container-1] [INFO] [1681137380.115673250] [cam_1]: setting ChunkDataControl/ChunkSelector to: Timestamp [component_container-1] [INFO] [1681137380.116628793] [cam_1]: setting ChunkDataControl/ChunkEnable to: 1 [component_container-1] [INFO] [1681137380.118601664] [cam_1]: setting TransportLayerControl/GigEVision/GevSCPSPacketSize to: 9000 [component_container-1] [WARN] [1681137380.118731905] [cam_1]: setting TransportLayerControl/GigEVision/GevSCPSPacketSize failed: node TransportLayerControl/GigEVision/GevSCPSPacketSize does not exist! [component_container-1] [WARN] [1681137380.118740855] [cam_1]: TransportLayerControl/GigEVision/GevSCPSPacketSize set to: -1 instead of: 9000 [component_container-1] [INFO] [1681137380.135604853] [cam_1]: camera has pixel format: BayerRG8 [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/cam_1' in container '/camera_container' [component_container-1] [INFO] [1681137383.094249539] [cam_0]: rate [Hz] in 20.05 out 0.00 drop 0% [component_container-1] [INFO] [1681137384.838820098] [cam_1]: rate [Hz] in 60.00 out 0.00 drop 0% [component_container-1] [INFO] [1681137388.094230776] [cam_0]: rate [Hz] in 20.02 out 0.00 drop 0% [component_container-1] [INFO] [1681137389.838822420] [cam_1]: rate [Hz] in 59.99 out 0.00 drop 0% [component_container-1] [INFO] [1681137393.094235931] [cam_0]: rate [Hz] in 20.01 out 0.00 drop 0% [component_container-1] [INFO] [1681137394.838851608] [cam_1]: rate [Hz] in 59.99 out 0.00 drop 0% [component_container-1] [INFO] [1681137398.094232291] [cam_0]: rate [Hz] in 20.00 out 0.00 drop 0% [component_container-1] [INFO] [1681137399.838823042] [cam_1]: rate [Hz] in 59.99 out 0.00 drop 0% [component_container-1] [INFO] [1681137403.094237381] [cam_0]: rate [Hz] in 19.99 out 0.00 drop 0% [component_container-1] [INFO] [1681137404.838825709] [cam_1]: rate [Hz] in 59.99 out 0.00 drop 0% [component_container-1] [INFO] [1681137408.094232545] [cam_0]: rate [Hz] in 20.00 out 0.00 drop 0% [component_container-1] [INFO] [1681137409.838824372] [cam_1]: rate [Hz] in 59.99 out 0.00 drop 0% [component_container-1] [INFO] [1681137413.094233501] [cam_0]: rate [Hz] in 20.01 out 0.00 drop 0% [component_container-1] [INFO] [1681137414.838824139] [cam_1]: rate [Hz] in 59.99 out 0.00 drop 0% [component_container-1] [INFO] [1681137418.094234114] [cam_0]: rate [Hz] in 20.01 out 0.00 drop 0% [component_container-1] terminate called after throwing an instance of 'Spinnaker::Exception' [component_container-1] what(): Spinnaker: Camera is not started. [-1002] [ERROR] [component_container-1]: process has died [pid 15977, exit code -6, cmd '/opt/ros/foxy/lib/rclcpp_components/component_container --ros-args -r __node:=camera_container -r __ns:=/'].

I realized, spinview both the camera has different parameters, for camera 2 it is set 60 fps while camera 1 set to 20 fps and unable to change the value. SpinView version 2.6.0.160

camera 1

image

camera 2

image

berndpfrommer commented 1 year ago

You are setting the frame rate to 20 hz in the config, so why does the cam_1 keep running at 60hz? It almost looks like cam_1 does not get the settings from the config file. It could be that the setting of the camera parameters doesn't work because of this:

component_container-1] [WARN] [1681137380.118731905] [cam_1]: setting TransportLayerControl/GigEVision/GevSCPSPacketSize failed: node TransportLayerControl/GigEVision/GevSCPSPacketSize does not exist!
[component_container-1] [WARN] [1681137380.118740855] [cam_1]: TransportLayerControl/GigEVision/GevSCPSPacketSize set to: -1 instead of: 9000

Do you get the same error message for cam_0? Are those two cameras identical? The SpinView menu looks differentl. Does cam_1 actually have a GevSCPSPacketSize parameter in SpinView? The error message says that the ROS driver cannot find it. Also, make sure you run on the latest camera firmware. Sometimes there are significant improvements and changes between firmware versions.

gitissues79 commented 1 year ago

I have set the config to 20hz for both the camera and both the camera has GevSCPSPacketSize and it is set to 9000. Cam_0 is blackfly s camera and cam_2 is blackfly pointgrey camera there's no error for cam_0, but cam_1 isnt reading/setting the paramerters properly may be because of this error

component_container-1] [WARN] [1681137380.118731905] [cam_1]: setting TransportLayerControl/GigEVision/GevSCPSPacketSize failed: node TransportLayerControl/GigEVision/GevSCPSPacketSize does not exist! [component_container-1] [WARN] [1681137380.118740855] [cam_1]: TransportLayerControl/GigEVision/GevSCPSPacketSize set to: -1 instead of: 9000

how to resolve this issue?

berndpfrommer commented 1 year ago

FLIR has documentation on how to upgrade the firmware. Please use your favorite search engine to find it.

Apparently the blackfly camera has a different way of setting the GevSCPSPacketSize (if it can be done at all) than the blackfly s. Here is a link to the documentation for the BlackFly. https://flir.netx.net/file/asset/13133/original/attachment You need to learn how to configure the GigE packet size on that camera, using SpinView. Then for the ROS driver you need to create a camera.cfg file for the blackfly GigE camera, similar to the blackfly s, where you set the parameters. The ROS driver README.md has documentation about how to create custom config files.

On Mon, Apr 10, 2023 at 3:31 PM gitissues79 @.***> wrote:

I have set the config to 20hz for both the camera and both the camera has GevSCPSPacketSize and it is set to 9000. Cam_0 is blackfly s camera and cam_2 is blackfly pointgrey camera there's no error for cam_0, but cam_1 isnt reading/setting the paramerters properly may be because of this error

component_container-1] [WARN] [1681137380.118731905] [cam_1]: setting TransportLayerControl/GigEVision/GevSCPSPacketSize failed: node TransportLayerControl/GigEVision/GevSCPSPacketSize does not exist! [component_container-1] [WARN] [1681137380.118740855] [cam_1]: TransportLayerControl/GigEVision/GevSCPSPacketSize set to: -1 instead of: 9000

How to update the firmware version?

— Reply to this email directly, view it on GitHub https://github.com/berndpfrommer/flir_spinnaker_ros2/issues/44#issuecomment-1502220310, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPLK2S3EWA2PBREYYHW26LXARNZ7ANCNFSM6AAAAAAWYJSH5E . You are receiving this because you commented.Message ID: @.***>

berndpfrommer commented 1 year ago

Closing this issue because it is likely a problem with a bad config file (Blackfly vs Blackfly s).