search

ros-drivers / flir_camera_driver

153 stars 145 forks source link

[galactic]Can't read any images from ROS2 topics #162

Closed SangHunHan92 closed 3 months ago

SangHunHan92 commented 3 months ago

System details

Is this behavior also observed when using SpinVIew? No, The images/videos shown through SpinVIew are very clear.

Describe the bug I have already installed "spinnaker-3.1.0.79-amd64-pkg" and "flycapture2-2.13.3.31-amd64-pkg_Ubuntu18.04". Also, during build ROS2 pkg using rosdep install --from-paths src --ignore-src and colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON, no warnings or errors were detected. But, If I run launchros2 launch spinnaker_camera_driver driver_node.launch.py, I can't show any camera/image from ROS2 topic. I check this using "rviz2" and "ros2 topic hz".

Some causes I guess are camera model and Camera calibration file. The camera model confirmed in the config is Blackfly S GigE, but the camera I use is Flea3 GigE. Can you tell me how to use it in Flea3 GigE? Also, how can I get the Camera calibration file?

Useful logging output Please cut-and-paste console output into this bug report. Do not attach screenshots.

$ ros2 launch spinnaker_camera_driver driver_node.launch.py camera_type:=blackfly_s serial:="'20306011'" camera_name:=cam1
[INFO] [launch]: All log files can be found below /home/ros20/.ros/log/2024-03-22-15-35-43-478048-ros20-MS-7B89-11630
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [camera_driver_node-1]: process started with pid [11632]
[camera_driver_node-1] [INFO] [1711089343.574080942] [cam1]: adjusting time stamps!
[camera_driver_node-1] [INFO] [1711089343.574168628] [cam1]: looking for serial number: 20306011
[camera_driver_node-1] [INFO] [1711089343.574177688] [cam1]: parameter definitions file: /home/ros20/workspace/install/spinnaker_camera_driver/share/spinnaker_camera_driver/config/blackfly_s.yaml
[camera_driver_node-1] [INFO] [1711089343.575407802] [cam1]: using default calibration URL
[camera_driver_node-1] [INFO] [1711089343.575421202] [cam1]: camera calibration URL: file:///home/ros20/.ros/camera_info/cam1.yaml
[camera_driver_node-1] [ERROR] [1711089343.575448301] [camera_calibration_parsers]: Unable to open camera calibration file [/home/ros20/.ros/camera_info/cam1.yaml]
[camera_driver_node-1] [WARN] [1711089343.575453780] [cam1]: Camera calibration file /home/ros20/.ros/camera_info/cam1.yaml not found
[camera_driver_node-1] [INFO] [1711089344.857822077] [cam1]: using spinnaker lib version: 3.1.0.79
[camera_driver_node-1] [INFO] [1711089344.858115294] [cam1]: found camera with serial number: 20306011
[camera_driver_node-1] [INFO] [1711089344.913566404] [cam1]: setting AnalogControl/GainAuto to: Continuous
[camera_driver_node-1] [INFO] [1711089344.915479619] [cam1]: setting AcquisitionControl/ExposureAuto to: Continuous
[camera_driver_node-1] [INFO] [1711089344.917322446] [cam1]: setting AcquisitionControl/AcquisitionFrameRateEnable to: 1
[camera_driver_node-1] [WARN] [1711089344.917392103] [cam1]: setting AcquisitionControl/AcquisitionFrameRateEnable failed: node AcquisitionControl/AcquisitionFrameRateEnable does not exist!
[camera_driver_node-1] [INFO] [1711089344.917427322] [cam1]: setting AcquisitionControl/AcquisitionFrameRate to: 30
[camera_driver_node-1] [WARN] [1711089344.918544461] [cam1]: setting AcquisitionControl/AcquisitionFrameRate failed: node AcquisitionControl/AcquisitionFrameRate not available!
[camera_driver_node-1] [INFO] [1711089344.918597639] [cam1]: setting AcquisitionControl/TriggerMode to: Off
[camera_driver_node-1] [INFO] [1711089344.919450850] [cam1]: setting ChunkDataControl/ChunkModeActive to: 1
[camera_driver_node-1] [WARN] [1711089344.919505608] [cam1]: setting ChunkDataControl/ChunkModeActive failed: node ChunkDataControl/ChunkModeActive does not exist!
[camera_driver_node-1] [INFO] [1711089344.919538536] [cam1]: setting ChunkDataControl/ChunkSelector to: FrameID
[camera_driver_node-1] [WARN] [1711089344.919568535] [cam1]: setting ChunkDataControl/ChunkSelector failed: node ChunkDataControl/ChunkSelector does not exist!
[camera_driver_node-1] [WARN] [1711089344.919578065] [cam1]: ChunkDataControl/ChunkSelector set to: UNKNOWN instead of: FrameID
[camera_driver_node-1] [INFO] [1711089344.919602264] [cam1]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [WARN] [1711089344.919645602] [cam1]: setting ChunkDataControl/ChunkEnable failed: node ChunkDataControl/ChunkEnable does not exist!
[camera_driver_node-1] [INFO] [1711089344.919695329] [cam1]: setting ChunkDataControl/ChunkSelector to: ExposureTime
[camera_driver_node-1] [WARN] [1711089344.919742527] [cam1]: setting ChunkDataControl/ChunkSelector failed: node ChunkDataControl/ChunkSelector does not exist!
[camera_driver_node-1] [WARN] [1711089344.919754287] [cam1]: ChunkDataControl/ChunkSelector set to: UNKNOWN instead of: ExposureTime
[camera_driver_node-1] [INFO] [1711089344.919784215] [cam1]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [WARN] [1711089344.919832193] [cam1]: setting ChunkDataControl/ChunkEnable failed: node ChunkDataControl/ChunkEnable does not exist!
[camera_driver_node-1] [INFO] [1711089344.919863342] [cam1]: setting ChunkDataControl/ChunkSelector to: Gain
[camera_driver_node-1] [WARN] [1711089344.919889821] [cam1]: setting ChunkDataControl/ChunkSelector failed: node ChunkDataControl/ChunkSelector does not exist!
[camera_driver_node-1] [WARN] [1711089344.919898650] [cam1]: ChunkDataControl/ChunkSelector set to: UNKNOWN instead of: Gain
[camera_driver_node-1] [INFO] [1711089344.919920929] [cam1]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [WARN] [1711089344.919945208] [cam1]: setting ChunkDataControl/ChunkEnable failed: node ChunkDataControl/ChunkEnable does not exist!
[camera_driver_node-1] [INFO] [1711089344.919967227] [cam1]: setting ChunkDataControl/ChunkSelector to: Timestamp
[camera_driver_node-1] [WARN] [1711089344.919991416] [cam1]: setting ChunkDataControl/ChunkSelector failed: node ChunkDataControl/ChunkSelector does not exist!
[camera_driver_node-1] [WARN] [1711089344.919999946] [cam1]: ChunkDataControl/ChunkSelector set to: UNKNOWN instead of: Timestamp
[camera_driver_node-1] [INFO] [1711089344.920020795] [cam1]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [WARN] [1711089344.920044634] [cam1]: setting ChunkDataControl/ChunkEnable failed: node ChunkDataControl/ChunkEnable does not exist!
[camera_driver_node-1] [INFO] [1711089344.927256720] [cam1]: camera has pixel format: BayerGB8
[camera_driver_node-1] [INFO] [1711089344.927309158] [cam1]: camera_driver_node started up!
[camera_driver_node-1] [INFO] [1711089348.574146518] [cam1]: rate [Hz] in  31.03 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089353.574129732] [cam1]: rate [Hz] in  31.01 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089358.574139210] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089363.574141878] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
...
$ ros2 topic hz /cam1/image_raw
(show nothing)
berndpfrommer commented 3 months ago

That you don't have a camera calibration file is not the problem. The error message can be ignored so long as you don't need the driver to publish a valid camera_info topic. If you do need a calibration file, please use this package in conjunction with the driver. Note that IMO the documentation for the calibration package is in poor shape and I cannot provide support for it. From the driver log output it looks as if the driver is connecting to the camera and getting input (it reports incoming images at a frame rate of 31Hz). The most likely reason you are not seeing any images is because something is not working right with the ROS2 middleware (rmw), which is not directly related to this driver. Couple of points to check:

SangHunHan92 commented 3 months ago 

- What do you get when doing ros2 node <driver_node_name> info? What topics is the node publishing?
Answer above.
- On the image topic, what do you see when you do ros2 topic info <name_of_image_topic>?
Sometimes a random ros2 daemon stop && ros2 daemon start clears up issues like that.
I command ```ros2 daemon stop && ros2 daemon start```, but the camera image is still not visible.

First of all, even if it is a Flea3 GigE Camera, does it have to work properly?
berndpfrommer commented 3 months ago

Define "properly". You are running the config file for a blackfly_s, so probably many of the parameters will not work properly. You will need to create your own file for the flea type cameras. I'd greatly appreciate your PR for that since I do not own a flea camera.

But the config file is not the problem. You should definitely see something with ros2 topic hz. Or ros2 topic echo /cam_test/image_raw. The driver is clearly getting images from the camera. Please restart your driver node, and your listener node. See if you can get the ROS2 talker/listener examples to work. Then from the same terminals (shell) launch the driver and do ros2 topic echo.

SangHunHan92 commented 3 months ago

This seems to be a problem caused by the ROS2 driver not having appropriate settings for the Flea3 GigE camera. I was lucky enough to get a Blackfly S GigE camera, and it works well without any problems.

So now I'm wondering what I need to modify code to make the Flea3 GigE camera work well. The result of running the GigE camera above shows that the node setting fails. (ex. AcquisitionControl/AcquisitionFrameRate, ChunkDataControl/ChunkModeActive, ...)

(“properly” means that Flea3 GigE will work with the Blackfly S settings without any code modification.)

$ ros20@ros20-MS-7B89:~/workspace/install/spinnaker_camera_driver/share/spinnaker_camera_driver/launch$ ros2 launch spinnaker_camera_driver driver_node.launch.py camera_type:=blackfly_s serial:="'20407111'" camera_name:=blackfly
[INFO] [launch]: All log files can be found below /home/ros20/.ros/log/2024-03-22-16-51-55-645147-ros20-MS-7B89-15659
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [camera_driver_node-1]: process started with pid [15661]
[camera_driver_node-1] [INFO] [1711093915.739765173] [blackfly]: adjusting time stamps!
[camera_driver_node-1] [INFO] [1711093915.739852663] [blackfly]: looking for serial number: 20407111
[camera_driver_node-1] [INFO] [1711093915.739861823] [blackfly]: parameter definitions file: /home/ros20/workspace/install/spinnaker_camera_driver/share/spinnaker_camera_driver/config/blackfly_s.yaml
[camera_driver_node-1] [INFO] [1711093915.741027423] [blackfly]: using default calibration URL
[camera_driver_node-1] [INFO] [1711093915.741040813] [blackfly]: camera calibration URL: file:///home/ros20/.ros/camera_info/blackfly.yaml
[camera_driver_node-1] [ERROR] [1711093915.741065833] [camera_calibration_parsers]: Unable to open camera calibration file [/home/ros20/.ros/camera_info/blackfly.yaml]
[camera_driver_node-1] [WARN] [1711093915.741070813] [blackfly]: Camera calibration file /home/ros20/.ros/camera_info/blackfly.yaml not found
[camera_driver_node-1] [INFO] [1711093917.031390957] [blackfly]: using spinnaker lib version: 3.1.0.79
[camera_driver_node-1] [INFO] [1711093917.031664547] [blackfly]: found camera with serial number: 20407111
[camera_driver_node-1] [INFO] [1711093917.333895918] [blackfly]: setting AnalogControl/GainAuto to: Continuous
[camera_driver_node-1] [INFO] [1711093917.340885195] [blackfly]: setting AcquisitionControl/ExposureAuto to: Continuous
[camera_driver_node-1] [INFO] [1711093917.347811954] [blackfly]: setting AcquisitionControl/AcquisitionFrameRateEnable to: 1
[camera_driver_node-1] [INFO] [1711093917.351906842] [blackfly]: setting AcquisitionControl/AcquisitionFrameRate to: 30
[camera_driver_node-1] [INFO] [1711093917.359296260] [blackfly]: setting AcquisitionControl/TriggerMode to: Off
[camera_driver_node-1] [INFO] [1711093917.362767818] [blackfly]: setting ChunkDataControl/ChunkModeActive to: 1
[camera_driver_node-1] [INFO] [1711093917.368112477] [blackfly]: setting ChunkDataControl/ChunkSelector to: FrameID
[camera_driver_node-1] [INFO] [1711093917.371762186] [blackfly]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [INFO] [1711093917.376229814] [blackfly]: setting ChunkDataControl/ChunkSelector to: ExposureTime
[camera_driver_node-1] [INFO] [1711093917.377291394] [blackfly]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [INFO] [1711093917.380904914] [blackfly]: setting ChunkDataControl/ChunkSelector to: Gain
[camera_driver_node-1] [INFO] [1711093917.381947053] [blackfly]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [INFO] [1711093917.385558601] [blackfly]: setting ChunkDataControl/ChunkSelector to: Timestamp
[camera_driver_node-1] [INFO] [1711093917.386611681] [blackfly]: setting ChunkDataControl/ChunkEnable to: 1
[camera_driver_node-1] [INFO] [1711093917.410612073] [blackfly]: camera has pixel format: BayerRG8
[camera_driver_node-1] [INFO] [1711093917.410648633] [blackfly]: camera_driver_node started up!
[camera_driver_node-1] [INFO] [1711093920.739829535] [blackfly]: rate [Hz] in  30.05 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093925.739827243] [blackfly]: rate [Hz] in  30.00 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093930.739816473] [blackfly]: rate [Hz] in  29.99 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093935.739813486] [blackfly]: rate [Hz] in  29.99 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093940.739810245] [blackfly]: rate [Hz] in  29.99 out   7.40 drop   0%
[camera_driver_node-1] [INFO] [1711093945.739815599] [blackfly]: rate [Hz] in  29.99 out  30.00 drop   0%
[camera_driver_node-1] [INFO] [1711093950.739815957] [blackfly]: rate [Hz] in  29.99 out  19.60 drop   0%
[camera_driver_node-1] [INFO] [1711093955.739813635] [blackfly]: rate [Hz] in  29.99 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093960.739823051] [blackfly]: rate [Hz] in  29.99 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093965.739812564] [blackfly]: rate [Hz] in  29.99 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093970.739815049] [blackfly]: rate [Hz] in  29.99 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711093975.739814326] [blackfly]: rate [Hz] in  29.99 out  30.00 drop   0%
[camera_driver_node-1] [INFO] [1711093980.739813002] [blackfly]: rate [Hz] in  29.99 out  30.00 drop   0%
[camera_driver_node-1] [INFO] [1711093985.739812084] [blackfly]: rate [Hz] in  29.99 out  30.00 drop   0%
[camera_driver_node-1] [INFO] [1711093990.739815209] [blackfly]: rate [Hz] in  29.99 out  30.00 drop   0%
[camera_driver_node-1] [INFO] [1711093995.739815587] [blackfly]: rate [Hz] in  29.99 out  30.00 drop   0%
berndpfrommer commented 3 months ago

I have a really hard time believing that. In your earlier log you can see a bunch of error messages that are caused because you were using the blackfly_s.yaml file, but then the driver says this:

[camera_driver_node-1] [INFO] [1711089344.927309158] [cam1]: camera_driver_node started up!
[camera_driver_node-1] [INFO] [1711089348.574146518] [cam1]: rate [Hz] in  31.03 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089353.574129732] [cam1]: rate [Hz] in  31.01 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089358.574139210] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089363.574141878] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%

That means it's getting images, and it should publish those. Once it gets images it should also publish them. Could you please repeat the experiment with the flea to make 100% sure that rostopic echo does not produce output?

berndpfrommer commented 3 months ago

Instructions for making your own camera config file are here

SangHunHan92 commented 3 months ago

I have a really hard time believing that. In your earlier log you can see a bunch of error messages that are caused because you were using the blackfly_s.yaml file, but then the driver says this:

[camera_driver_node-1] [INFO] [1711089344.927309158] [cam1]: camera_driver_node started up!
[camera_driver_node-1] [INFO] [1711089348.574146518] [cam1]: rate [Hz] in  31.03 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089353.574129732] [cam1]: rate [Hz] in  31.01 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089358.574139210] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711089363.574141878] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%

That means it's getting images, and it should publish those. Once it gets images it should also publish them. Could you please repeat the experiment with the flea to make 100% sure that rostopic echo does not produce output?

Yes, I already did that, but ros2 topic echo shows nothing and out hz also shows 0.

$ ros20@ros20-MS-7B89:~/workspace/code$ ros2 topic echo /cam1/image_raw
(show nothing)
(after command "ros2 topic echo")
[camera_driver_node-1] [INFO] [1711097593.223064011] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711097598.223062704] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711097603.223061516] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
[camera_driver_node-1] [INFO] [1711097608.223063417] [cam1]: rate [Hz] in  31.00 out   0.00 drop   0%
berndpfrommer commented 3 months ago

Thanks for checking! One more last one: can you post the output of ros2 topic info /cam1/image_raw while you have both the ros2 topic echo and the driver running? If that shows a publisher and subscriber, then it's not a ROS2 rmw issue.

Are you compiling from source? If yes, then can you re-instate this printout to see if that causes the images not to be published? It's the only alternative I can think of right now and it could be an issue with GigE cameras.

berndpfrommer commented 3 months ago

BTW the "image incomplete failure" is usually due to the NIC MTU not set to 9000 and/or the camera transport MTU not set to 9000. The blackfly_s.yaml file defines parameters for setting the MTU on the camera, but from your driver logs it looks like you are not trying to set the MTU. It would likely fail anyways because you'd have to adapt the node name in the .yaml file to match what you see in SpinView.

SangHunHan92 commented 3 months ago

As you mentioned, I didn't set MTU to 9000. Now I set MTU to 9000 with /etc/netplan/01-network-manager-all.yaml, but error message such as Image incomplete: Image has missing packets. Potential fixes include enabling jumbo packets and adjusting packet size/delay. For more information see https://www.flir.com/support-center/iis/machine-vision/application-note/troubleshooting-image-consistency-errors/ is still occurring intermittently when run driver. But this isn't a big problem.

$ ifconfig
enp37s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet 192.168.1.15  netmask 255.255.255.0  broadcast 192.168.1.255
(...)

Below is the result of ros2 topic info /cam1/image_raw when I run both driver and ros2 topic echo /cam1/image_raw.

$ ros2 topic info /cam1/image_raw
Type: sensor_msgs/msg/Image
Publisher count: 1
Subscription count: 1

I also tried saving the topic with rosbag, but nothing was saved.

I'm still not familiar with ros and c++, so it might take some time to re-instate the printout.

berndpfrommer commented 3 months ago

Ok so the topic info shows that there is indeed a subscriber to the topic, and if anything is being sent by the driver, it should receive it. You were able to compile the driver from source, so modifying is no big deal. Use any editor you like to modify the file spinnaker_wrapper_impl.cpp here such that it looks like this:

  if (imgPtr->IsIncomplete()) {
    numIncompleteImages_++;
#if 1
    // Retrieve and print the image status description
    std::cout << "Image incomplete: "
              << Spinnaker::Image::GetImageStatusDescription(imgPtr->GetImageStatus()) << std::endl;
#endif
  } else {
    std::cout << "Image complete!" << std::endl;
    float expTime = 0;
    float gain = 0;

and then build the workspace again using colcon. If you see any printouts that say "Image complete" then you hopefully will also see images being published. If not, then none of your images arrive completely, i.e. too many packages are dropped on the network. Not sure how you got the "Image incomplete" printouts without enabling the error message above. And how you concluded it isn't a big problem.

SangHunHan92 commented 3 months ago 
ros20@ros20-MS-7B89:~/workspace$ ros2 launch spinnaker_camera_driver driver_node.launch.py camera_type:=blackfly_s serial:="'20306011'" camera_name:=cam1
[INFO] [launch]: All log files can be found below /home/ros20/.ros/log/2024-03-26-14-25-26-706959-ros20-MS-7B89-47305
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [camera_driver_node-1]: process started with pid [47307]
[camera_driver_node-1] [INFO] [1711430726.797673886] [cam1]: adjusting time stamps!
[camera_driver_node-1] [INFO] [1711430726.797762566] [cam1]: looking for serial number: 20306011
[camera_driver_node-1] [INFO] [1711430726.797772096] [cam1]: parameter definitions file: /home/ros20/workspace/install/spinnaker_camera_driver/share/spinnaker_camera_driver/config/blackfly_s.yaml
[camera_driver_node-1] [INFO] [1711430726.798959960] [cam1]: using default calibration URL
[camera_driver_node-1] [INFO] [1711430726.798973320] [cam1]: camera calibration URL: file:///home/ros20/.ros/camera_info/cam1.yaml
[camera_driver_node-1] [ERROR] [1711430726.798996491] [camera_calibration_parsers]: Unable to open camera calibration file [/home/ros20/.ros/camera_info/cam1.yaml]
[camera_driver_node-1] [WARN] [1711430726.799001721] [cam1]: Camera calibration file /home/ros20/.ros/camera_info/cam1.yaml not found
[camera_driver_node-1] [INFO] [1711430728.091736411] [cam1]: using spinnaker lib version: 3.1.0.79
[camera_driver_node-1] [INFO] [1711430728.092019832] [cam1]: found camera with serial number: 20306011
[camera_driver_node-1] [INFO] [1711430728.146473866] [cam1]: setting AnalogControl/GainAuto to: Continuous
[camera_driver_node-1] [INFO] [1711430728.148289873] [cam1]: setting AcquisitionControl/ExposureAuto to: Continuous
[camera_driver_node-1] [INFO] [1711430728.150032850] [cam1]: setting AcquisitionControl/AcquisitionFrameRateEnable to: 1
[camera_driver_node-1] [WARN] [1711430728.150154160] [cam1]: setting AcquisitionControl/AcquisitionFrameRateEnable failed: node AcquisitionControl/AcquisitionFrameRateEnable does not exist!
[camera_driver_node-1] [INFO] [1711430728.150214680] [cam1]: setting AcquisitionControl/AcquisitionFrameRate to: 30
(say setting is failed ...)
[camera_driver_node-1] [INFO] [1711430728.219827581] [cam1]: camera has pixel format: BayerGB8
[camera_driver_node-1] [INFO] [1711430728.219895871] [cam1]: camera_driver_node started up!
[camera_driver_node-1] Image complete!
[camera_driver_node-1] Image complete!
(...)
[camera_driver_node-1] [INFO] [1711430731.797757983] [cam1]: rate [Hz] in  31.05 out   0.00 drop   0%
[camera_driver_node-1] Image complete!
[camera_driver_node-1] Image complete!

Nice! I can get "Image complete!" message. Actually, I compiled it with Tags:2.1.10 version. The latest version, Branch:humble-devel, does not compile due to a target link problem. Below is the code I modified.

   if (imgPtr->IsIncomplete()) {
     // Retrieve and print the image status description
     std::cout << "Image incomplete: "
               << Spinnaker::Image::GetImageStatusDescription(imgPtr->GetImageStatus()) << std::endl;
   } else {
     std::cout << "Image complete!" << std::endl;
     const Spinnaker::ChunkData & chunk = imgPtr->GetChunkData();
     const float expTime = chunk.GetExposureTime();
     const float gain = chunk.GetGain();
     const int64_t stamp = chunk.GetTimestamp();
     const uint32_t maxExpTime =
       static_cast<uint32_t>(is_readable(exposureTimeNode_) ? exposureTimeNode_->GetMax() : 0);

What should I try next?

SangHunHan92 commented 3 months ago

The reason I concluded that it was not a big problem is because after installing the flir driver, there was intermittent data loss (about once every 10 seconds) when reading the camera through cv2 in python. Nevertheless, there was generally no problem reading the camera's image sequence. I think this is a problem with my settings, not a problem with the ROS2 driver.

berndpfrommer commented 3 months ago

Are you still not seeing any images with e.g. rqt_image_view, although you are now getting "image complete" messages? That's the part I cannot understand. My guess what's going on (or what was going on) is this: Your yaml file is incorrect because it's a new camera type (flea). That means the camera parameters, in particular the MTU is not set right. This in turn causes such severe packet loss that only incomplete images arrive at the ROS driver and hence it doesn't publish any. If you are still not seeing the driver publish images, despite it now printing out "image complete", then I'm out of ideas why it's not working. I see you are still getting error messages on startup because you have not adjusted the blackfly_s.yaml file. I suggest you make a copy of it (call it flea.yaml). Then hack the launch file to point to that file. Look at the node names in SpinView, and modify your flea.yaml file correspondingly. Then you can set e.g. the frame rate, the MTU etc via ros parameters. No need to write code, you just need to edit the flea.yaml file until you get no error messages on startup. And you only have to touch the ROS parameters that you really need, like frame rate enabled, frame rate etc.

berndpfrommer commented 3 months ago

Oh, and can you post the target link error message you are getting with the latest humble driver?

SangHunHan92 commented 3 months ago

Sure.

$ colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
Starting >>> flir_camera_msgs
Starting >>> rosbag2_tools                     
Starting >>> flir_camera_description                                              
Starting >>> sbg_driver                                                                                            
Finished <<< flir_camera_description [0.93s]                                                                                           
Finished <<< rosbag2_tools [1.63s]                                                                                         
Starting >>> ros2bag_tools
Finished <<< flir_camera_msgs [2.06s]                                                                                 
Finished <<< sbg_driver [1.85s]                                                     
Starting >>> spinnaker_camera_driver
Finished <<< ros2bag_tools [1.56s]                                                             
--- stderr: spinnaker_camera_driver                         
CMake Error at CMakeLists.txt:96 (add_library):
  Target "camera_driver" links to target "rclcpp_components::component" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:96 (add_library):
  Target "camera_driver" links to target
  "camera_info_manager::camera_info_manager" but the target was not found.
  Perhaps a find_package() call is missing for an IMPORTED target, or an
  ALIAS target is missing?

CMake Error at CMakeLists.txt:96 (add_library):
  Target "camera_driver" links to target "image_transport::image_transport"
  but the target was not found.  Perhaps a find_package() call is missing for
  an IMPORTED target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:129 (add_executable):
  Target "camera_driver_node" links to target "rclcpp_components::component"
  but the target was not found.  Perhaps a find_package() call is missing for
  an IMPORTED target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:129 (add_executable):
  Target "camera_driver_node" links to target
  "camera_info_manager::camera_info_manager" but the target was not found.
  Perhaps a find_package() call is missing for an IMPORTED target, or an
  ALIAS target is missing?

CMake Error at CMakeLists.txt:129 (add_executable):
  Target "camera_driver_node" links to target
  "image_transport::image_transport" but the target was not found.  Perhaps a
  find_package() call is missing for an IMPORTED target, or an ALIAS target
  is missing?

CMake Error at CMakeLists.txt:96 (add_library):
  Target "camera_driver" links to target "rclcpp_components::component" but
  the target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:96 (add_library):
  Target "camera_driver" links to target
  "camera_info_manager::camera_info_manager" but the target was not found.
  Perhaps a find_package() call is missing for an IMPORTED target, or an
  ALIAS target is missing?

CMake Error at CMakeLists.txt:96 (add_library):
  Target "camera_driver" links to target "image_transport::image_transport"
  but the target was not found.  Perhaps a find_package() call is missing for
  an IMPORTED target, or an ALIAS target is missing?

CMake Generate step failed.  Build files cannot be regenerated correctly.
make: *** [Makefile:460: cmake_check_build_system] Error 1
---
Failed   <<< spinnaker_camera_driver [1.26s, exited with code 2]

Summary: 5 packages finished [5.13s]
  1 package failed: spinnaker_camera_driver
  1 package had stderr output: spinnaker_camera_driver
  1 package not processed
berndpfrommer commented 3 months ago

strange. It's not finding any of the basic ROS packages. Almost as if you hadn't sourced /opt/ros/galactic/setup.bash

I just built successfully on Ubuntu 22.04/Humble. You may have to do a rm -rf build install log to clear out state from the existing build. Hard to imagine that the new version would not build on Ubuntu 20.04/Galactic.

colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
Starting >>> flir_camera_msgs
Starting >>> flir_camera_description
--- stderr: flir_camera_description                                                                       
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_COMPILE_COMMANDS

---
Finished <<< flir_camera_description [1.13s]
Finished <<< flir_camera_msgs [5.11s]                     
Starting >>> spinnaker_camera_driver
[Processing: spinnaker_camera_driver]                         
Finished <<< spinnaker_camera_driver [52.2s]                       
Starting >>> spinnaker_synchronized_camera_driver
Finished <<< spinnaker_synchronized_camera_driver [11.0s]                           

Summary: 4 packages finished [1min 9s]
  1 package had stderr output: flir_camera_description
SangHunHan92 commented 3 months ago

I deleted all build/ install/ log/ folders, ran the command source /opt/ros/galactic/setup.bash and then ran colcon build, but a target link error still occurs.

However, with Tags:2.1.10 version, it builds without any problem.

$ rosdep install --from-paths src --ignore-src && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON && . install/setup.bash
#All required rosdeps installed successfully
[1.797s] WARNING:colcon.colcon_ros.prefix_path.ament:The path '/home/ros20/workspace/install/sbg_driver' in the environment variable AMENT_PREFIX_PATH doesn't exist
[1.797s] WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/home/ros20/workspace/install/sbg_driver' in the environment variable CMAKE_PREFIX_PATH doesn't exist
Starting >>> flir_camera_msgs
Starting >>> rosbag2_tools                     
Starting >>> flir_camera_description                                              
Starting >>> sbg_driver                                                               
Finished <<< flir_camera_description [0.96s]                                                     
Finished <<< flir_camera_msgs [1.40s]                                                         
Starting >>> spinnaker_camera_driver
Finished <<< rosbag2_tools [1.55s]                                                         
Starting >>> ros2bag_tools
Finished <<< ros2bag_tools [1.21s]                               
Finished <<< sbg_driver [4.29s]                                  
Finished <<< spinnaker_camera_driver [16.0s]                       

Summary: 6 packages finished [19.0s]
SangHunHan92 commented 3 months ago

I confirmed two facts through several experiments.

  1. I modified spinnaker_wrapper_impl.cpp as below and got meaningful output from blackfly_s camera.
  2. Even though I created a yaml file specifically for the flea camera, I still can't get the image. 
    
### spinnaker_wrapper_impl.cpp ###
if (imgPtr->IsIncomplete()) {
// Retrieve and print the image status description
std::cout << "Image incomplete: "
          << Spinnaker::Image::GetImageStatusDescription(imgPtr->GetImageStatus()) << std::endl;
} else {
// std::cout << "Image complete!" << std::endl;
const Spinnaker::ChunkData & chunk = imgPtr->GetChunkData();
const float expTime = chunk.GetExposureTime();
const float gain = chunk.GetGain();
const int64_t stamp = chunk.GetTimestamp();
const uint32_t maxExpTime =
  static_cast<uint32_t>(is_readable(exposureTimeNode_) ? exposureTimeNode_->GetMax() : 0);

if 1

std::cout << "got image: " << imgPtr->GetWidth() << "x"
          << imgPtr->GetHeight() << " stride: " << imgPtr->GetStride()
          << " ts: " << stamp << " exp time: " << expTime
          << " gain: " << gain << " bpp: " << imgPtr->GetBitsPerPixel()
          << " chan: " << imgPtr->GetNumChannels()
          << " tl payload type: " << imgPtr->GetTLPayloadType()
          << " tl pix fmt: " << imgPtr->GetTLPixelFormat()
          << " payload type: " << imgPtr->GetPayloadType()
          << " pixfmt enum: " << imgPtr->GetPixelFormat()
          << " fmt: " << imgPtr->GetPixelFormatName()
          << " int type: " << imgPtr->GetPixelFormatIntType()
          << " frame id: " << imgPtr->GetFrameID()
          << " img id: " << imgPtr->GetID() << std::endl;

endif

$ ros2 launch spinnaker_camera_driver driver_node.launch.py camera_type:=blackfly_s serial:="'20407111'" camera_name:=blackfly [INFO] [launch]: All log files can be found below /home/ros20/.ros/log/2024-03-26-16-45-27-670663-ros20-MS-7B89-85098 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [camera_driver_node-1]: process started with pid [85100] [camera_driver_node-1] [INFO] [1711439127.759432862] [blackfly]: adjusting time stamps! [camera_driver_node-1] [INFO] [1711439127.759521882] [blackfly]: looking for serial number: 20407111 [camera_driver_node-1] [INFO] [1711439127.759530702] [blackfly]: parameter definitions file: /home/ros20/workspace/install/spinnaker_camera_driver/share/spinnaker_camera_driver/config/blackfly_s.yaml (setting blackfly_s config ...) [camera_driver_node-1] [INFO] [1711439129.435888178] [blackfly]: camera has pixel format: BayerRG8 [camera_driver_node-1] [INFO] [1711439129.435931218] [blackfly]: camera_driver_node started up! [camera_driver_node-1] got image: 2048x1536 stride: 2048 ts: 3169312330352 exp time: 7789 gain: 0 bpp: 8 chan: 1 tl payload type: 1 tl pix fmt: 17301513 payload type: 5 pixfmt enum: 4 fmt: BayerRG8 int type: 0 frame id: 1 img id: 93834223826448 [camera_driver_node-1] got image: 2048x1536 stride: 2048 ts: 3169345679752 exp time: 7789 gain: 0 bpp: 8 chan: 1 tl payload type: 1 tl pix fmt: 17301513 payload type: 5 pixfmt enum: 4 fmt: BayerRG8 int type: 0 frame id: 2 img id: 93834223826544 [camera_driver_node-1] got image: 2048x1536 stride: 2048 ts: 3169379029000 exp time: 7789 gain: 5.29926 bpp: 8 chan: 1 tl payload type: 1 tl pix fmt: 17301513 payload type: 5 pixfmt enum: 4 fmt: BayerRG8 int type: 0 frame id: 3 img id: 93834223826640 [camera_driver_node-1] got image: 2048x1536 stride: 2048 ts: 3169412378392 exp time: 10165 gain: 8.49917 bpp: 8 chan: 1 tl payload type: 1 tl pix fmt: 17301513 payload type: 5 pixfmt enum: 4 fmt: BayerRG8 int type: 0 frame id: 4 img id: 93834223826768 (...)

Even though I created a yaml file specifically for flea camera and no error occurred in the settings, I still cannot get images.
Below are my .yaml file and result.

flea.yaml

parameters: #

-------- image format control

#

Therefore I guess that the value(e.f. IsIncomplete(), GetWidth(), ...) does not exist in imgPtr.

berndpfrommer commented 3 months ago

Great, you got a working yaml file! I believe the reason you are seeing this problem is because chunkData is not enabled. This will cause an exception that causes the images to be dropped. I did not catch this problem earlier because I was looking at the most recent source code, where this problem has been fixed. It is however a problem with the older version you are running (2.1.10). So you can try and look at the most recent source code and see the changes, in particular the exception handling here. There is a comment in the code about that. The other alternative you have is to enable all Chunk Data (exposure, time stamp, frame number etc, all of it), if that is possible on the flea (should be).

There's something funky going on about the driver not building on galactic. See also issue #164 I'm debugging it right now under PR #165

SangHunHan92 commented 3 months ago

Unfortunately, unlike the Blackfly S, the Flea3 does not have any options regarding Chunk Data. Therefore, I don't know if it is possible to enable ChunkData. I inevitably upload two photos of SpinView for comparison. This is why I removed the settings regarding Chunk from flea.yaml.

image image

berndpfrommer commented 3 months ago

Too bad. Can you try the latest driver I pushed under PR #165 ? It should build on galactic and fix the bugs you are seeing.

berndpfrommer commented 3 months ago

never mind, still some bugs under galactic.

berndpfrommer commented 3 months ago

The latest driver now builds under galactic. If you don't already have spinnaker installed it will download and compile Spinnaker v2.x which is fairly old. You can build against a more recent version of spinnaker by installing it (under /opt/spinnaker) before you compile the ros driver. Please let me know if this works for you.

SangHunHan92 commented 3 months ago

I succeeded in building the ROS2 driver, but I cannot run it. I use Spinnaker-2.2.0.48-Linux, and SpinView is working well.

$ colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
Starting >>> flir_camera_msgs
Starting >>> rosbag2_tools                     
Starting >>> flir_camera_description                                              
Starting >>> sbg_driver                                                                                            
Finished <<< rosbag2_tools [1.55s]                                                                                           
Starting >>> ros2bag_tools
Finished <<< flir_camera_description [1.82s]                                                                      
Finished <<< ros2bag_tools [1.96s]                                                                                        
Finished <<< flir_camera_msgs [4.85s]                                                               
Starting >>> spinnaker_camera_driver
Finished <<< spinnaker_camera_driver [24.8s]                                                                   
Starting >>> spinnaker_synchronized_camera_driver
Finished <<< spinnaker_synchronized_camera_driver [8.33s]
Finished <<< sbg_driver [49.7s]
Summary: 7 packages finished [52.0s]
$ ros2 launch spinnaker_camera_driver driver_node.launch.py camera_type:=blackfly_s serial:="'20407111'"
[INFO] [launch]: All log files can be found below /home/ros20/.ros/log/2024-03-27-20-25-21-951610-ros20-MS-7B89-1058654
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [camera_driver_node-1]: process started with pid [1058656]
[camera_driver_node-1] /home/ros20/workspace2/install/spinnaker_camera_driver/lib/spinnaker_camera_driver/camera_driver_node: symbol lookup error: /home/ros20/workspace2/install/spinnaker_camera_driver/lib/libcamera_driver.so: undefined symbol: _ZN12class_loader4impl22AbstractMetaObjectBaseC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_S9_
[ERROR] [camera_driver_node-1]: process has died [pid 1058656, exit code 127, cmd '/home/ros20/workspace2/install/spinnaker_camera_driver/lib/spinnaker_camera_driver/camera_driver_node --ros-args -r __node:=flir_camera --params-file /tmp/launch_params_xxwhxmv9 --params-file /tmp/launch_params_45zy2yxb -r ~/control:=/exposure_control/control'].
berndpfrommer commented 3 months ago

Hmm. Works in my Ubuntu 20.04/Galactic docker container (output below). Also works on my laptop (Ubuntu 22.04/Humble). Please double check that you have done a rm -rf build log install at the top of your workspace. The error you are getting is really weird. Also double check that you sourced /opt/ros/galactic/setup.bash, and not something else. Maybe best to start a fresh shell.

pfrommer@galactic_dev:~/Documents/flir$ ros2 launch spinnaker_camera_driver driver_node.launch.py 
[INFO] [launch]: All log files can be found below /home/pfrommer/.ros/log/2024-03-27-12-43-18-060835-galactic_dev-16997
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [camera_driver_node-1]: process started with pid [17003]
[camera_driver_node-1] [INFO] [1711543398.199997123] [20435008]: reading ros parameters for camera with serial: 20435008
[camera_driver_node-1] [INFO] [1711543398.200076952] [20435008]: adjusting time stamps!
[camera_driver_node-1] [INFO] [1711543398.200264338] [20435008]: parameter definitions file: /home/pfrommer/Documents/flir/install/spinnaker_camera_driver/share/spinnaker_camera_driver/config/blackfly_s.yaml
[camera_driver_node-1] [INFO] [1711543398.202156290] [flir_camera]: using default calibration URL
[camera_driver_node-1] [INFO] [1711543398.202219497] [flir_camera]: camera calibration URL: file:///home/pfrommer/.ros/camera_info/flir_camera.yaml
[camera_driver_node-1] [ERROR] [1711543398.202274742] [camera_calibration_parsers]: Unable to open camera calibration file [/home/pfrommer/.ros/camera_info/flir_camera.yaml]
[camera_driver_node-1] [WARN] [1711543398.202283892] [flir_camera]: Camera calibration file /home/pfrommer/.ros/camera_info/flir_camera.yaml not found
[camera_driver_node-1] [INFO] [1711543399.089992477] [20435008]: using spinnaker lib version: 2.2.0.48
[camera_driver_node-1] [WARN] [1711543399.090570630] [20435008]: no camera found with serial: 20435008 on try # 1
SangHunHan92 commented 3 months ago

Wow, incredible! Both blackfly_s & flea camera are working well !! Thank you for your efforts.

Now I just want to solve 2 simple problem.

  1. Both blackfly_s and flea camera show grayscale image captured with ROS2 driver. However, Spinview displays RGB images well.
  2. flea camera image taken with the ROS2 driver show subtle grid-structure noise.

It seems like these can be solved easily. How can control these?

untitled blackfly_s with SpinView

1711593934900163768 blackfly_s with ROS2 driver

untitled flea with SpinView

1711605084365472269 flea with ROS2 driver

berndpfrommer commented 3 months ago

You are looking at a bayer image. Google bayer filters, and how color cameras work. Try setting the pixel_format parameter for the blackfly_s to an allowed value, see here. There could be more values allowed, you can explore those with spinview. What viewer are you using? rqt_image_view? For me the viewer usually de-bayers the image automatically. Whatever, play around with pixel_format to see if it helps. Next, figure out how to set the pixel format for flea (you will need to modify the flea.yaml file to introduce a pixel_format parameter). Would love to see a PR with your flea.yaml file once you have it working.

SangHunHan92 commented 3 months ago

I was able to show a color image after changing setting RGB8Packed in SpinView. When I set the PixelFormat to various settings via .yaml alone before, I still only show the bayer image with rqt_image_view. I don't know why. However, this is simply a visualization issue; there was no problem converting the Bayer image to RGB. Thanks !

flea3.yaml was configured with minimal settings to avoid errors. I also need to modify driver_node.launch.py as well. One thing to note, you should use "BayerGB8" as default and not "BayerRG8" for flea3 camera!

flea3.yaml

# 
# config file for Flea3 cameras (GigE)
#
# This file maps the ros parameters to the corresponding Spinnaker "nodes" in the camera.
# For more details on how to modify this file, see the README on camera configuration files.

parameters:
  #
  # -------- image format control
  #
  - name: pixel_format
    type: enum
    # Check available values with SpinView. Not all are supported by ROS!
    # Some formats are e.g. "Mono8", "Mono16", "Mono12Packed", "BayerGB8", "BayerGB12Packed", "BayerGB16", "RGB8Packed"
    # default is "BayerGB8"
    node: ImageFormatControl/PixelFormat
  - name: binning_x
    type: int
    node: ImageFormatControl/BinningHorizontal
  - name: binning_y
    type: int
    node: ImageFormatControl/BinningVertical
  - name: image_width
    type: int
    node: ImageFormatControl/Width
  - name: image_height
    type: int
    node: ImageFormatControl/Height
  - name: offset_x # offset must come after image width reduction!
    type: int
    node: ImageFormatControl/OffsetX
  - name: offset_y
    type: int
    node: ImageFormatControl/OffsetY

  #
  # -------- analog control
  #
  - name: gain_auto
    type: enum
    # valid values are "Continuous", "Off"
    node: AnalogControl/GainAuto
  - name: gain
    type: float
    node: AnalogControl/Gain
  - name: gamma_enabled
    type: bool
    node: AnalogControl/GammaEnabled
  #
  # -------- device link throughput limiting
  #
  - name: device_link_throughput_limit
    type: int
    node: DeviceControl/DeviceLinkThroughputLimit
  #
  # -------- transport layer control (GigE)
  #
  - name: gev_scps_packet_size
    type: int
    # default is 1400. Set to 9000 to enable jumbo frames, ensure NIC MTU set >= 9000
    node: TransportLayerControl/GigEVision/GevSCPSPacketSize
  #
  # -------- digital IO control
  #
  - name: line0_selector # black wire: opto-isolated input
    type: enum
    node: DigitalIOControl/LineSelector
  - name: line1_selector # white wire: opto-isolated output
    type: enum
    node: DigitalIOControl/LineSelector
  - name: line1_linemode # valid values: "Input", "Output"
    type: enum
    node: DigitalIOControl/LineMode
  - name: line2_selector # red wire: non-isolated input/output
    type: enum
    node: DigitalIOControl/LineSelector
  - name: line2_v33enable # red wire: 3.3V power
    type: bool
    node: DigitalIOControl/V3_3Enable
  - name: line3_selector # green wire: aux voltage input and non-isolated input
    type: enum
    node: DigitalIOControl/LineSelector
  - name: line3_linemode # valid values: "Input", "Output"
    type: enum
    node: DigitalIOControl/LineMode
  #
  # -------- acquisition control
  #
  - name: exposure_auto
    type: enum
    # valid values are "Off", "Continuous"
    node: AcquisitionControl/ExposureAuto
  - name: exposure_time
    type: float
    node: AcquisitionControl/ExposureTime
  # - name: frame_rate_enable
  #   type: bool
  #   node: AcquisitionControl/AcquisitionFrameRateEnable
  # - name: frame_rate
  #   type: float
  #   node: AcquisitionControl/AcquisitionFrameRate
  - name: trigger_selector
    type: enum
    # valid values are e.g. "FrameStart", "AcquisitionStart", "FrameBurstStart"
    node: AcquisitionControl/TriggerSelector
  - name: trigger_mode
    type: enum
    # valid values are "On" and "Off"
    node: AcquisitionControl/TriggerMode
  - name: trigger_source
    type: enum
    # valid values are "Line<0,1,2>", "UserOutput<0,1,2>", "Counter<0,1><Start/End>",
    # "LogicBlock<0,1>
    node: AcquisitionControl/TriggerSource
  - name: trigger_delay
    # value >= 9
    type: float
    node: AcquisitionControl/TriggerDelay
  - name: trigger_overlap
    type: enum
    # valid values: "Off" and "ReadOut"
    node: AcquisitionControl/TriggerOverlap
  - name: trigger_activation
    type: enum
    # valid values: "LevelLow", "LevelHigh", "FallingEdge", "RisingEdge", "AnyEdge",
    node: AcquisitionControl/TriggerActivation

driver_node.launch.py

# -----------------------------------------------------------------------------
# Copyright 2022 Bernd Pfrommer <bernd.pfrommer@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#

from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument as LaunchArg
from launch.actions import OpaqueFunction
from launch.substitutions import LaunchConfiguration as LaunchConfig
from launch.substitutions import PathJoinSubstitution

from launch_ros.actions import Node
from launch_ros.substitutions import FindPackageShare

example_parameters = {
    'blackfly_s': {
        'debug': False,
        'compute_brightness': False,
        'adjust_timestamp': True,
        'dump_node_map': False,
        # set parameters defined in blackfly_s.yaml
        'gain_auto': 'Continuous',
        # 'pixel_format': 'BayerRG8',
        'exposure_auto': 'Continuous',
        # These are useful for GigE cameras
        # 'device_link_throughput_limit': 380000000,
        # 'gev_scps_packet_size': 9000,
        # ---- to reduce the sensor width and shift the crop
        # 'image_width': 1408,
        # 'image_height': 1080,
        # 'offset_x': 16,
        # 'offset_y': 0,
        # 'binning_x': 1,
        # 'binning_y': 1,
        # 'connect_while_subscribed': True,
        'frame_rate_auto': 'Off',
        'frame_rate': 10.0,
        'frame_rate_enable': True,
        'buffer_queue_size': 10,
        'trigger_mode': 'Off',
        'chunk_mode_active': True,
        'chunk_selector_frame_id': 'FrameID',
        'chunk_enable_frame_id': True,
        'chunk_selector_exposure_time': 'ExposureTime',
        'chunk_enable_exposure_time': True,
        'chunk_selector_gain': 'Gain',
        'chunk_enable_gain': True,
        'chunk_selector_timestamp': 'Timestamp',
        'chunk_enable_timestamp': True},
    'flea3': {
        'debug': False,
        'compute_brightness': False,
        'adjust_timestamp': True,
        'dump_node_map': False,
        # set parameters defined in flea3.yaml
        'gain_auto': 'Continuous',
        'gamma_enabled': False,
        'pixel_format': 'BayerGB8', # "Mono8", "Mono16", "Mono12Packed", "BayerGB8", "BayerGB12Packed", "BayerGB16", "RGB8Packed"
        'exposure_auto': 'Continuous',
        # These are useful for GigE cameras
        # 'device_link_throughput_limit': 380000000,
        # 'gev_scps_packet_size': 9000,
        # ---- to reduce the sensor width and shift the crop
        # 'image_width': 1408,
        # 'image_height': 1080,
        # 'offset_x': 16,
        # 'offset_y': 0,
        # 'binning_x': 1,
        # 'binning_y': 1,
        # 'connect_while_subscribed': True,
        'frame_rate_auto': 'Off',
        'frame_rate': 40.0,
        'frame_rate_enable': True,
        'buffer_queue_size': 10,
        'trigger_mode': 'Off'},
    'chameleon': {
        'debug': False,
        'compute_brightness': False,
        'dump_node_map': False,
        # set parameters defined in chameleon.yaml
        'gain_auto': 'Continuous',
        'exposure_auto': 'Continuous',
        'offset_x': 0,
        'offset_y': 0,
        'image_width': 2048,
        'image_height': 1536,
        'pixel_format': 'RGB8',  # 'BayerRG8, 'RGB8' or 'Mono8'
        'frame_rate_continous': True,
        'frame_rate': 100.0,
        'trigger_mode': 'Off',
        'chunk_mode_active': True,
        'chunk_selector_frame_id': 'FrameID',
        'chunk_enable_frame_id': True,
        'chunk_selector_exposure_time': 'ExposureTime',
        'chunk_enable_exposure_time': True,
        'chunk_selector_gain': 'Gain',
        'chunk_enable_gain': True,
        'chunk_selector_timestamp': 'Timestamp',
        'chunk_enable_timestamp': True},
    'grasshopper': {
        'debug': False,
        'compute_brightness': False,
        'dump_node_map': False,
        # set parameters defined in grasshopper.yaml
        'gain_auto': 'Continuous',
        'exposure_auto': 'Continuous',
        'frame_rate_auto': 'Off',
        'frame_rate': 100.0,
        'trigger_mode': 'Off',
        'chunk_mode_active': True,
        'chunk_selector_frame_id': 'FrameID',
        'chunk_enable_frame_id': True,
        'chunk_selector_exposure_time': 'ExposureTime',
        'chunk_enable_exposure_time': True,
        'chunk_selector_gain': 'Gain',
        'chunk_enable_gain': True,
        'chunk_selector_timestamp': 'Timestamp',
        'chunk_enable_timestamp': True}
    }

def launch_setup(context, *args, **kwargs):
    """Launch camera driver node."""
    parameter_file = LaunchConfig('parameter_file').perform(context)
    camera_type = LaunchConfig('camera_type').perform(context)
    if not parameter_file:
        parameter_file = PathJoinSubstitution(
            [FindPackageShare('spinnaker_camera_driver'), 'config',
             camera_type + '.yaml'])
    if camera_type not in example_parameters:
        raise Exception('no example parameters available for type ' + camera_type)

    node = Node(package='spinnaker_camera_driver',
                executable='camera_driver_node',
                output='screen',
                name=[LaunchConfig('camera_name')],
                parameters=[example_parameters[camera_type],
                            {'ffmpeg_image_transport.encoding': 'hevc_nvenc',
                             'parameter_file': parameter_file,
                             'serial_number': [LaunchConfig('serial')]}],
                remappings=[('~/control', '/exposure_control/control'), ])

    return [node]

def generate_launch_description():
    """Create composable node by calling opaque function."""
    return LaunchDescription([
        LaunchArg('camera_name', default_value=['flir_camera'],
                  description='camera name (ros node name)'),
        LaunchArg('camera_type', default_value='blackfly_s',
                  description='type of camera (blackfly_s, chameleon...)'),
        LaunchArg('serial', default_value="'20435008'",
                  description='FLIR serial number of camera (in quotes!!)'),
        LaunchArg('parameter_file', default_value='',
                  description='path to ros parameter definition file (override camera type)'),
        OpaqueFunction(function=launch_setup)
        ])
berndpfrommer commented 3 months ago

Thanks for posting. I will add the file to the repo. Does the driver spit out any error messages when you set the pixel_format to RGB8Packed? Or does it simply not do anything? If you do ros2 topic echo /image_topic_here | grep encoding you can see what encoding is published. Should no longer by bayer then.

SangHunHan92 commented 3 months ago

No, it doesn't spit out error messages depending on pixel_format. After setting RGB8Packed just once in SpinView, I can visualize the color image well for all types of pixel_format in ROS2. It seems that rqt_image_view simply does not display the Bayer image properly in RGB. I think I confused you a bit.

ros2 topic echo /image_topic | grep encoding also shows pixel_format properly what I set.

By the way, is there an option to publish topic such like /flir_camera/image_RGB instead of /flir_camera/image_raw ? When I record a topic image_raw published with Bayer, I have to convert it to RGB format. I would like to subscribe RGB images topic.

berndpfrommer commented 3 months ago

I am indeed confused, but let's not further dwell on that :)

As far as the images go: by ROS convention the topic name "image_raw" is used to indicate that the image is not rectified. If you want to publish a converted image in a different pixel format, the conventional way would be to start your conversion node in a "name space" such as /flir_camera/converted. Then the converted image would then be /flir_camera/converted/image_raw (and the bayer image still available under /flir_camera/image_raw). Read up about name spaces, and how to specify them in the launch file. I'd like to close this issue now as solved, since we are moving further and further away from the original issue.

SangHunHan92 commented 3 months ago

Thank you !! The flea3 camera and build issue that I had originally concerned were resolved. I'll close this issue.