search

neufieldrobotics / spinnaker_sdk_camera_driver

Point Grey (FLIR) Spinnaker based camera driver (Blackfly S etc.)
MIT License
125 stars 91 forks source link

ROS publish rate is significantly below camera framerate #128

Open rgov opened 3 years ago

rgov commented 3 years ago

System Description Camera Name and Model: FLIR Blackfly S BFS-U3-50S5C-C Operating System: Ubuntu 18.04 Spinnaker Version: 2.4.0.143 (arm64) ROS Version: Melodic

Computer details Processor: Jetson TX2 (4 ARMv8 cores) RAM: 8 GB

Describe the bug A clear and concise description of what the bug is.

My camera is capable of 20 FPS. I confirmed this by modifying the Acquisition.cpp example file from the Spinnaker SDK to capture 200 frames (without saving them to the disk) and the timing worked out to precisely 20 FPS.

However, when I run either node_acquisition.launch or acquisition.launch, configured to send to ROS but not save to disk, the speed is about 10.2 FPS, almost half.

rgov commented 3 years ago

Attaching a callgrind trace: callgrind.listener.19483.zip

ghost commented 3 years ago

@rgov one issue i have often seen is when the actual exposure time is longer than required for frame rate. You can quickly test by setting the exposure to something like 2000 us.

Additionally try lowering the required datarate by reducing resolution by setting binning to 2 and the color to grayscale. This will help understand where the latency comes from.

rgov commented 3 years ago

With exposure_time set to 2000 (µs, seems undocumented?) I get a modest increase, to 12.4 FPS. But the Spinnaker Acquisition.cpp demo does not set exposure time and seems to get 20 FPS.

vik748 commented 3 years ago

@rgov this could be due to the overhead associated with ROS's camera transport. 5MP is quite a bit of data for serializing / deserializing. Do you get any improvement with grayscale (1/3 reduction in data) and setting binning to 2 (1/4 reduction in data) ?

vik748 commented 3 years ago

Also, we have seen issues with newer version of spinnaker, can you try with spinnaker 2.2.0.48

vik748 commented 3 years ago

@rgov any updates on your issue?

rgov commented 3 years ago

It is not resolved yet but I have been working on other parts of my system. I'd like to get to the bottom of it.

nevangeliouNYUAD commented 3 years ago

A personal view on this: I think the soft triggering methodology creates some sort of overhead.

The Acquisition.cpp from FLIR achieves on a GigE Blackfly-S 35Hz in color mode on an i7 Intel NUC (Ubuntu 18, Spinnaker SDK 2.0.0.48). I added the image conversion and ROS publishing and rate reduces to 32Hz. If I use this package with same camera settings, best rate I could get was 18Hz. Exposure time@5000us.

For users who want high rates you can also have a look at the FLIR SDK, convert the image with padding and publish to ros with cv_bridge/image_transport. Not all packages can be super-efficient at all configs. This package is definitely a life-saver w.r.t. master triggering though.

With thanks to the authors.

vik748 commented 3 years ago

@nevangeliouNYUAD There is also another existing ROS driver https://github.com/ros-drivers/flir_camera_driver which might work better for some.