Open cedricpradalier opened 10 years ago
mikepurvis
commented
10 years ago Thanks for doing this investigation—performance has been one of the many reasons this driver hasn't received the love it needs from us. If gscam works well for the video portion, then this can evolve into just a stub for managing the PTZ, and the rest of it can be put out of its misery.
cedricpradalier
commented
10 years ago Hi,
Tried the gstreamer-based axis acquisition on the kingfisher today. The souphttpsrc is working fine, with very little delay, but it uses an insane amount of CPU, which seems proportional to the file size. With compression at 0, it uses 20% of a CPU, compression at 10, 8%, compression at 30, 3-4%. For comparison, the python axis driver using curl, uses 3-4% at compression 10.
Next, I'll try the neonhttpsrc and we'll see if this gets a bit better.
Regards
On Wed, Nov 27, 2013 at 1:25 AM, Mike Purvis notifications@github.comwrote:
Thanks for doing this investigation—performance has been one of the many reasons this driver hasn't received the love it needs from us. If gscam works well for the video portion, then this can evolve into just a stub for managing the PTZ, and the rest of it can be put out of its misery.
— Reply to this email directly or view it on GitHubhttps://github.com/clearpathrobotics/axis_camera/issues/26#issuecomment-29350360 .
Cedric Pradalier
cedricpradalier
commented
10 years ago Hi,
Tested today on the boat with neonhttpsrc (had to compile it from source in the gstreamer-plugin-bad package). The performance seems has good as the python script in terms of CPU load (4% CPU load at compression 10), and much better in terms of latency. We'll test it on the lake at the next opportunity.
The configuration file is available on https://github.com/cedricpradalier/axis_camera/blob/master/axis_boot.launch(replace neonhttpsrc with souphttpsrc to test with the default http gstreamer source).
HTH
On Mon, Dec 2, 2013 at 11:24 PM, Cedric Pradalier < cedric.pradalier@gmail.com> wrote:
Hi,
Tried the gstreamer-based axis acquisition on the kingfisher today. The souphttpsrc is working fine, with very little delay, but it uses an insane amount of CPU, which seems proportional to the file size. With compression at 0, it uses 20% of a CPU, compression at 10, 8%, compression at 30, 3-4%. For comparison, the python axis driver using curl, uses 3-4% at compression 10.
Next, I'll try the neonhttpsrc and we'll see if this gets a bit better.
Regards
On Wed, Nov 27, 2013 at 1:25 AM, Mike Purvis notifications@github.comwrote:
Thanks for doing this investigation—performance has been one of the many reasons this driver hasn't received the love it needs from us. If gscam works well for the video portion, then this can evolve into just a stub for managing the PTZ, and the rest of it can be put out of its misery.
— Reply to this email directly or view it on GitHubhttps://github.com/clearpathrobotics/axis_camera/issues/26#issuecomment-29350360 .
Cedric Pradalier
Cedric Pradalier
cedricpradalier
commented
10 years ago Hi Mike,
Further tests with optimizing the Axis performances. Now that we have gstreamer taking care of capturing the data and publishing them as jpegs, I wanted to be able to use the image on-board as well. Problem: decompression and republish with the standard image_transport tools takes 30% of a CPU (this is not too bad, gstreamer takes 85%). However, a bit of investigation and I found out that the jpeg arrive in color (YCbCr) and are converted to RGB by the republish plugin. Then the first thing I do with them is convert them to grayscale for image processing. Both convertions are taking a huge time, but they should not be since the Y component of YCbCr is the grayscale value.
The imdecode function of opencv does support grayscale decoding directly (somewhat, it does not use the fact that libjpeg can do direct grayscale decoding if I remember correctly, but does YCbCr2Gray afterward), but this is not yet exported in the image_transport plugin. I've raised an issue ( https://github.com/ros-perception/image_transport_plugins/issues/8) but I've also added a simple node which does that to my axis_camera fork ( https://github.com/cedricpradalier/axis_camera/blob/master/src/republish.cpp). This is unlikely to get merged because it brings in dependencies on opencv2/cv_bridge...
On my laptop this save 60% cpu for the decoding. I'll try to check tomorrow how it behaves on the boat.
Just for information...
On Tue, Dec 3, 2013 at 5:52 PM, Cedric Pradalier <cedric.pradalier@gmail.com
wrote:
Hi,
Tested today on the boat with neonhttpsrc (had to compile it from source in the gstreamer-plugin-bad package). The performance seems has good as the python script in terms of CPU load (4% CPU load at compression 10), and much better in terms of latency. We'll test it on the lake at the next opportunity.
The configuration file is available on https://github.com/cedricpradalier/axis_camera/blob/master/axis_boot.launch(replace neonhttpsrc with souphttpsrc to test with the default http gstreamer source).
HTH
On Mon, Dec 2, 2013 at 11:24 PM, Cedric Pradalier < cedric.pradalier@gmail.com> wrote:
Hi,
Tried the gstreamer-based axis acquisition on the kingfisher today. The souphttpsrc is working fine, with very little delay, but it uses an insane amount of CPU, which seems proportional to the file size. With compression at 0, it uses 20% of a CPU, compression at 10, 8%, compression at 30, 3-4%. For comparison, the python axis driver using curl, uses 3-4% at compression 10.
Next, I'll try the neonhttpsrc and we'll see if this gets a bit better.
Regards
On Wed, Nov 27, 2013 at 1:25 AM, Mike Purvis notifications@github.comwrote:
Thanks for doing this investigation—performance has been one of the many reasons this driver hasn't received the love it needs from us. If gscam works well for the video portion, then this can evolve into just a stub for managing the PTZ, and the rest of it can be put out of its misery.
— Reply to this email directly or view it on GitHubhttps://github.com/clearpathrobotics/axis_camera/issues/26#issuecomment-29350360 .
Cedric Pradalier
Cedric Pradalier
Cedric Pradalier
cedricpradalier
commented
10 years ago Hi Mike,
Just for the record, I've modified the command message (and handling) in my version of the axis driver, to add a field specifying what part of the command is relevant. This allows controlling the pan,tilt without touching the exposure or zoom. This also allows implementing exposure control, independently of the pan-tilt control. Good for tracking objects on the shore while keeping a good exposure balance.
We've tested the gray-level republish on the boat today and it cuts the decoding cost by 50%, from 25% to 13% CPU load.
I hope that helps.
On Sun, Dec 8, 2013 at 11:22 PM, Cedric Pradalier < cedric.pradalier@gmail.com> wrote:
Hi Mike,
Further tests with optimizing the Axis performances. Now that we have gstreamer taking care of capturing the data and publishing them as jpegs, I wanted to be able to use the image on-board as well. Problem: decompression and republish with the standard image_transport tools takes 30% of a CPU (this is not too bad, gstreamer takes 85%). However, a bit of investigation and I found out that the jpeg arrive in color (YCbCr) and are converted to RGB by the republish plugin. Then the first thing I do with them is convert them to grayscale for image processing. Both convertions are taking a huge time, but they should not be since the Y component of YCbCr is the grayscale value.
The imdecode function of opencv does support grayscale decoding directly (somewhat, it does not use the fact that libjpeg can do direct grayscale decoding if I remember correctly, but does YCbCr2Gray afterward), but this is not yet exported in the image_transport plugin. I've raised an issue ( https://github.com/ros-perception/image_transport_plugins/issues/8) but I've also added a simple node which does that to my axis_camera fork ( https://github.com/cedricpradalier/axis_camera/blob/master/src/republish.cpp). This is unlikely to get merged because it brings in dependencies on opencv2/cv_bridge...
On my laptop this save 60% cpu for the decoding. I'll try to check tomorrow how it behaves on the boat.
Just for information...
On Tue, Dec 3, 2013 at 5:52 PM, Cedric Pradalier < cedric.pradalier@gmail.com> wrote:
Hi,
Tested today on the boat with neonhttpsrc (had to compile it from source in the gstreamer-plugin-bad package). The performance seems has good as the python script in terms of CPU load (4% CPU load at compression 10), and much better in terms of latency. We'll test it on the lake at the next opportunity.
The configuration file is available on https://github.com/cedricpradalier/axis_camera/blob/master/axis_boot.launch(replace neonhttpsrc with souphttpsrc to test with the default http gstreamer source).
HTH
On Mon, Dec 2, 2013 at 11:24 PM, Cedric Pradalier < cedric.pradalier@gmail.com> wrote:
Hi,
Tried the gstreamer-based axis acquisition on the kingfisher today. The souphttpsrc is working fine, with very little delay, but it uses an insane amount of CPU, which seems proportional to the file size. With compression at 0, it uses 20% of a CPU, compression at 10, 8%, compression at 30, 3-4%. For comparison, the python axis driver using curl, uses 3-4% at compression 10.
Next, I'll try the neonhttpsrc and we'll see if this gets a bit better.
Regards
On Wed, Nov 27, 2013 at 1:25 AM, Mike Purvis notifications@github.comwrote:
Thanks for doing this investigation—performance has been one of the many reasons this driver hasn't received the love it needs from us. If gscam works well for the video portion, then this can evolve into just a stub for managing the PTZ, and the rest of it can be put out of its misery.
— Reply to this email directly or view it on GitHubhttps://github.com/clearpathrobotics/axis_camera/issues/26#issuecomment-29350360 .
Cedric Pradalier
Cedric Pradalier
Cedric Pradalier
Cedric Pradalier
mikepurvis
commented
9 years ago Looks like the next step here is to just implement a minimal axis-specific C++ framegrabber which is able to be used as both a node and nodelet. I think the ideal would be if axis_camera's driver could advertise separate topics for raw, compressed, and greyscale, and (like other drivers) only produce them when a subscription exists:
compressed would pass through the frames directly from the camera's http feed.raw would use libjpeg to decode the frames and pass them through as bitmaps.greyscale would do similar, but use the jpeg_transform_info::force_grayscale option to decode only the Y channel.
Just for information, because python is sometimes introducing huge delay (like 4-5 seconds!), I was investigating another option to get the camera video stream.
Using gstreamer and gscam seems to work but uses a lot of CPU if it does the decoding itself. The patch below to gscam allows publishing the jpeg stream, same as the axis_driver. https://github.com/ros-drivers/gscam/pull/11
This looks like a good alternative on the kingfisher, although I haven't found yet how to tell gstreamer to use additional parameter in the get request (like resolution, fps, image size).