typical processing usage

[mbudris]

This is on i3-8100T(4core@3.1GHz), 16GB, M2 sata machine

This is what I get when using the package. Is this typical? What do you think are minimal and recommended hardware requirements?

[jveitchmichaelis]

Hi,

Yes this is pretty typical of stereo image processing. The image matching algorithms are provided by OpenCV (via the ROS stereo_image_proc package) which are probably multi-threaded.

It really depends on what sort of performance you're after. What FPS do you need?

Thanks!

On Mon, May 13, 2019, 09:13 mbudris notifications@github.com wrote:

[image: image] https://user-images.githubusercontent.com/8166666/57605563-a0235300-756f-11e9-8694-828ae87b64cd.png This is on i3-8100T(4core@3.1GHz), 16GB, M2 sata machine

This is what I get when using the package. Is this typical? What do you think are minimal and recommended hardware requirements?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/i3drobotics/deimos-ros/issues/11, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYDMJ5PXD6L4CVBXIQM5HLPVEPKPANCNFSM4HMNMCFQ .

[mbudris]

10 would be nice, 1 I could use, 0.1 is what I get now.

[jveitchmichaelis]

For example, on a mid-range i5 quad you should easily manage 30fps with the block matcher at VGA resolution.

Also check what parameters you're using for matching, it's possible you just have an enormous window size which is slowing you down. You can do this with dynamic reconfigure.

http://wiki.ros.org/dynamic_reconfigure

On Mon, May 13, 2019, 12:52 mbudris notifications@github.com wrote:

10 would be nice, 1 I could use, 0.1 is what I get now.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/i3drobotics/deimos-ros/issues/11?email_source=notifications&email_token=AAYDMJ53IOONTR4TA5PXUO3PVFI6TA5CNFSM4HMNMCF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVICHAI#issuecomment-491791233, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYDMJZRPARKGSJ4LDO23STPVFI6TANCNFSM4HMNMCFQ .

[mbudris]

This is on i3-8100T(4core@3.1GHz), 16GB, M2 sata disk machine.

Since I just started to use the package I don't know much about it so I'm using default deimos settings. Tried to use dynamic reconfigure, but can't go above 2 FPS.

On Mon, 13 May 2019 at 17:30, Josh Veitch-Michaelis < notifications@github.com> wrote:

What spec is your machine?

For example, on a mid-range i5 quad you should easily manage 30fps with the block matcher at VGA resolution.

Also check what parameters you're using for matching, it's possible you just have an enormous window size which is slowing you down. You can do this with dynamic reconfigure.

http://wiki.ros.org/dynamic_reconfigure

On Mon, May 13, 2019, 12:52 mbudris notifications@github.com wrote:

10 would be nice, 1 I could use, 0.1 is what I get now.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/i3drobotics/deimos-ros/issues/11?email_source=notifications&email_token=AAYDMJ53IOONTR4TA5PXUO3PVFI6TA5CNFSM4HMNMCF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVICHAI#issuecomment-491791233 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAYDMJZRPARKGSJ4LDO23STPVFI6TANCNFSM4HMNMCFQ

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/i3drobotics/deimos-ros/issues/11?email_source=notifications&email_token=AB6J2CS7DVEWVBGQHULBPY3PVF3OZA5CNFSM4HMNMCF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVIPXHQ#issuecomment-491846558, or mute the thread https://github.com/notifications/unsubscribe-auth/AB6J2CQHNTZRATHAX4EMRPLPVF3OZANCNFSM4HMNMCFQ .

[mbudris]

Is it normal to have bandwidth of 80-100 MB/s for /stereo/points2?

On Tue, 14 May 2019 at 08:40, Mantas Budraitis mantas.budraitis@gmail.com wrote:

This is on i3-8100T(4core@3.1GHz), 16GB, M2 sata disk machine.

Since I just started to use the package I don't know much about it so I'm using default deimos settings. Tried to use dynamic reconfigure, but can't go above 2 FPS.

On Mon, 13 May 2019 at 17:30, Josh Veitch-Michaelis < notifications@github.com> wrote:

What spec is your machine?

For example, on a mid-range i5 quad you should easily manage 30fps with the block matcher at VGA resolution.

Also check what parameters you're using for matching, it's possible you just have an enormous window size which is slowing you down. You can do this with dynamic reconfigure.

http://wiki.ros.org/dynamic_reconfigure

On Mon, May 13, 2019, 12:52 mbudris notifications@github.com wrote:

10 would be nice, 1 I could use, 0.1 is what I get now.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/i3drobotics/deimos-ros/issues/11?email_source=notifications&email_token=AAYDMJ53IOONTR4TA5PXUO3PVFI6TA5CNFSM4HMNMCF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVICHAI#issuecomment-491791233 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAYDMJZRPARKGSJ4LDO23STPVFI6TANCNFSM4HMNMCFQ

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/i3drobotics/deimos-ros/issues/11?email_source=notifications&email_token=AB6J2CS7DVEWVBGQHULBPY3PVF3OZA5CNFSM4HMNMCF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVIPXHQ#issuecomment-491846558, or mute the thread https://github.com/notifications/unsubscribe-auth/AB6J2CQHNTZRATHAX4EMRPLPVF3OZANCNFSM4HMNMCFQ .

[mbudris]

The camera alone uses quite a lot of CPU:

[jveitchmichaelis]

Hi,

This is possibly an issue with ROS or OpenCV, not the camera itself. Are you just acquiring images (no 3D)?

Can you check with eg http://wiki.ros.org/cv_camera and see if you get similar cpu usage? And ideally also check in a loop with OpenCV alone (no ROS overhead).

If it's similar then that would suggest ROS or OpenCV is the culprit. Otherwise we can have a look at where this bottleneck is happening.

Sorry for not replying to your previous comment - it depends how the point cloud is being sent. For a dense 3D image, say 300k pixels each having 3xfloat coordinates plus a colour is 3-5MB (if stored in binary). So it's not unreasonable to see 100MB/s in normal use. However you're reporting extremely slow framerates so I'm not sure.

I've seen this in situations where the PC is underpowered (typically with higher res systems) and ROS will start dropping messages. But it's quite difficult to diagnose.

Do you have a Windows machine available? You could also try our desktop software https://github.com/i3drobotics/stereo-vision-toolkit to see if you still use full cpu when acquiring.

Thanks

[mbudris]

cv_camera works nicely, unless I subscribe to theora, then CPU bumps up but still not by that much.

[jveitchmichaelis]

Might well be the capture code then. We didn't write the original package - it uses some ROS UVC wrapper (with some additional code to talk to the camera's extension unit). I thought it was OpenCV tbh, but apparently not.

I'll add an issue to look into switching to OpenCV for acquisition. Probably makes more sense.

[mbudris]

Yes it looks like it spends a lot of time in https://github.com/i3drobotics/i3dr_deimos-ros/blob/master/src/uvc_cam.cpp#L524, which is custom to UVC (maybe Y16 conversion?).

[jveitchmichaelis]

Edit: Ok I think the reason it's doing Y16 is because the left/right images stored by the camera are interleaved as different channels. You can also return images as BGR which is what we do with OpenCV.

There are some other inefficiencies in that code - for example it memsets the buffer to zero every time, which is redundant since the buffer gets overwritten by the frame anyway. Maybe that's required by V4L2, but it seems odd that it's necessary. It's also in the doc examples though.

The loop is also a bit slow - e.g. why is there a conditional in the middle? So you might as well put:

for (unsigned int i = 0; i < width_ * height_ * 2; i +=2){ 
        concat_frame_ [ (i/(2*width_)) * width_ + (i / 2) ] = py16 [i];
        left_frame_ [i / 2 ] = py16 [i];

        int j = i+1;
        concat_frame_ [ (j / (2*width_)) * width_ + ( j / 2) + width_ ] = py16 [j];
        right_frame_ [j / 2 ] = py16 [j];
}

I don't have a camera tot test right now, but you can try with that simple tweak and see if it helps. I wouldn't put it past a good compiler to have already optimised it though.

[mbudris]

No benefit changing the for loop.

How do I switch to cv_camera? I see this now, and I have no clue how to split it to left and right.

[jveitchmichaelis]

Ok, it was worth a try. We'd probably need to actually profile the code to figure out where it's choking.

I don't think there's a ROS package that will do this for you, hence this package. You need to split the images manually, e.g. write a node that captures the images and then publishes the separate channels. I would suggest doing it this way over writing a separate node that subscribes to the single image because you're just wasting bandwidth pushing data around.

It's quite simple in OpenCV (especially in Python), something like

import rospy
from sensor_msgs.msg import Image
from cv_bridge import CvBridge
import cv2

class StereoPublisher(object):
    def __init__(self, devid=0):
        self.br = CvBridge()
        self.pub_left = rospy.Publisher('left', Image)
        self.pub_right = rospy.Publisher('right', Image)
        self.devid = devid

    def start(self):
        cap = cv2.VideoCapture(self.devid)
        while not rospy.is_shutdown():
            res, im = cap.read()

            if res:
                self.pub_left.publish(self.br.cv2_to_imgmsg(im[0,:,:]))
                self.pub_right.publish(self.br.cv2_to_imgmsg(im[1,:,:]))

if __name__ == '__main__':
    rospy.init_node("stereo_publisher", anonymous=True)
    node = StereoPublisher()
    node.start()

(off the top of my head, so that code may not work, but it'll be close ;))

When I have some time I'll try and update this package to use OpenCV properly, so you still have access to the IMU and exposure times etc.