Increasing Frame Rate/Transfer Speed on RP4

[LCDTengr498]

I'd like to simultaneously run two a2A3840-45ucBAS USB3.0 cameras on my RPi4 with 8GB RAM (Using Raspbian 10). I've already used setup-usb.sh and have PylonViewer installed, but I'd like to use Pypylon for simultaneous video capture. For my project, only one of the cameras needs to be operating at the full 8.3 MP resolution, but the other can be smaller. The frame rate is important as i'm incorporating both cameras for real-time object detection and tracking for software controlling a servo and stepper motor that's also running on the Pi. I can set the actual acquisition rate without issue, but what is actually seen in the video stream is much less.

I've only been using one camera so far and can only get a frame rate of at most 10 fps, but somewhere between 25-30 fps is needed on both cameras. Is this possible? What could I do to fix this?

Here's what I've been using for video capture of just one camera. Just running this alone results in a CPU usage of about 50% for my Pi:

from pypylon import pylon
import cv2

# Conecting to the first available camera and Initializing
camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
camera.Open()

#Initializing Camera Resolution Format
camera.Width.SetValue(3840)
camera.Height.SetValue(2160)

# Grabing Continusely (video) with minimal delay
camera.StartGrabbing(pylon.GrabStrategy_LatestImageOnly)

# Enabling Control of Frame Rate and Changing it
camera.AcquisitionFrameRateEnable.SetValue(True);
camera.AcquisitionFrameRate.SetValue(30.0)

# Converter needed to view Images
converter = pylon.ImageFormatConverter()

# Converting to opencv bgr format
converter.OutputPixelFormat = pylon.PixelType_BGR8packed
converter.OutputBitAlignment = pylon.OutputBitAlignment_MsbAligned
h = camera.PixelFormat.GetValue()
print(h)

while camera.IsGrabbing():
    grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)

    if grabResult.GrabSucceeded():
        # Access the image data
        image = converter.Convert(grabResult)
        img = image.GetArray()
        cv2.namedWindow('Video Feed', cv2.WINDOW_NORMAL)
        cv2.imshow('Video Feed', img)
        #Frame Rate
        d = camera.ResultingFrameRate.GetValue()
        print(d)
        #Press ESC Key to Stop Video Feed
        k = cv2.waitKey(1)
        if k == 27:
            break
    grabResult.Release()

# Releasing the resource and closing windows    
camera.StopGrabbing()
cv2.destroyAllWindows()

On another note, is there any way to rotate a video stream by 90 degrees (portrait mode instead of landscape) with pypylon when my camera is on its side so that the image is upright?

[thiesmoeller]

The RPi4 has two USB3 ports, but they share the same host controller. Thus you have to split the required bandwidth between the two cameras. It is important to inform the cameras about the split. This is where the feature DeviceLinkThroughputLimit is used: https://docs.baslerweb.com/device-information-parameters#devicelinkthroughputlimit

cam.DeviceLinkThroughputLimitMode ="On"
cam.DeviceLinkThroughputLimit = <bytes per second you allocate to your camera>

If you already used setup-usb.sh, the usbfs memory has been increased which is good. You still have to increase the TransferSize, as this will massively reduce the system overhead to capture video data. See also: https://docs.baslerweb.com/stream-grabber-parameters#maximum-transfer-size

This will set the max size for one DMA ( URB ) transfer to 4MB cam.StreamGrabber.MaxTransferSize = 4 * 1024 * 1024

[LCDTengr498]

Thanks for the reply.

Is there some initialization that needs to be done before using StreamGrabber? I tried to use camera.StreamGrabber.MaxTransferSize = 4194304 and I keep getting the following error:

genicam.AccessException: Node is not writable. : AccessException thrown in node 'MaxTransferSize' while calling 'MaxTransferSize.SetValue()' (file 'IntegerT.h', line 77)

[thiesmoeller]

Call it after open and before startgrabbing

[LCDTengr498]

The stream works now, but there does not seem to be any noticeable increase in frame rate and the video stream is still choppy.

Comparatively, I ran a stream using a lower (1920 x 1080) resolution at 10 fps, and it is much smoother and faster, so I can see that my 3840 x 2160 video is not actually running at 10 fps.

Is there anything I can still do to mitigate this?

[thiesmoeller]

Would recommend to first focus on required bandwidth of both cameras. And disable the display and conversion.

So first get the correct configuration to get your two cameras over the shared usb3 connection.

You can use the pylon viewer / bandwidth manager to have a tool to assist in optimizing this

[LCDTengr498]

I currently only have access to one of my two cameras, but will have access to the other one within the next couple of days.

With just one a2A3840-45ucBAS USB3.0 camera, How may I calculate the theoretical limit on stream FPS at full resolution on the RPi 4?

Thanks.

[zoldaten]

camera.ExposureTime = 2000

but you have to add external light. a lot of light )

[SMA2016a]

@LCDTengr498 if you set the pixelformat to bayer8 then you have 1byte per pixel. so you need 8.3MP/frame