l1npengtul / nokhwa

Cross Platform Rust Library for Powerful Webcam/Camera Capture
Apache License 2.0
537 stars 137 forks source link

🐛 v4l: fix perf issue introduce by disabled arena buffer #129

Closed OlivierLDff closed 1 year ago

OlivierLDff commented 1 year ago

fix #128

I used the following python script to benchmark expected camera behavior:

import cv2

# Create a VideoCapture object
cap = cv2.VideoCapture(0)

# Check if camera opened successfully
if cap.isOpened() == False:
    print("Unable to read camera feed")

# Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file.
window_name = "Image"
cv2.namedWindow(window_name, cv2.WINDOW_NORMAL)

fps_count = 0
last_fps_time_updated = 0

while True:
    ret, frame = cap.read()

    # print frame.shape
    print(f"Frame: {frame.shape}")

    if ret == True:
        # Display the resulting frame
        cv2.imshow(window_name, frame)

        fps_count += 1
        current_time = cv2.getTickCount()
        if (current_time - last_fps_time_updated) > cv2.getTickFrequency():
            print("FPS: {}".format(fps_count))
            fps_count = 0
            last_fps_time_updated = current_time

        # Press Q on keyboard to stop recording
        if cv2.waitKey(1) & 0xFF == ord("q"):
            break

    # Break the loop
    else:
        break

And the following rust code:

                let mut camera = Camera::new(
                    index.clone(),
                    RequestedFormat::new::<RgbFormat>(RequestedFormatType::Closest(
                        CameraFormat::new(
                            Resolution {
                                width_x: 320,
                                height_y: 240,
                            },
                            FrameFormat::MJPEG,
                            30,
                        ),
                    )),
                )
                .unwrap();

                println!("format = {:?}", camera.camera_format());
                let mut last_fps_updated = std::time::Instant::now();
                let mut frame_count = 0;
                camera.open_stream().unwrap();
                loop {
                    let frame = camera.frame().unwrap();
                    println!(
                        "Captured frame of size {}, with size {:?}, format {:?}",
                        frame.buffer().len(),
                        frame.resolution(),
                        frame.source_frame_format()
                    );
                    let decoded = frame.decode_image::<RgbFormat>().unwrap();
                    println!(
                        "DecodedFrame of {}, resolution {:?}x{:?}",
                        decoded.len(),
                        decoded.width(),
                        decoded.height()
                    );
                    frame_count += 1;

                    let now = std::time::Instant::now();
                    if now - last_fps_updated > std::time::Duration::from_secs(1) {
                        println!("FPS: {}", frame_count);

                        frame_count = 0;
                        last_fps_updated = now;
                    }
                }

The performance regression can easily be observed in v4l examples too by adding stream.start:

https://github.com/raymanfx/libv4l-rs/blob/867d9b122fde603ff7f1bfcdf19de48845c9e1f7/examples/stream_capture_mmap.rs#L26-L29

OlivierLDff commented 1 year ago

I've correctly rebased on 0.10 to fix conflict introduced by my other PR.

l1npengtul commented 1 year ago

Can you undo the rebase? Apologies, I was fixing the PR in my IDE, 0.10 is dead and is in "unless-this-kills-your-system-no-more-updates" mode

OlivierLDff commented 1 year ago

I'm not sure to understand, this PR was already targetting 0.10 branch, like my other PR. There were a conflict due to the lock introduced, I needed to fix for the merge to be possible. I guess this is quite an important bug fix, having frame rate divided by 2 doesn't kill the system, but make the library unusable on linux. 13 fps instead of 25 for a camera isn't acceptable

OlivierLDff commented 1 year ago

I would gladly cherry pick my multiples commits later and create other PR for the senpai branch, but it doesn't compile at the moment so I can't check if this works.

l1npengtul commented 1 year ago

Ah. Apologies. Yeah, I'll make sure to cherry pick this and #127 for later.

OlivierLDff commented 1 year ago

Yes, I will do as soon as senpai branch will be compiling.