Low FPS with high resolution and MJPEG

[adoyle-h]

I run the ustreamer in docker container.

The /dev/video1 is my HDMI capture device. It supports MJPG with 1920x1080, 60fps. But client's fps is only 10~16. While it uses 640x480, the client's fps will be 60. The higher resolution, the lower fps.

I tried -c CPU, -c HW, -c M2M-VIDEO, -c NOOP, but it have no effect.

$ v4l2-ctl --list-formats-ext -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                        Interval: Discrete 0.040s (25.000 fps)
                        Interval: Discrete 0.050s (20.000 fps)
                        Interval: Discrete 0.100s (10.000 fps)

$ sudo docker run --rm -it --device /dev/video1:/dev/video0 -p 9999:8080 pikvm/ustreamer:v5.43 -r 1920x1080 -f 60 -m MJPEG --encoder M2M-VIDEO
-- INFO  [26900.891      main] -- Starting PiKVM uStreamer 5.43 ...
-- INFO  [26900.891      main] -- Using internal blank placeholder
-- INFO  [26900.892      main] -- Listening HTTP on [0.0.0.0]:8080
-- INFO  [26900.892    stream] -- Using V4L2 device: /dev/video0
-- INFO  [26900.893    stream] -- Using desired FPS: 60
-- INFO  [26900.893      http] -- Starting HTTP eventloop ...
================================================================================
-- INFO  [26901.016    stream] -- Device fd=8 opened
-- INFO  [26901.017    stream] -- Using input channel: 0
-- INFO  [26901.018    stream] -- Using resolution: 1920x1080
-- INFO  [26901.018    stream] -- Using format: MJPEG
-- INFO  [26901.019    stream] -- Using HW FPS: 60
-- ERROR [26901.019    stream] -- Device doesn't support setting of HW encoding quality parameters
-- INFO  [26901.019    stream] -- Using IO method: MMAP
-- INFO  [26901.034    stream] -- Requested 5 device buffers, got 5
-- INFO  [26901.077    stream] -- Capturing started
-- INFO  [26901.077    stream] -- Switching to HW encoder: the input is (M)JPEG ...
-- INFO  [26901.077    stream] -- Using JPEG quality: encoder default
-- INFO  [26901.077    stream] -- Creating pool JPEG with 1 workers ...
-- INFO  [26901.078    stream] -- Capturing ...
-- INFO  [26905.496      http] -- HTTP: NEW client (now=1): [192.168.1.198]:54960, id=7602da8d8a6ec6a2
^C-- INFO  [26971.391      main] -- ===== Stopping by SIGINT =====
-- INFO  [26971.392      http] -- HTTP eventloop stopped
-- INFO  [26971.426    stream] -- Destroying workers pool JPEG ...
-- INFO  [26971.441    stream] -- Capturing stopped
-- INFO  [26971.455    stream] -- Device fd=8 closed
-- INFO  [26971.457      main] -- Bye-bye

The /state:

{
    "ok": true,
    "result": {
        "instance_id": "",
        "encoder": {
            "type": "HW",
            "quality": 0
        },
        "source": {
            "resolution": {
                "width": 1920,
                "height": 1080
            },
            "online": true,
            "desired_fps": 60,
            "captured_fps": 30
        },
        "stream": {
            "queued_fps": 30,
            "clients": 1,
            "clients_stat": {
                "7602da8d8a6ec6a2": {
                    "fps": 13,
                    "extra_headers": false,
                    "advance_headers": false,
                    "dual_final_frames": false,
                    "zero_data": false,
                    "key": "0"
                }
            }
        }
    }
}

I have tested my HDMI capture device with OBS. It looks good.

When ustreamer uses YUYV format, it shows HW FPS: 60 -> 5 (coerced). Why?

$ sudo docker run --rm -it --device /dev/video1:/dev/video0 -p 9999:8080 pikvm/ustreamer:v5.43 -r 1920x1080 -f 60
-- INFO  [27994.001      main] -- Starting PiKVM uStreamer 5.43 ...
-- INFO  [27994.001      main] -- Using internal blank placeholder
-- INFO  [27994.002      main] -- Listening HTTP on [0.0.0.0]:8080
-- INFO  [27994.003    stream] -- Using V4L2 device: /dev/video0
-- INFO  [27994.003    stream] -- Using desired FPS: 60
================================================================================
-- INFO  [27994.004      http] -- Starting HTTP eventloop ...
-- INFO  [27994.120    stream] -- Device fd=8 opened
-- INFO  [27994.121    stream] -- Using input channel: 0
-- INFO  [27994.122    stream] -- Using resolution: 1920x1080
-- INFO  [27994.122    stream] -- Using format: YUYV
-- INFO  [27994.124    stream] -- Using HW FPS: 60 -> 5 (coerced)
-- INFO  [27994.124    stream] -- Using IO method: MMAP
-- INFO  [27994.138    stream] -- Requested 5 device buffers, got 5
-- INFO  [27994.175    stream] -- Capturing started
-- INFO  [27994.176    stream] -- Using JPEG quality: 80%
-- INFO  [27994.176    stream] -- Creating pool JPEG with 4 workers ...
-- INFO  [27994.178    stream] -- Capturing ...

[mdevaev]

Hello.

The higher resolution, the lower fps.

Because MJPEG traffic is extremely heavy and does not fit into the network. Or your client is not able to process such a stream from the network. In any case, this is not a problem on the ustreamer side.

When ustreamer uses YUYV format, it shows HW FPS: 60 -> 5 (coerced). Why?

The driver does this. You are probably running into the bandwidth of the USB bus.

[adoyle-h]

Thanks for your reply.