search

Hexxeh / rpi-firmware

Firmware files for the Raspberry Pi
Other
775 stars 207 forks source link

uvcvideo: performance problem #183

Closed VigibotDev closed 5 years ago

VigibotDev commented 5 years ago

I get frames duplicate&drop even with a low resolution like 320x240 (25FPS) ! There is a huge performance problem with USB / uvcvideo on raspberry PI 3+ (or a FFmpeg bug!)

I need to do soon more testing :

I post all my work here.


(root|~) /var/www/processdiffusion -loglevel debug -f v4l2 -framerate 25 -video_size 320x240 -i /dev/video0 -codec:v h264_omx -profile:v baseline -b:v 1400000 -maxrate 1400000 -bufsize 1400000 -f rawvideo tcp://127.0.0.1:8004
ffmpeg version git-2018-08-11-00ae349 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  configuration: --arch=armhf --enable-gpl --enable-nonfree --enable-mmal --enable-omx --enable-omx-rpi
  libavutil      56. 19.100 / 56. 19.100
  libavcodec     58. 22.101 / 58. 22.101
  libavformat    58. 17.101 / 58. 17.101
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 26.100 /  7. 26.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'v4l2'.
Reading option '-framerate' ... matched as AVOption 'framerate' with argument '25'.
Reading option '-video_size' ... matched as AVOption 'video_size' with argument '320x240'.
Reading option '-i' ... matched as input url with argument '/dev/video0'.
Reading option '-codec:v' ... matched as option 'codec' (codec name) with argument 'h264_omx'.
Reading option '-profile:v' ... matched as option 'profile' (set profile) with argument 'baseline'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '1400000'.
Reading option '-maxrate' ... matched as AVOption 'maxrate' with argument '1400000'.
Reading option '-bufsize' ... matched as AVOption 'bufsize' with argument '1400000'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option 'tcp://127.0.0.1:8004' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url /dev/video0.
Applying option f (force format) with argument v4l2.
Successfully parsed a group of options.
Opening an input file: /dev/video0.
[video4linux2,v4l2 @ 0x1b19370] fd:4 capabilities:84200001
[video4linux2,v4l2 @ 0x1b19370] Current input_channel: 0, input_name: Camera 1, input_std: 0
[video4linux2,v4l2 @ 0x1b19370] The V4L2 driver changed the pixel format from 0x32315559 to 0x56595559
[video4linux2,v4l2 @ 0x1b19370] Trying to set codec:rawvideo pix_fmt:yuv420p
[video4linux2,v4l2 @ 0x1b19370] The V4L2 driver changed the pixel format from 0x32315559 to 0x56595559
[video4linux2,v4l2 @ 0x1b19370] Trying to set codec:rawvideo pix_fmt:yuv420p
[video4linux2,v4l2 @ 0x1b19370] The V4L2 driver changed the pixel format from 0x32315659 to 0x56595559
[video4linux2,v4l2 @ 0x1b19370] Trying to set codec:rawvideo pix_fmt:yuv422p
[video4linux2,v4l2 @ 0x1b19370] The V4L2 driver changed the pixel format from 0x50323234 to 0x56595559
[video4linux2,v4l2 @ 0x1b19370] Trying to set codec:rawvideo pix_fmt:yuyv422
[video4linux2,v4l2 @ 0x1b19370] Setting time per frame to 1/25
[video4linux2,v4l2 @ 0x1b19370] All info found
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 259.225212, bitrate: 30720 kb/s
    Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (YUY2 / 0x32595559), yuyv422, 320x240, 0/1, 30720 kb/s, 25 fps, 25 tbr, 1000k tbn, 1000k tbc
Successfully opened the file.
Parsing a group of options: output url tcp://127.0.0.1:8004.
Applying option codec:v (codec name) with argument h264_omx.
Applying option profile:v (set profile) with argument baseline.
Applying option b:v (video bitrate (please use -b:v)) with argument 1400000.
Applying option f (force format) with argument rawvideo.
Successfully parsed a group of options.
Opening an output file: tcp://127.0.0.1:8004.
[tcp @ 0x1b1ce30] No default whitelist set
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
detected 4 logical cores
[graph 0 input from stream 0:0 @ 0x1b26680] Setting 'video_size' to value '320x240'
[graph 0 input from stream 0:0 @ 0x1b26680] Setting 'pix_fmt' to value '1'
[graph 0 input from stream 0:0 @ 0x1b26680] Setting 'time_base' to value '1/1000000'
[graph 0 input from stream 0:0 @ 0x1b26680] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x1b26680] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x1b26680] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 0x1b26680] w:320 h:240 pixfmt:yuyv422 tb:1/1000000 fr:25/1 sar:0/1 sws_param:flags=2
[format @ 0x1b19070] Setting 'pix_fmts' to value 'yuv420p'
[auto_scaler_0 @ 0x1b26e00] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 0x1b26e00] w:iw h:ih flags:'bicubic' interl:0
[format @ 0x1b19070] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 0x1b25e20] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
[auto_scaler_0 @ 0x1b26e00] w:320 h:240 fmt:yuyv422 sar:0/1 -> w:320 h:240 fmt:yuv420p sar:0/1 flags:0x4
[h264_omx @ 0x1b1c2c0] Using OMX.broadcom.video_encode
[h264_omx @ 0x1b1c2c0] OMX state changed to 2
[h264_omx @ 0x1b1c2c0] OMX state changed to 3
[h264_omx @ 0x1b1c2c0] OMX port 201 settings changed
Output #0, rawvideo, to 'tcp://127.0.0.1:8004':
  Metadata:
    encoder         : Lavf58.17.101
    Stream #0:0, 0, 1/25: Video: h264 (h264_omx), 1 reference frame, yuv420p, 320x240, 0/1, q=2-31, 1400 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.22.101 h264_omx
Clipping frame in rate conversion by 0.000008
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
*** 5 dup!
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
    Last message repeated 2 times
Clipping frame in rate conversion by 0.097847
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.099571
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.198418
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.199211
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.299095
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.397789
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.399391
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.498192
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.499138
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Clipping frame in rate conversion by 0.598946
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6400.68 bitrate=  22.8kbits/s dup=5 drop=0 speed=1.36x
Clipping frame in rate conversion by 0.599388
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Past duration 0.699425 too large
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Past duration 0.798286 too large
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Past duration 0.798393 too large
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Past duration 0.898949 too large
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Past duration 0.899117 too large
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
Past duration 0.999199 too large
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
    Last message repeated 5 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6401.20 bitrate=  19.7kbits/s dup=5 drop=0 speed=1.19x
    Last message repeated 13 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6401.72 bitrate=  18.5kbits/s dup=5 drop=0 speed=1.14x
*** dropping frame 45 from stream 0 at ts 43
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
    Last message repeated 11 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6402.24 bitrate=  17.9kbits/s dup=5 drop=1 speed=1.11x
    Last message repeated 4 times
*** dropping frame 61 from stream 0 at ts 59
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
    Last message repeated 8 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6402.76 bitrate=  17.5kbits/s dup=5 drop=2 speed=1.09x
    Last message repeated 7 times
*** dropping frame 77 from stream 0 at ts 75
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
    Last message repeated 4 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6403.24 bitrate=  17.4kbits/s dup=5 drop=3 speed=1.07x
    Last message repeated 11 times
*** dropping frame 93 from stream 0 at ts 91
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
    Last message repeated 1 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6403.76 bitrate=  17.1kbits/s dup=5 drop=4 speed=1.06x
    Last message repeated 12 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6404.28 bitrate=  17.0kbits/s dup=5 drop=4 speed=1.06x
    Last message repeated 1 times
*** dropping frame 109 from stream 0 at ts 107
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 640
    Last message repeated 10 times
[rawvideo @ 0x1b1ab60] PACKET SIZE: 153600, STRIDE: 6404.76 bitrate=  17.0kbits/s dup=5 drop=5 speed=1.05x
    Last message repeated 3 times
[h264_omx @ 0x1b1c2c0] OMX event 4 c9 1
frame=  124 fps= 26 q=-0.0 Lsize=      11kB time=00:00:04.96 bitrate=  17.8kbits/s dup=5 drop=6 speed=1.05x
video:11kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Input file #0 (/dev/video0):
  Input stream #0:0 (video): 125 packets read (19200000 bytes); 125 frames decoded; 
  Total: 125 packets (19200000 bytes) demuxed
Output file #0 (tcp://127.0.0.1:8004):
  Output stream #0:0 (video): 124 frames encoded; 125 packets muxed (11012 bytes); 
  Total: 125 packets (11012 bytes) muxed
125 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x1b24ec0] Statistics: 0 seeks, 125 writeouts
[h264_omx @ 0x1b1c2c0] OMX state changed to 2
[h264_omx @ 0x1b1c2c0] OMX state changed to 1
Exiting normally, received signal 2.
(root|~)
popcornmix commented 5 years ago

This isn't the right place for this. This repo is purely for distributing binary firmware updates.

If you can confirm it is a kernel issue then https://github.com/raspberrypi/linux/issues would be the right place, but it may well be a configuration issue (e.g. ffmpeg doing unnecessary format conversions)

I think the forum may be a better place to start this. You may find someone who has succeeded in doing something similar.

VigibotDev commented 5 years ago

Sorry I found the problem. I must add a "-r 25" option before the "-i /dev/video0" Even if ffmpeg AND v4l2-ctl say 25 FPS ! The -framerate parameter do not work. And it's invisible Ffmpeg say "[graph 0 input from stream 0:0 @ 0x1b26680] Setting 'frame_rate' to value '25/1'" but it's Fake ! No more frame drop or duplicate problem at full resolution...