search

OpenIPC / majestic

Majestic Community edition integration kit
MIT License
33 stars 5 forks source link

Игнорируются настройки FPS #21

Closed vukitoso closed 2 years ago

vukitoso commented 2 years ago

Здравствуйте. 3516ev200 majestic master+6fbbadd, built on 2021-10-08

Игнорируются настройки FPS. Пример, поток 0 - включён, фпс 15 поток 1 - выключен. Поток 0 выдаёт максимальный фпс в 25

widgetii commented 2 years ago

Fixed in master+b048991

vukitoso commented 2 years ago

3516ev200 imx307 master+b924f45, 2021-10-17

FPS ещё игнорируются

мой конфиг - majestic.yaml.txt

$ ffplay -v verbose -f mjpeg -i http://10.1.1.102/mjpeg
ffplay version 4.1.6-1~deb10u1 Copyright (c) 2003-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Initialized opengl renderer.
[tcp @ 0x733ce0003600] Starting connection attempt to 10.1.1.102 port 80
[tcp @ 0x733ce0003600] Successfully connected to 10.1.1.102 port 80 
Input #0, mjpeg, from 'http://10.1.1.102/mjpeg':B sq=    0B f=0/0   
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: mjpeg, 1 reference frame, yuvj420p(pc, bt470bg/unknown/unknown, center), 640x360 [SAR 1:1 DAR 16:9], 25 tbr, 1200k tbn, 25 tbc
[ffplay_buffer @ 0x733cd80642c0] w:640 h:360 pixfmt:yuvj420p tb:1/1200000 fr:25/1 sar:1/1 sws_param:
[auto_scaler_0 @ 0x733cd8066880] w:iw h:ih flags:'bicubic' interl:0
[ffplay_buffersink @ 0x733cd8065740] auto-inserting filter 'auto_scaler_0' between the filter 'ffplay_buffer' and the filter 'ffplay_buffersink'
[swscaler @ 0x733cd8067d40] deprecated pixel format used, make sure you did set range correctly
[auto_scaler_0 @ 0x733cd8066880] w:640 h:360 fmt:yuvj420p sar:1/1 -> w:640 h:360 fmt:yuv420p sar:1/1 flags:0x4
Created 640x360 texture with SDL_PIXELFORMAT_IYUV.
$ ffplay -rtsp_transport tcp rtsp://10.1.1.102:554/stream=1
ffplay version 4.1.6-1~deb10u1 Copyright (c) 2003-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, rtsp, from 'rtsp://10.1.1.102:554/stream=1':   0B f=0/0   
  Metadata:
    title           : RTSP Session
  Duration: N/A, start: 0.080067, bitrate: N/A
    Stream #0:0: Video: h264 (Baseline), yuvj420p(pc, bt709, progressive), 640x360, 25 tbr, 90k tbn, 180k tbc
[swscaler @ 0x7495a4050ac0] deprecated pixel format used, make sure you did set range correctly
j0rd commented 2 years ago

fps ignored for me as well. OpenIPC v2.1.10.17 h3516EV300+imx335 majestic master+b924f45, 2021-10-17

vukitoso commented 2 years ago

на текущей прошивке можно ограничивать максимальный FPS у сенсора вручную в ini файле в параметре Isp_FrameRate Но в конфиге маджестика уменьшение всё равно игнорится.

widgetii commented 2 years ago

Какую версию Majestic вы сейчас используете?

vukitoso commented 2 years ago

master+a92988c, 2021-10-24

widgetii commented 2 years ago

У меня проблема с вашим конфигом не воспроизводится:

# cat /proc/umap/vpss
-------------------------------VPSS CHN ATTR---------------------------------------------------------
   GrpID  PhyChnID  Enable    Mode   Width  Height  SrcFRate  DstFRate   Depth   Align  MirrorEn  FlipEn  bBufferShare  ProcMode
       0         1       Y    USER    1920    1080        25        15       0       0         N       N             N     VIDEO
       0         2       Y    USER     640     360        25        15       0       0         N       N             N     VIDEO
$ ffmpeg -rtsp_transport tcp -i rtsp://10.216.128.97:554/stream=1 -vcodec copy test.mp4 
Input #0, rtsp, from 'rtsp://10.216.128.97:554/stream=1':
  Metadata:
    title           : RTSP Session
  Duration: N/A, start: 0.079989, bitrate: N/A
  Stream #0:0: Video: h264 (Baseline), yuvj420p(pc, bt709, progressive), 640x360, 25 tbr, 90k tbn, 180k tbc
Output #0, mp4, to 'test.mp4':
  Metadata:
    title           : RTSP Session
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 640x360, q=2-31, 25 tbr, 90k tbn, 90k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[mp4 @ 0x7fd3c6001000] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 @ 0x7fd3c6001000] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
frame=  589 fps= 16 q=-1.0 Lsize=     908kB time=00:00:39.12 bitrate= 190.2kbits/s speed=1.05x    
video:901kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.771071%
vukitoso commented 2 years ago

Да, теперь FPS текущего потока соответствует настройкам в majestic.yaml, хотя везде указывается что 25 fps. Например как у вас "25 tbr". Спасибо.

j0rd commented 2 years ago

OpenIPC version: v2.1.10.31 Majestic version: master+a0018fe, 2021-10-31

The device is currently capturing the proper amount of frames from camera (I notice the stream is 5 & 10 FPS as I've configured)

The problem is (for file size) is that the video encoder it still set to 30 FPS. The video output should be same amount of frames as the amount of frames we're capturing from camera.

So this problem is now 1/2 fixed. Previously, we'd also capture 30 fps, not we capture proper amount of frames, but video encoder output is still 30 fps.

I reset my camera as well with

set_allocator cma
firstboot

Then reset the config with FPS settings, and video still encoding at 30 FPS, while it's only capturing my 5 & 10 FPS via the camera.

j0rd commented 2 years ago

majestic output

10:41:36 INFO  [main] main@124: Majestic Free for HiSilicon, version master+a0018fe, built on 2021-10-31
10:41:36 INFO  [app_config] load_config@122: Using /etc/majestic.yaml as main configuration
10:41:36 DEBUG [sdk] sdk_specific_config@3431: Autodetected sensor as 'imx335_i2c'
10:41:36 DEBUG [sdk] find_sensor_config@3386: matched sensor config: imx335_i2c_4M.ini
10:41:36 INFO  [sdk] find_sensor_config@3405: Using /etc/sensors/imx335_i2c_4M.ini as sensor configuration
10:41:36 DEBUG [watchdog] watchdog_start@71: Detected Hisilicon Watchdog
10:41:36 DEBUG [watchdog] watchdog_start@85: Watchdog timeout set to 10 seconds
10:41:36 TRACE [crypto] setup_ssl@70: Using mbed TLS 2.25.0 with libevent 2.2.0-alpha-dev
10:41:36 DEBUG [sdk] start_sdk@181: App was built with MPP version: Hi3516EV200_MPP_V1.0.1.2.B030 Release
10:41:36 DEBUG [sdk] start_sdk@184: Current MPP version: HI_VERSION=Hi3516EV200_MPP_V1.0.1.2 B030 Release
10:41:36 TRACE [sdk] start_sdk@194: chipid 3516E300
10:41:36 DEBUG [sdk] start_sdk@203: sensor_config.sensor_type stSnsImx335Obj
10:41:36 DEBUG [sdk] start_sdk@205: sensor_config.vichn.pix_format 26
10:41:36 DEBUG [sdk] start_sdk@207: sensor_config.input_mode 0
10:41:36 DEBUG [sdk] start_sdk@208: sensor_config.WDR 0
10:41:36 DEBUG [sdk] start_sdk@213: sensor_config.isp [2592x1520]@0x0 25fps
10:41:36 DEBUG [sdk] start_sdk@214: sensor_config.isp_bayer 0
10:41:36 DEBUG [sensor] tryLoadLibrary@23: try to load: /usr/lib/sensors/libsns_imx335.so
10:41:36 TRACE [hal] hisi_free_mem_ex@694: Free MMZ mem before allocation: 98272KB
10:41:36 DEBUG [sdk] init_mem@2400: u32AlignWidth: 8
10:41:36 TRACE [sdk] dump_vb_configuration@2353: VB configuration:
10:41:36 TRACE [sdk] dump_vb_configuration@2363:   [0]: 5909760 x 5
10:41:36 TRACE [sdk] dump_vb_configuration@2363:   [1]: 7879680 x 1
10:41:36 TRACE [sdk] dump_vb_configuration@2363:   [2]: 608256 x 1
10:41:36 TRACE [hal] hisi_free_mem_ex@694: Free MMZ mem after allocation: 61108KB
10:41:36 DEBUG [sdk] init_sensor@2562: Sensor driver has been loaded
10:41:36 TRACE [sdk] init_vi@2991: SDK is in 'VI_OFFLINE_VPSS_OFFLINE' mode
10:41:36 INFO  [puts] @0: linear mode
10:41:36 INFO  [puts] @0: binit false IMX335 i2c init
10:41:36 INFO  [puts] @0: -------Sony IMX335_init_5M_2592x1944_12bit_linear30 Initial OK!-------
10:41:36 TRACE [sdk] init_vpss@3049: HI_MPI_SYS_Bind: VIU(0)->VPSS(grp 0)
10:41:36 DEBUG [sdk] log_venc_chn@1495: H.264 2592x1520 10fps 4096Kbit  10 GOP
10:41:36 DEBUG [sdk] create_vpss_chn@1257:   new venc: 0   vpss_grp: 0,   vpss_chn: 0
10:41:36 TRACE [sdk] create_vpss_chn@1341:   create_vpss_chn  VPSS(0)->VENC(0)
10:41:36 TRACE [hal] show_blocks@624:   allocated Str x 1=2896, Rcn x 2=11752, PInfo x 2=496, Node x 1=44
10:41:36 TRACE [sdk] start_video@2062:   venc took 15188KB of MMZ [2885]
10:41:36 DEBUG [sdk] log_venc_chn@1495: H.264 704x576 5fps 1024Kbit  5 GOP
10:41:36 DEBUG [sdk] create_vpss_chn@1257:   new venc: 1   vpss_grp: 0,   vpss_chn: 1
10:41:36 TRACE [sdk] create_vpss_chn@1341:   create_vpss_chn  VPSS(1)->VENC(1)
10:41:36 TRACE [hal] show_blocks@624:   allocated Node x 1=16, Str x 1=308, Rcn x 2=1200, PInfo x 2=56
10:41:36 TRACE [sdk] start_video@2062:   venc took 1580KB of MMZ [297]
10:41:36 DEBUG [sdk] init_chn@1530: JPEG snapshot snapshot venc_chn 2  2592x1520
10:41:36 TRACE [sdk] bind_vpss_venc@1373:   bind_vpss_venc  VPSS(0)->VENC(2)
10:41:36 TRACE [hal] show_blocks@624:   allocated Str x 1=972
10:41:36 TRACE [sdk] start_jpeg@1615:   jpeg_enc took 972KB of MMZ [961]
10:41:36 INFO  [image_tuning] start_image_params_tuning@63: Image tuning task has been started
10:41:36 DEBUG [root_media_supplier] RootMediaSupplier_new@10: new RootMediaSupplier(NAL)
10:41:36 DEBUG [root_media_supplier] RootMediaSupplier_new@10: new RootMediaSupplier(MJPEG)
10:41:36 DEBUG [sdk] start_sdk@736: HiSilicon SDK has been started.
10:41:36 TRACE [hal] hisi_free_mem_ex@694: Free MMZ mem finally: 37068KB
10:41:36 INFO  [evhttp] new_http_server@290: HTTP server has been started on :::80
10:41:36 DEBUG [root_media_supplier] RootMediaSupplier_new@10: new RootMediaSupplier(MP4)
10:41:36 DEBUG [root_media_supplier] RootMediaSupplier_new@10: new RootMediaSupplier(MP4A)
10:41:36 INFO  [rtsp] rtsp_init@153: RTSP server started on port 554
10:41:37 INFO  [rtsp] listener_cb@118: New RTSP client from: 10.0.0.20:53464
10:41:37 DEBUG [client] RtspClient_dispatch@405: RTSP OPTIONS rtsp://10.0.0.101:554?stream=0 CSeq=2 from 10.0.0.20:53464
10:41:37 DEBUG [client] RtspClient_dispatch@405: RTSP DESCRIBE rtsp://10.0.0.101:554?stream=0 CSeq=3 from 10.0.0.20:53464
10:41:37 DEBUG [client] describe_sdp_video@504: RTSP chn 0 sps_len 0
10:41:37 DEBUG [client] describe_sdp_video@507: RTSP chn 0 pps_len 0
10:41:37 DEBUG [client] describe_sdp_video@513: RTSP chn 0 profile_level_id 000000
10:41:37 DEBUG [client] RtspClient_dispatch@405: RTSP SETUP rtsp://10.0.0.101:554?stream=0/video CSeq=4 from 10.0.0.20:53464
10:41:37 DEBUG [client] RtspClient_setup@735: UDP RTSP SETUP done, stream_id=0 from 10.0.0.20:53464
10:41:37 DEBUG [client] RtspClient_dispatch@405: RTSP PLAY rtsp://10.0.0.101:554?stream=0 CSeq=5 from 10.0.0.20:53464
10:41:37 ERROR [nal] RtspNalStream_play@139: REQUEST for 0 stream!
10:41:41 TRACE [watchdog] watchdog_keepalive@103: Watchdog keepalive
10:41:43 INFO  [rtsp] listener_cb@118: New RTSP client from: 10.0.0.16:15298
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP OPTIONS rtsp://10.0.0.101:554?stream=1 CSeq=1 from 10.0.0.16:15298
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP DESCRIBE rtsp://10.0.0.101:554?stream=1 CSeq=2 from 10.0.0.16:15298
10:41:43 DEBUG [client] describe_sdp_video@504: RTSP chn 1 sps_len 0
10:41:43 DEBUG [client] describe_sdp_video@507: RTSP chn 1 pps_len 0
10:41:43 DEBUG [client] describe_sdp_video@513: RTSP chn 1 profile_level_id 000000
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP SETUP rtsp://10.0.0.101:554?stream=1//video CSeq=3 from 10.0.0.16:15298
10:41:43 DEBUG [client] RtspClient_setup@735: TCP RTSP SETUP done, stream_id=1 from 10.0.0.16:15298
10:41:43 INFO  [rtsp] listener_cb@118: New RTSP client from: 10.0.0.16:13343
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP PLAY rtsp://10.0.0.101:554?stream=1 CSeq=4 from 10.0.0.16:15298
10:41:43 ERROR [nal] RtspNalStream_play@139: REQUEST for 1 stream!
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP OPTIONS rtsp://10.0.0.101:554?stream=0 CSeq=1 from 10.0.0.16:13343
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP DESCRIBE rtsp://10.0.0.101:554?stream=0 CSeq=2 from 10.0.0.16:13343
10:41:43 DEBUG [client] describe_sdp_video@504: RTSP chn 0 sps_len 15
10:41:43 DEBUG [client] describe_sdp_video@507: RTSP chn 0 pps_len 3
10:41:43 DEBUG [client] describe_sdp_video@513: RTSP chn 0 profile_level_id 003296
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP SETUP rtsp://10.0.0.101:554?stream=0//video CSeq=3 from 10.0.0.16:13343
10:41:43 DEBUG [client] RtspClient_setup@735: TCP RTSP SETUP done, stream_id=0 from 10.0.0.16:13343
10:41:43 DEBUG [client] RtspClient_dispatch@405: RTSP PLAY rtsp://10.0.0.101:554?stream=0 CSeq=4 from 10.0.0.16:13343
10:41:43 ERROR [nal] RtspNalStream_play@139: REQUEST for 0 stream!
10:41:46 TRACE [watchdog] watchdog_keepalive@103: Watchdog keepalive
10:41:51 TRACE [watchdog] watchdog_keepalive@103: Watchdog keepalive
10:41:56 TRACE [watchdog] watchdog_keepalive@103: Watchdog keepalive
widgetii commented 2 years ago

How do you measure FPS rate of the video encoder?

j0rd commented 2 years ago

How do you measure FPS rate of the video encoder?

I use VLC . ctrl+j while watching video will show video encoder FPS.

For my other non-openipc cameras (same hardware) the proper FPS is shown in video encoder. If I set the camera to 5 FPS the output video is also 5 FPS.

On openipc, if I set the FPS to 5, 5 frames are captured , but video encoding is 30 FPS (which should lead to higher file sizes)

Here's the screenshot https://imgur.com/a/Xz7Rso2

widgetii commented 2 years ago

VLC shows FPS rate of video stream incorrectly (it tries to parse H264/H265 bitstream which is produced by hardware encoder), so it's not an OpenIPC issue

j0rd commented 2 years ago

Seems to work fine with my other cameras.

I have a feeling it's the video encoder.

These are both set to 10 FPS.

OpenIPC https://imgur.com/a/Xz7Rso2

vs. HiSilicon default software https://imgur.com/a/sCmJxo3

I'll download my clips and check the FPS with ffmpeg

j0rd commented 2 years ago

Here's the ffprobe from clip made by OpenIPC

$ ffprobe  57.14.mp4 2>&1 | grep Baseline
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc, bt709), 2592x1520, 2434 kb/s, 10.07 fps, 25 tbr, 90k tbn, 180k tbc (default)

OpenIPC at 1080p

$ ffprobe 15.07.mp4
Duration: 00:00:09.88, start: 0.000000, bitrate: 1192 kb/s
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080, 1190 kb/s, 10.12 fps, 25 tbr, 90k tbn, 180k tbc (default)

Here's ffprobe from video made with default software

$ ffprobe  hisi/17.59.mp4

Duration: 00:00:10.01, start: 0.000000, bitrate: 1794 kb/s
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080, 1793 kb/s, 9.99 fps, 10 tbr, 10240 tbn, 20 tbc (default)

I think the issue is with regards to TBR, which is what's reported as framerate in most programs.

I think you can reduce TBR by reducing TBN. I have no idea what these settings are, but that's what google says.

If this "fix" doesn't lower filesize, or decrease CPU processing of decoding video files. It doesn't really matter.

https://topic.alibabacloud.com/a/the-significance-of-ffmpegs-tbr-tbc-and-tbn_8_8_31517208.html

j0rd commented 2 years ago

OpenIPC file

$ ffprobe 15.07.mp4
Duration: 00:00:09.88, start: 0.000000, bitrate: 1192 kb/s
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080, 1190 kb/s, 10.12 fps, 25 tbr, 90k tbn, 180k tbc (default)

Duration 9.88 seconds. 10 FPS, 25 TBR.

Dump frames via

$ ffmpeg -i 15.07.mp4 'dump-openipc-%04d.png'
$  ls -ltrh dump-openipc-0*png | wc -l
248
$ md5sum dump-openipc-0*png  | awk '{print $1}' | wc -l
248
$ md5sum dump-openipc-0*png  | awk '{print $1}' | sort -u | wc -l
100

~250 frames. 150 frames are duplicates. I believe this would cause some performance issues during decoding, due to duplicate frames which don't need to exist.

Standard software non-openipc

$ ffprobe  hisi/17.59.mp4

Duration: 00:00:10.01, start: 0.000000, bitrate: 1794 kb/s
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080, 1793 kb/s, 9.99 fps, 10 tbr, 10240 tbn, 20 tbc (default)

Duration 10.1 seconds. 10 FPS, 10 tbr

$ ffmpeg -i hisi/17.59.mp4 'dump-frontdoor-%04d.png'
$ ls -ltrh dump-frontdoor-0*png | wc -l
100
$ md5sum dump-frontdoor-0*png  | awk '{print $1}' | wc -l
100
$ md5sum dump-frontdoor-0*png  | awk '{print $1}' | sort -u | wc -l 
100

~100 frames. 0 frames are duplicates.

widgetii commented 2 years ago

Use -vcodec copy for ffmpeg to avoid transcoding and save exactly the same bitstream it receive

j0rd commented 2 years ago

Use -vcodec copy for ffmpeg to avoid transcoding and save exactly the same bitstream it receive

OpenIPC

$ ffmpeg -i openipc-1080.mp4 -c:v copy  'dump-vcopy-openipc-%04d.png'
$ ls -ltrh dump-vcopy-openipc-0*  | wc -l
100
$ md5sum dump-vcopy-openipc-0*png | awk '{print $1}' | sort -u | wc -l
100

Standard Software

$ ffmpeg -i frontdoor.mp4 -c:v copy  'dump-vcopy-frontdoor-%04d.png'
$ ls -ltrh dump-vcopy-frontdoor-0*  | wc -l
100
$ md5sum dump-vcopy-frontdoor-0*png | awk '{print $1}' | sort -u | wc -l
100

Seems to work, which would explain why filesize is similar between the two. I'd still recommend looking into a fix to match FPS, as neglecting to do so, might cause issues with NVR's downstream potentially.

j0rd commented 2 years ago

Hard to debug with out the code, but I believe this discrepancy has something to do with these settings.

stVui.stVuiTimeInfo.num_units_in_tick = 1;
stVui.stVuiTimeInfo.time_scale = _framerate * 2;

aka

HI_U32 num_units_in_tick;                      /* default value: n/a. > 0. */
HI_U32 time_scale;   /* default value: n/a. > 0. */

https://github.com/licaibiao/hisi_sdk_develop/blob/1858c3d5c4183e15ea4c0599ad1db83e1d5726df/Hi3521a_mpp/biao_debug/include/hi_comm_venc.h

typedef struct hiVENC_PARAM_H264_VUI_TIME_INFO_S
{
    HI_U8  timing_info_present_flag;               /* default value: 0. If 1, timing info belows will be encoded into vui. {0,1} */
    HI_U8  fixed_frame_rate_flag;                  /* default value: n/a. {0,1} */
    HI_U32 num_units_in_tick;                      /* default value: n/a. > 0. */
    HI_U32 time_scale;                             /* default value: n/a. > 0. */

}VENC_PARAM_VUI_H264_TIME_INFO_S;

and in here:

typedef struct hiVENC_PARAM_H264_VUI_S
{
    VENC_PARAM_VUI_ASPECT_RATIO_S      stVuiAspectRatio;    /* the param of aspect ratio */     
    VENC_PARAM_VUI_H264_TIME_INFO_S    stVuiTimeInfo;       /* the param of time info */    
    VENC_PARAM_VUI_VIDEO_SIGNAL_S      stVuiVideoSignal;    /* the param of video signal */
}VENC_PARAM_H264_VUI_S;
HI_S32 HI_MPI_VENC_SetH264Vui(VENC_CHN VeChn, const VENC_PARAM_H264_VUI_S *pstH264Vui);
HI_S32 HI_MPI_VENC_GetH264Vui(VENC_CHN VeChn, VENC_PARAM_H264_VUI_S *pstH264Vui);
    if (_encoding == H264) {
        VENC_PARAM_H264_VUI_S stVui;
        if ((s32Ret = HI_MPI_VENC_GetH264Vui(_chnid, &stVui)) == HI_SUCCESS) {
            stVui.stVuiTimeInfo.timing_info_present_flag = 1;
            stVui.stVuiTimeInfo.num_units_in_tick = 1;
            stVui.stVuiTimeInfo.time_scale = _framerate * 2;
            if ((s32Ret = HI_MPI_VENC_SetH264Vui(_chnid, &stVui)) != HI_SUCCESS) {
                HIMPP_PRINT("HI_MPI_VENC_SetH264Vui(%d) failed [%#x]\n",
                            _chnid, s32Ret);
            }
        }

^ if you have some code like this, make sure the _framerate is the FPS set in config and not what ever framerate is defaulted in code.

Unsure if anything like this will work, but from Googling, it appears that "time_scale" is what causes the TBR & TBN to be out of sync with framerate.

widgetii commented 2 years ago

In the latest build I've added HI_MPI_VENC_SetH264Vui/HI_MPI_VENC_SetH265Vui explicit calls to HiSilicon SDK. It seems it does the trick at least with VLC media info

j0rd commented 2 years ago

Updated to OpenIPC v2.1.11.03 Majestic master+70ebb25, 2021-11-03

FPS is fixed for me in VLC now. Thanks for your work on this.

I consider this a closed issue now.

widgetii commented 2 years ago

Thank you for your attention to the issue as well