search

wang-bin / fvp

Flutter video player plugin for all desktop+mobile platforms. download prebuilt examples from github actions. https://pub.dev/packages/fvp
BSD 3-Clause "New" or "Revised" License
126 stars 20 forks source link

Inconsistent getPosition result #84

Open tannker opened 1 month ago

tannker commented 1 month ago

Describe the bug After loading video, I call after few second delay videoController.positon.then((value){print(value)}. The result is inconsistent. Sometimes it returns 0, sometimes 20 or 80 milliseconds. The same issue occurs when calling position after videoController.seekTo. After seeking to the start (Duration.zero or Duration(milliseconds: 0) or to the end, the position value returned varies. The variation is in some milliseconds, but on most cases it is not equal to actual start or end. I made some debugging and it seems that this error is already present at video_player_mdk.dart function getPosition. This function I guess already gets input from libmdk.

Additionally, I updated mdk pod from 0.25.1 to the latest 0.27 one, but the behaviour remained the same. The test is made with latest version of fvp - 0.17.0. I am testing on MacOS Sonoma 14.4.1.

Expected behavior Position value is consistent across all calls.

Log

flutter: mdk.INFO: 14:30:39.909: default0x15908c470 new FrameReader...
flutter: mdk.INFO: 14:30:39.909: Registered audio backends: AudioQueue OpenAL null 
flutter: mdk.INFO: 14:30:39.909: default0x159089690 new FrameReader...
flutter: mdk.INFO: 14:30:39.909: Registered audio backends: AudioQueue OpenAL null 
flutter: mdk.INFO: 14:30:39.909: 0x1590971c0mdk::Player::Player()
flutter: mdk.INFO: 14:30:39.910: player property: video.decoder = shader_resource=0
flutter: mdk.INFO: 14:30:39.910: player property: avformat.strict = experimental
flutter: mdk.INFO: 14:30:39.910: player property: avio.protocol_whitelist = file,rtmp,http,https,tls,rtp,tcp,udp,crypto,httpproxy,data,concatf,concat,subfile
flutter: mdk.INFO: 14:30:39.910: player property: avformat.rtsp_transport = tcp
flutter: mdk.INFO: 14:30:39.910: player property: keep_open = 1
flutter: mdk.INFO: 14:30:39.910: video decoders: "VT"
flutter: mdk.INFO: 14:30:39.910: video decoders: "VT"
flutter: mdk.INFO: 14:30:39.910: default FrameReader0x15908c470 state: 0=>0=>0, 0
flutter: mdk.INFO: 14:30:39.910: default0x15908c470 virtual void mdk::FrameReader::stop(), current state: 0
flutter: mdk.INFO: 14:30:39.910: default FrameReader0x15908c470 state: 0=>0=>0, 0
flutter: mdk.INFO: 14:30:39.910: default0x15908c470 virtual void mdk::FrameReader::stop(), current state: 0
flutter: mdk.INFO: 14:30:39.910: AudioQueueStart ERROR: 0
flutter: mdk.INFO: 14:30:39.910: 0x15a047800 MediaControl.prepare(0, ...) file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.910: default 0x15908c470 FrameReader.start(0, ...)
flutter: mdk.INFO: 14:30:39.910: default 0x15908c470 FrameReader::update MediaStatus 0=>0X2
flutter: fvp.FINE: 14:30:39.913: 468818744 player5788597168 onMediaStatus: MediaStatus(noMedia) => MediaStatus(+loading)
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state: 0=>0=>1, 0
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state: 0=>1=>1, 0
flutter: mdk.INFO: 14:30:39.914: start frame reader thread: 0x178667000
flutter: mdk.INFO: 14:30:39.914: Trying MediaIO FFmpeg
flutter: mdk.INFO: 14:30:39.914: 0x600001d1e880 open url: file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.914: 0x600001d1e880 url opened
flutter: mdk.INFO: 14:30:39.914: format adts is used for file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.914: default 0x15908c470 FrameReader::update MediaStatus 0X2=>0X2
flutter: mdk.INFO: 14:30:39.914: 0x15906dcd0 avio set class
flutter: mdk.INFO: 14:30:39.914: before avformat_open_input. io: 0x600001d1e880(FFmpeg/pb: 0x15906dcd0, url: file:///Volumes/Macintosh HD/Users//Downloads/test_videos/test.MP4
flutter: mdk.INFO: 14:30:39.914: after avformat_open_input. pb: 0x15906dcd0, iformat: 0x15c61dcf0 context flags: 2097280, input format flags: 67141640
flutter: mdk.INFO: 14:30:39.914: unsupported avformat options:  rtsp_transport=tcp
flutter: mdk.INFO: 14:30:39.914: Unsupported stream type: data, codec: unknown
flutter: mdk.INFO: 14:30:39.914: Unsupported stream type: data, codec: bin_data
flutter: mdk.INFO: 14:30:39.914: Format: mov,mp4,m4a,3gp,3g2,mj2, range: 0 +19136ms, bitrate: 59788566, size: 0
Metadata:
  firmware: H22.01.02.20.00
  creation_time: 2024-01-17T10:24:21.000000Z
  minor_version: 538120216
  compatible_brands: mp41
  major_brand: mp41
Streams: 4
 Video:
  stream#0, range: 0 +19130ms, frames: 1913
  codec: hevc tag: 'hvc1' profile: 1 level: 153, yuv420p, bpc:8, bpp:12(8,8,8), channels:(1,1,1), map: 0 1 2 0, bitrate: 59513249, 1920x1080, fps: 100, bframes: 0, primaries: BT709, trc: BT709, matrix: BT709, range: Full
  extra data(117): 01 01 60 00 00 00 00 00 00 00 00 00 99 F0 00 FC FD F8 F8 00 00 0F 03 A0 00 01 00 21 40 01 0C 01 FF FF 01 60 00 00 03 00 00 03 00 00 03 00 00 03 00 99 BC 0C 00 00 0F A0 00 06 1A 81 40 A1 00 01 00 27 42 01 01 01 60 00 00 03 00 00 03 00 00 03 00 00 03 00 99 A0 03 C0 80 10 E5 96 F9 24 C0 8F E5 CF 4B 01 6E 02 02 02 01 A2 00 01 00 07 44 01 E0 2D AC 09 90 
  Metadata:
   encoder: GoPro H.265 encoder
   handler_name: GoPro H.265
   vendor_id: [0][0][0][0]
   language: eng
   timecode: 11:24:21:82
   creation_time: 2024-01-17T10:24:21.000000Z
 Audio:
  stream#1, range: 0 +19136ms, frames: 897
  codec: aac tag: 'mp4a' profile: 1 level: -99, sample size: 0/24, block align: 0, frame_size: 1024, f32p stereo(2) @48000Hz, bitrate: 189255
  extra data(5): 11 90 00 00 00 
  Metadata:
   handler_name: GoPro AAC  
   vendor_id: [0][0][0][0]
   language: eng
   timecode: 11:24:21:82
   creation_time: 2024-01-17T10:24:21.000000Z

flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 request to pause 1, loaded: 0.
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state: 0=>1=>2, 0
flutter: mdk.INFO: 14:30:39.914: default FrameReader0x15908c470 state requested: 2, current: 0
flutter: mdk.INFO: 14:30:39.915: virtual void mdk::AudioBackendAudioQueue::pause(bool)138 ERROR>>> AudioQueuePause(queue_): -50
flutter: mdk.INFO: 14:30:39.915: 
flutter: mdk.INFO: 14:30:39.915: 0x15a047800 prepared callback is invoked
flutter: fvp.FINE: 14:30:39.925: 468818744 player5788597168 onEvent: metadata -  - 0
flutter: mdk.INFO: 14:30:39.925: default 0x15908c470 FrameReader::update MediaStatus 0X2=>0X4
flutter: fvp.FINE: 14:30:39.925: 468818744 player5788597168 onMediaStatus: MediaStatus(+loading) => MediaStatus(+loaded)
flutter: mdk.INFO: 14:30:39.926: ***buffering progress 0%***
flutter: mdk.INFO: 14:30:39.926: default 0x15908c470 FrameReader::update MediaStatus 0X4=>0X14
flutter: mdk.INFO: 14:30:39.926: virtual void mdk::AudioBackendAudioQueue::pause(bool)138 ERROR>>> AudioQueuePause(queue_): -50
flutter: mdk.INFO: 14:30:39.926: 
flutter: fvp.FINE: 14:30:39.926: 468818744 player5788597168 onMediaStatus: MediaStatus(+loaded) => MediaStatus(+loaded+buffering)
flutter: mdk.INFO: 14:30:39.926: ++++++++++++BUFFERING START++++++++++++
flutter: mdk.INFO: 14:30:39.926: 
flutter: fvp.FINE: 14:30:39.926: 468818744 player5788597168 onEvent: reader.buffering -  - 0
flutter: fvp.FINE: 14:30:39.927: 468818744 player5788597168 onEvent: metadata -  - 0
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 state: 0=>2=>1, 1
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 update state: 0=>1
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onPlaybackStateChanged: PlaybackState.stopped => PlaybackState.playing
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 state: 1=>1=>2, 1
flutter: mdk.INFO: 14:30:39.928: default FrameReader0x15908c470 update state: 1=>2
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onPlaybackStateChanged: PlaybackState.playing => PlaybackState.paused
flutter: mdk.INFO: 14:30:39.928: starting decode loop thread mdk.vdec0@0x17877f000
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onEvent: thread.video -  - 1
flutter: mdk.INFO: 14:30:39.928: video stream#0 starting decoding loop from decoder index 0...
flutter: mdk.INFO: 14:30:39.928: creating video decoder: VT...
flutter: mdk.INFO: 14:30:39.928: 0x159243058mdk::NativeVideoBufferPool::NativeVideoBufferPool()
flutter: mdk.INFO: 14:30:39.928: Try to use IOSurface OpenGL(ES) interop: true
flutter: mdk.INFO: 14:30:39.928: opening video decoder: VT...
flutter: mdk.INFO: 14:30:39.928: starting decode loop thread mdk.adec1@0x17880b000
flutter: fvp.FINE: 14:30:39.928: 468818744 player5788597168 onEvent: thread.audio -  - 1
flutter: mdk.INFO: 14:30:39.929: audio stream#1 starting decoding loop from decoder index 0...
flutter: mdk.INFO: 14:30:39.929: creating audio decoder: auto...
flutter: mdk.INFO: 14:30:39.929: opening audio decoder: FFmpeg...
flutter: mdk.INFO: 14:30:39.929: opening ffmpeg audio decoder: aac ...
flutter: mdk.INFO: 14:30:39.929: AVCodec.Audio[aac.] decoder: 1 Single threads
flutter: mdk.INFO: 14:30:39.929: aac f32p, stereo @48000Hz
flutter: fvp.FINE: 14:30:39.929: 468818744 player5788597168 onEvent: decoder.audio - FFmpeg - 0
flutter: mdk.INFO: 14:30:39.929: ***buffering progress 100%***
flutter: mdk.INFO: 14:30:39.929: default 0x15908c470 FrameReader::update MediaStatus 0X14=>0X124
flutter: mdk.INFO: 14:30:39.929: virtual void mdk::AudioBackendAudioQueue::pause(bool)138 ERROR>>> AudioQueuePause(queue_): -50
flutter: mdk.INFO: 14:30:39.929: 
flutter: fvp.FINE: 14:30:39.929: 468818744 player5788597168 onMediaStatus: MediaStatus(+loaded+buffering) => MediaStatus(+loaded+prepared+buffered)
flutter: mdk.INFO: 14:30:39.929: ++++++++++++BUFFERING END++++++++++++
flutter: mdk.INFO: 14:30:39.929: 
flutter: fvp.FINE: 14:30:39.929: 468818744 player5788597168 onEvent: reader.buffering -  - 100
flutter: mdk.INFO: 14:30:39.929: audio stream#1 sending 1 invalid AOT frame @0.000000s. seeking: 0
flutter: mdk.INFO: 14:30:39.929: 0x15a047800 1st audio frame @0.000000
flutter: mdk.INFO: 14:30:39.929: 0x15a047800 seek end audio frame @0.000000 seek_pos_: -1, sync_ao_ 1
flutter: mdk.INFO: 14:30:39.929: AudioRenderer format: f32, stereo @48000Hz, requested: f32p, stereo @48000Hz
flutter: mdk.INFO: 14:30:39.929: AudioRenderer format changed and reinitialize
flutter: mdk.INFO: 14:30:39.929: codec 'hvc1' for VT
flutter: mdk.INFO: 14:30:39.929: 0x1590971d0 setRenderAPI(3, 0x0)
flutter: mdk.INFO: 14:30:39.929: getVideoOutContext 0x0 => 0x60000317b030. vo count: 1
flutter: mdk.INFO: 14:30:39.929: 0x1590971d0 setVideoSurfaceSize(1920x1080, 0x0)
flutter: mdk.INFO: 14:30:40.041: VT hardware decode support: 1
flutter: mdk.INFO: 14:30:40.041: NALU length bytes: 4
flutter: mdk.INFO: 14:30:40.041: <CMVideoFormatDescription 0x600003ca8f90 [0x1f29b48c0]> {
    mediaType:'vide' 
    mediaSubType:'hvc1' 
    mediaSpecific: {
        codecType: 'hvc1'       dimensions: 1920 x 1080 
    } 
    extensions: {{
    CVImageBufferColorPrimaries = "ITU_R_709_2";
    CVImageBufferTransferFunction = "ITU_R_709_2";
    CVImageBufferYCbCrMatrix = "ITU_R_709_2";
    Depth = 24;
    FormatName = hvc1;
    FullRangeVideo = 1;
    SampleDescriptionExtensionAtoms =     {
        hvcC = {length = 117, bytes = 0x01016000 00000000 00000000 99f000fc ... 074401e0 2dac0990 };
    };
}}
}
flutter: mdk.INFO: 14:30:40.041: setLoop(0), now 0/0
flutter: mdk.INFO: 14:30:40.041: 0x15a047800 virtual void mdk::MediaControlPush::setState(PlaybackState)@1051 requested state 2=>2, current state 2. status: 0X124
flutter: mdk.INFO: 14:30:40.041: UsingHardwareAcceleratedVideoDecoder: 1
flutter: fvp.FINE: 14:30:40.041: 468818744 player5788597168 onEvent: decoder.video - VT - 0
flutter: mdk.INFO: 14:30:40.042: decode stored recovery packets: 0
flutter: mdk.INFO: 14:30:40.042: video stream#0 sending 1 invalid AOT frame @0.000000s. seeking: 0
flutter: mdk.INFO: 14:30:40.042: 0x15a047800 1st video frame to render @0.000000s, sync time: -0.001000
flutter: mdk.INFO: 14:30:40.042: VideoRenderer clear buffered frames
flutter: mdk.INFO: 14:30:40.042: 0-track seek end video frame @0.000000 seek_pos_: -1
flutter: mdk.INFO: 14:30:40.042: 0x15a047800 1st video frame to render @0.000000s, sync time: -0.001000
flutter: fvp.FINE: 14:30:40.042: 468818744 player5788597168 onEvent: video - size - 0
flutter: mdk.INFO: 14:30:40.042: VideoRenderer WxH=1920.000000x1080.000000 1920.000000x1080.000000, frame: 1920x1080
flutter: mdk.INFO: 14:30:40.042: update transform 2d. scale: (1.000000, 1.000000), orientation: 0
flutter: mdk.INFO: 14:30:40.042: m: mat<4, 4>(
flutter: mdk.INFO: 14:30:40.042: 1  0  0  0  
flutter: mdk.INFO: 14:30:40.042: 0  1  0  0  
flutter: mdk.INFO: 14:30:40.042: 0  0  1  0  
flutter: mdk.INFO: 14:30:40.042: 0  0  0  1  
flutter: mdk.INFO: 14:30:40.042: )
flutter: mdk.INFO: 14:30:40.042: unknown, bpc:0, bpp:0(), channels:(), map: 0 0 0 0 => mapped nv12, bpc:8, bpp:12(8,16), channels:(1,2), map: 0 1 2 0
flutter: mdk.INFO: 14:30:40.042: Metal pixel format 0: MTLPixelFormatR8Unorm
flutter: mdk.INFO: 14:30:40.042: Metal pixel format 1: MTLPixelFormatRG8Unorm
flutter: mdk.INFO: 14:30:40.042: frame color: BT709 BT709 BT709 Full. gamma: 2.200000, signal peak: 1.000000
flutter: mdk.INFO: 14:30:40.042:  - 
flutter: mdk.INFO: 14:30:40.042: output color: BT709 UNSPECIFIED RGB Full. gamma: 2.200000, signal peak: 1.000000
flutter: mdk.INFO: 14:30:40.042: rgb primaries matrix: mat<3, 3>(
flutter: mdk.INFO: 14:30:40.042: 1  0  0  
flutter: mdk.INFO: 14:30:40.042: 0  1  0  
flutter: mdk.INFO: 14:30:40.042: 0  0  1  
flutter: mdk.INFO: 14:30:40.043: )
flutter: mdk.INFO: 14:30:40.043: scale: 1.000000, color transform: mat<4, 4>(
flutter: mdk.INFO: 14:30:40.043: 1  0  1.5748  -0.7874  
flutter: mdk.INFO: 14:30:40.043: 1  -0.187324  -0.468124  0.327724  
flutter: mdk.INFO: 14:30:40.043: 1  1.8556  0  -0.9278  
flutter: mdk.INFO: 14:30:40.043: 0  0  0  1  
flutter: mdk.INFO: 14:30:40.043: )
flutter: mdk.INFO: 14:30:40.043: EQ: mat<4, 4>(
flutter: mdk.INFO: 14:30:40.043: 1  0  0  0  
flutter: mdk.INFO: 14:30:40.043: 0  1  0  0  
flutter: mdk.INFO: 14:30:40.043: 0  0  1  0  
flutter: mdk.INFO: 14:30:40.043: 0  0  0  1  
flutter: mdk.INFO: 14:30:40.043: )
flutter: mdk.INFO: 14:30:40.043: shader id: 2321
flutter: mdk.INFO: 14:30:40.043: dump UBO layout:
align @offset +size /stride address name ~ path: type -> parent
16@0 +64/16 0x60000104daa8 u_colorMatrix ~ u_colorMatrix: mat4 -> 0x0            

flutter: fvp.FINE: 14:30:40.043: 468818744 player5788597168 onEvent: render.video - 1st_frame - 0
flutter: mdk.INFO: 14:30:40.043: Using audio backend: AudioQueue
flutter: mdk.INFO: 14:30:40.043: default FrameReader0x15908c470 request to pause 1, loaded: 4.
flutter: mdk.INFO: 14:30:40.043: default FrameReader0x15908c470 state: 2=>2=>2, 0
flutter: mdk.INFO: 14:30:40.043: 0x15a047800 ao opened: 1, reopen: 1, bad ao: 0
flutter: mdk.INFO: 14:30:40.043: 0x6000002268e8 filter graph: 
flutter: mdk.INFO: 14:30:40.043: +-----------+
|    src    |default--[48000Hz fltp:stereo]--swr:default
| (abuffer) |
+-----------+

                                          +---------------+
swr:default--[48000Hz flt:stereo]--default|      dst      |
                                          | (abuffersink) |
                                          +---------------+

                                           +-------------+
src:default--[48000Hz fltp:stereo]--default|     swr     |default--[48000Hz flt:stereo]--dst:default
                                           | (aresample) |
                                           +-------------+
wang-bin commented 1 month ago

If video is paused after load, the position should be the first video frame timestamp(there is a bug in current code and will be fixed soon). If not paused, the position varies. seek is the same.

tannker commented 1 month ago

Ok. Yes, what I have noticed is the position after initial load (sometimes 0, 20, 80 ms). I also noticed the same issue with seekTo to either to the start (Duration.zero - again 0, 20, 80 ms) or to the end (videoController.value.duration - position value differs slightly from the actual duration).

wang-bin commented 1 month ago

upgrade libmdk

pod cache clean mdk
pod deintegrate ios/Runner.xcodeproj
rm ios/Podfile.lock
rm -rf ios/Pods