Video Scrubbing Issue

[henrique-marques-present]

Plugin version: ^2.3.4

Issue:

When scrubbing through a video, it appears that the preview is stacking frames from the seek requests instead of displaying the correct frame corresponding to the scrubbed position. Is there a way to fix this issue with the Media Foundation API or this management has to be done on frontend?

Current behavior

https://github.com/jakky1/video_player_win/assets/147726319/a7afd232-dfa1-4cb8-bd12-458fd5869211

Code sample

// main.dart
import 'dart:developer';
import 'dart:io';

import 'package:flutter/material.dart';
import 'package:video_player/video_player.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatefulWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  VideoPlayerController? controller;
  double value = 0;

  void reload() {
    controller?.dispose();
    controller = VideoPlayerController.file(File("C:\\video.mp4"));
    //controller = WinVideoPlayerController.file(File("E:\\test_youtube.mp4"));
    //controller = VideoPlayerController.networkUrl(Uri.parse("https://media.w3.org/2010/05/sintel/trailer.mp4"));
    //controller = WinVideoPlayerController.file(File("E:\\Downloads\\0.FDM\\sample-file-1.flac"));

    controller!.initialize().then((value) {
      if (controller!.value.isInitialized) {
        controller!.play();
        setState(() {});

        controller!.addListener(() {
          if (controller!.value.isCompleted) {
            log("ui: player completed, pos=${controller!.value.position}");
          }
        });
      } else {
        log("video file load failed");
      }
    }).catchError((e) {
      log("controller.initialize() error occurs: $e");
    });
    setState(() {});
  }

  @override
  void initState() {
    super.initState();
    reload();
  }

  @override
  void dispose() {
    super.dispose();
    controller?.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        appBar: AppBar(
          title: const Text('video_player_win example app'),
        ),
        body: Stack(children: [
          VideoPlayer(controller!),
          Positioned(
              bottom: 0,
              child: Column(children: [
                ValueListenableBuilder<VideoPlayerValue>(
                  valueListenable: controller!,
                  builder: ((context, value, child) {
                    int minute = value.position.inMinutes;
                    int second = value.position.inSeconds % 60;
                    String timeStr = "$minute:$second";
                    if (value.isCompleted) timeStr = "$timeStr (completed)";
                    return Text(timeStr,
                        style: Theme.of(context).textTheme.headline6!.copyWith(
                            color: Colors.white,
                            backgroundColor: Colors.black54));
                  }),
                ),
                ElevatedButton(
                    onPressed: () => restart(), child: const Text("Reload")),
                ElevatedButton(
                    onPressed: () => controller?.play(),
                    child: const Text("Play")),
                ElevatedButton(
                    onPressed: () => controller?.pause(),
                    child: const Text("Pause")),
                ElevatedButton(
                    onPressed: () => controller?.seekTo(Duration(
                        milliseconds:
                        controller!.value.position.inMilliseconds +
                            1 * 1000)),
                    child: const Text("Forward")),
                ElevatedButton(
                    onPressed: () {
                      int ms = controller!.value.duration.inMilliseconds;
                      var tt = Duration(milliseconds: ms - 1000);
                      controller?.seekTo(tt);
                    },
                    child: const Text("End")),
                Slider(
                  value: value,
                  onChanged: (double value) {
                    setState(() {
                      this.value = value;
                      controller!.seekTo(
                        Duration(
                            milliseconds:
                            (value * Duration.millisecondsPerSecond)
                                .toInt()),
                      );
                    });
                  },
                  min: 0,
                  max: controller!.value.duration.inSeconds.toDouble(),
                )
              ])),
        ]),
      ),
    );
  }

  restart() {
    controller!.seekTo(Duration.zero);
  }
}

Flutter doctor

[√] Flutter (Channel stable, 3.16.0, on Microsoft Windows [Version 10.0.22631.3155], locale en-US)
    • Flutter version 3.16.0 on channel stable at C:\Users\Teste\fvm\versions\3.16.0
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision db7ef5bf9f (3 months ago), 2023-11-15 11:25:44 -0800
    • Engine revision 74d16627b9
    • Dart version 3.2.0
    • DevTools version 2.28.2

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at C:\Users\Teste\AppData\Local\Android\sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: C:\Program Files\Android\Android Studio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-b2043.56-10550314)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.9.0)
    • Visual Studio at C:\Program Files\Microsoft Visual Studio\2022\Community
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [Version 10.0.22631.3155]
    • Chrome (web)      • chrome  • web-javascript • Google Chrome 121.0.6167.187
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 121.0.2277.128

[√] Network resources
    • All expected network resources are available.

• No issues found!

[jakky1]

I have noticed this problem before. However, I have no idea how to fix it :(

In Media Foundation API, it always force playing after seeking, even if the video paused before seeking. So I call pause() immediately after user calling seek() in paused video, but this seems cause problem.

I think the Media Foundation didn't take this situation into consideration. When I playing video by built-in Windows Media Player (WMP), pause it, the video frame won't refresh after seeking. I think WMP didn't call seek() when seeking paused video, just because the API doesn't support seeking in paused video.

[henrique-marques-present]

One way to slightly improve this behavior is by implementing a debounce feature for the seek operation, which helps prevent rapid or unnecessary triggering. Wouldn't it be feasible to perform this debouncing on the plugin side?

  // debounce duration
  final Duration debounceTime = const Duration(milliseconds: 100);

  // Debounce timer
  Timer? _scrubDebounceTimer;
  DateTime? _timerStartMoment;

  // ...

  /// Scrub the video and the animation with a debounce using a duration
  /// of [debounceTime].
  ///
  /// The debounce prevents the seek from being updated during any type of
  /// scrub operation.
  void scrub(Duration duration) {
    // if is active create a new timer with the new seek value
    if (_scrubDebounceTimer?.isActive ?? false) {
      Duration elapsedTime = DateTime.now().difference(_timerStartMoment!);

      // cancel the previous action
      _scrubDebounceTimer!.cancel();

      // evaluate the left duration to throw a seek event
      final Duration leftDurationToUpdate = debounceTime - elapsedTime;

      // re-recreate a timer with the new duration
      _scrubDebounceTimer = Timer(leftDurationToUpdate, () => controller!.seekTo(duration);
      return;
    }

    //
    // If the timer is not active or not yet initialized, the timer
    // and the datetime are initialize
    //

    _timerStartMoment = DateTime.now(); // time reference for when the [_seekDebounceTimer] started
    _scrubDebounceTimer = Timer(debounceTime, () => controller!.seekTo(duration);
  }

  // ...

[jakky1]

I prefer not to do it in plugin side because I think API should do only the essential task as possible. And it may be not a good idea if user just only want to call seek() once, in this case they can find the seekTo() called with a 100ms delay.

In UI level, developer can know if user just want to call seekTo() once (ex. click slider, press arrow key once) or user long-press arrow key / dragging slider to make multiple seekTo() calls. So I think the "delay" code should implement in UI code level, not in plugin side.

[glanium]

Media Foundation document has a page for scrub How to Perform Scrubbing.

Seeking, Fast Forward, and Reverse Play

In this example, seeking requests are queued but source code are complicated >.<

I think it is necessary to handle Media foundation's async events for better seeking(scrub) but complicated >.<

[jakky1]

Oops... I used to think that video scrubbing was simply a series of fast seeks, but I didn't know that the key point was to display the current video frame after each seek operation.

It seems easy to implement according to the webpage you mentioned above, But, unfortunately... I tried to all SetRate(0), media foundation return 0 (OK) but video still playing... it seems SetRate(0) not working ( but SetRate(0.5) works ). Then I tried to call Pause() -> SetRate(0) -> a lot of Seek(ms)... frames not update after each seek operation...

So far I have no idea how to implemet it... orz