[video_player_avfoundation] Flickering on macOS

[cbenhagen]

Is there an existing issue for this?

• [X] I have searched the existing issues
• [X] I have read the guide to filing a bug

Steps to reproduce

Run the example in video_player_avfoundation on macOS

Expected results

No flickering

Actual results

Background flickering through the video at random times and always at the end of the video [Edit: as of PR 5078 landing the "always at the end of video" part of this should no longer be true]. Most likely due to returning NULL here:

https://github.com/flutter/packages/blob/c070b0a7a80a54d5fad254fecdfd98ffe764bd4e/packages/video_player/video_player_avfoundation/darwin/Classes/FVPVideoPlayerPlugin.m#L560-L567

While on iOS the last frame would be repeated, on macOS the underlying widget will be visible which results in the flashing effect.

Code sample

Code sample

Use the example in `video_player_avfoundation` on macOS

Screenshots or Video

Screenshots / Video demonstration

https://github.com/flutter/flutter/assets/814785/61ab8a44-0450-4169-971b-ffb27c0ed559

Logs

No response

Flutter Doctor output

Doctor output

```console [✓] Flutter (Channel stable, 3.13.6, on macOS 14.0 23A344 darwin-arm64, locale en-US) • Flutter version 3.13.6 on channel stable at /Users/ben/development/flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision ead455963c (8 days ago), 2023-09-26 18:28:17 -0700 • Engine revision a794cf2681 • Dart version 3.1.3 • DevTools version 2.25.0 [✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/ben/Library/Android/sdk • Platform android-33, build-tools 31.0.0 • Java binary at: /Users/ben/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS (Xcode 15.0) • Xcode at /Applications/Xcode.app/Contents/Developer • Build 15A240d • CocoaPods version 1.13.0 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 2022.3) • Android Studio at /Users/ben/Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231) [✓] IntelliJ IDEA Ultimate Edition (version 2023.2.2) • IntelliJ at /Users/ben/Applications/IntelliJ IDEA Ultimate.app • Flutter plugin version 75.1.4 • Dart plugin version 232.9559.10 [✓] VS Code (version 1.82.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.70.0 [✓] Connected device (4 available) [redacted] [✓] Network resources • All expected network resources are available. ```

[cbenhagen]

/cc @stuartmorgan

[stuartmorgan]

There's a race in the time handling there that I didn't consider when writing the update driver, which is probably the cause of the flicker.

[stuartmorgan]

Please let me know if https://github.com/stuartmorgan/packages/tree/video-player-macos-fallback-frame resolves this for you. I see a lot less flickering than it sounds like you do (I was only seeing one flicker when it loops, and even then only sometimes), but this resolved that for me. (It's not a PR yet since I still need to unit test it, and won't have time to add that today.)

[cbenhagen]

Thanks for the fast response! Yes that branch resolves the flickering. However the texture is not updated on seek while paused anymore. It could also be smoother on seek when playing.

[stuartmorgan]

However the texture is not updated on seek while paused anymore.

It looks like it updates every other seek, both with and without that change, so it seems to be unrelated. We'll need to file that to investigate separately.

[cbenhagen]

@stuartmorgan unfortunately while already a lot better the issue still persists even on your branch at the latest commit https://github.com/stuartmorgan/packages/commit/64d289822b76686f6e0199a170f2986060fb5df2

[stuartmorgan]

Okay, I've updated the PR description so that it won't auto-close this. I won't have time to re-investigate the remaining issue in the near future, so I'll land that PR as-is and leave this for future work.

[cliftonlabrum]

For what it's worth, I think this might be related to a Mac's ProMotion display or potentially the display resolution.

If I use my M3 MBP screen, I see flickering every couple seconds as shown here:

https://github.com/flutter/flutter/assets/2513753/c69ea038-6e56-4f5a-a178-fee2537fcb9a

If I use my Studio Display, the same video does not flicker:

https://github.com/flutter/flutter/assets/2513753/af1b614d-d6a3-4fc2-b610-084800901b66

I'm on Flutter 3.16 and video_player 2.8.1. Thank you for your help! 😊

[cbenhagen]

@cliftonlabrum can you test this with Stuart's branch?

dependency_overrides:
  video_player_avfoundation:
    git:
      url: https://github.com/stuartmorgan/packages.git
      path: packages/video_player/video_player_avfoundation

[cliftonlabrum]

@cbenhagen Thank you for the suggestion. It doesn't look like that package is available anymore. I don't see it in here: https://github.com/stuartmorgan/packages/tree/master/packages

[gspencergoog]

@stuartmorgan did your frame management patch resolve this issue? (Desktop Triage)

[stuartmorgan]

I thought it would, but https://github.com/flutter/flutter/issues/135999#issuecomment-1781273605 indicated it didn't completely fix it.

[cliftonlabrum]

@stuartmorgan If you can make your fix available again, I'm happy to test it in my MBP vs. Studio Display scenario up above.

[stuartmorgan]

@stuartmorgan If you can make your fix available again

The fix is published, you just need to flutter pub upgrade.

[cliftonlabrum]

The issue seems to be resolved for me now that I'm using @stuartmorgan's fix. Thank you, Stuart! 😄

[cbenhagen]

For me the published version is still not fixing the flickering issues. It does not matter if I use a ProMotion display or not. Tested with video_player_avfoundation 2.5.2 on Flutter 3.16 and macOS 14.1.1. Duplicating the last frame when null is received like the engine does on iOS is not an option on macOS I guess?

[cbenhagen]

@stuartmorgan Most of the time the flickering I am seeing seems not to be due to the background being rendered but because frames being displayed out of order. The background still flickers through from time to time but not as often as before.

You can easily reproduce the issue by slightly resizing the window but it also happens if you just leave it running: https://github.com/flutter/flutter/assets/814785/ab42d4fc-5adb-415d-aa84-098e3ff31a0a

Here you can see the background flickering through and a generally a playback which is otherwise stuttering and missing frames compared to the same video played back next to it with ffplay. https://github.com/flutter/flutter/assets/814785/6f80e472-957c-431f-b6b3-fa9007e3b5ba

The video I used is this: https://github.com/flutter/flutter/assets/814785/6a42f1ea-f6b7-4d15-935d-7f4f28f08afb

The code is the video_player_avfoundation example using the url below run with Flutter 3.16 in release mode. https://github-production-user-asset-6210df.s3.amazonaws.com/814785/285154603-6a42f1ea-f6b7-4d15-935d-7f4f28f08afb.mp4

While this video is in 60fps to make the bad playback performance very obvious, our usual clips are 24 or 25 fps.

@cbracken, @hellohuanlin, @tarrinneal as you have been reviewing the PRs related to video_player_avfoundation maybe one of you has an idea how to further improve its frame management and general performance?

[stuartmorgan]

Most of the time the flickering I am seeing seems not to be due to the background being rendered but because frames being displayed out of order.

Does adding self.frameUpdater.lastKnownAvailableTime = outputItemTime here fix that?

[cbenhagen]

No unfortunately it does not fix it. Have you been able to reproduce the issue?

[stuartmorgan]

I haven't had time to investigate it, so I haven't tried; I just looked briefly at the code again. I'm happy to answer questions about the code if someone in the community wants to investigate, but I don't expect to work on this directly in the near future.

[cbenhagen]

@knopp could this be related to https://github.com/flutter/flutter/issues/49757 and fixed by https://github.com/flutter/engine/pull/49159 or if not, do you see a similar fix for this issue?

[knopp]

@knopp could this be related to #49757 and fixed by flutter/engine#49159 or if not, do you see a similar fix for this issue?

Unlikely. My hunch there is that this is either the problem with external texture implementation or player itself.

[tannker]

Has there been any progress with this issue or is any investigation / fixing planned? Has anyone found any workaround?

I have encountered flickering on MacOS as well.

[knopp]

Can you confirm that this persists with latest main?

[tannker]

Yes, it still persists. [✓] Flutter (Channel stable, 3.19.6, on macOS 14.4.1 23E224 darwin-arm64) video_player: ^2.8.6 video_player_avfoundation: ^2.5.7

[cbenhagen]

@knopp yes this can still be reproduced on main. All comments in https://github.com/flutter/flutter/issues/135999#issuecomment-1824123379 are still valid.

[✓] Flutter (Channel main, 3.22.0-16.0.pre.61, on macOS 14.4.1 23E224 darwin-arm64, locale en-CH)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2023.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2024.1)
[✓] VS Code (version 1.71.2)
[✓] Connected device (3 available)
[✓] Network resources

• No issues found!

[cbenhagen]

@knopp Actually, there is a difference to the video I linked in that comment. The background is not visible anymore when the video flickers and stutters. It's still as bad as before though. Part of it might be that it's not synced to the monitors refresh rate. But there also frames displayed completely out of order. To reproduce, I recommend to use the 60fps video linked above. Instead of waiting you can slightly resize the window. But it also happens when you just leave it running.