Some VP9 Webms don't produce video output

[skullchap]

Audio is playing, while nothing for the video itself. Playback was tested on Android. Not sure if related, but FFprobe'ing links in code example below showed vague similarity in encoding between non working examples - vp9 (Profile 0), yuv420p(tv, progressive), while working examples were just vp9 (Profile 0), yuv420p(tv), or either vp8. But strangely two examples were exactly vp9 (Profile 0), yuv420p(tv), while music one with just static image didn't produced any video output, while other one did. Example webms were obtained from 4chan wsg board, and considering tight upload limits, anons sometimes use weird encodings to fit in upload limits, especially for music videos with just static image as a video output. If you try to open those non working example links in browser, or just by commenting registerWith video output will be shown. Other decoders like amediacodec or just mediacodec also didnt work for me.

more webms with similar issues can be obtained here: https://boards.4chan.org/wsg/thread/5480250, https://boards.4chan.org/wsg/thread/5465067 or similar "music" threads at https://boards.4chan.org/wsg. here's the code to test working and non working webm links:

import 'dart:developer';
import 'package:flutter/material.dart';
import 'package:fvp/fvp.dart';
import 'package:video_player/video_player.dart';
import 'package:logging/logging.dart';

void main() {
  Logger.root.level = Level.ALL;
  Logger.root.onRecord.listen((record) {
    log('${record.loggerName}.${record.level.name}: ${record.time}: ${record.message}');
  });
  registerWith(options: {
    'video.decoders': ['FFmpeg'],
  });
  WidgetsFlutterBinding.ensureInitialized();
  runApp(const MainApp());
}

// non working examples
var l = "https://i.4cdn.org/wsg/1712187049637653.webm";
// var l = "https://i.4cdn.org/wsg/1709160710225921.webm"; // vp9 (Profile 0), yuv420p(tv)
// var l = "https://i.4cdn.org/wsg/1712211856616927.webm";
// var l = "https://i.4cdn.org/wsg/1710450881845449.webm";
// var l = "https://i.4cdn.org/wsg/1710450752546962.webm";
// var l = "https://i.4cdn.org/wsg/1710450508972514.webm";
// var l = "https://i.4cdn.org/wsg/1710451662889288.webm";
// var l = "https://i.4cdn.org/wsg/1710598093848058.webm";

// working examples
// var l = "https://i.4cdn.org/wsg/1710822728906286.webm"; // vp9 (Profile 0), yuv420p(tv)
// var l = "https://i.4cdn.org/wsg/1710607050026870.webm"; // vp8
// var l = "https://i.4cdn.org/wsg/1710775017468277.webm"; // vp8

class MainApp extends StatefulWidget {
  const MainApp({super.key});

  @override
  State<MainApp> createState() => _MainAppState();
}

class _MainAppState extends State<MainApp> {
  late final VideoPlayerController vc;

  @override
  void initState() {
    super.initState();
    vc = VideoPlayerController.networkUrl(Uri.parse(l));
    vc.initialize().then((_) => setState(() => vc.play()));
  }

  @override
  void dispose() {
    vc.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
        home: Scaffold(
            bottomNavigationBar: BottomAppBar(
                child:
                    Row(mainAxisAlignment: MainAxisAlignment.center, children: [
              IconButton(
                  onPressed: () => vc.play(),
                  icon: const Icon(Icons.play_arrow)),
              IconButton(
                  onPressed: () => vc.pause(), icon: const Icon(Icons.pause))
            ])),
            body: Center(child: VideoPlayer(vc))));
  }
}

[skullchap]

I found out that most of those non-working webms were 2 pass encoded this way:

ffmpeg -i "image.jpg" -c:v libvpx-vp9 -g 300 -row-mt 1 -pix_fmt yuv420p -b:v 0 -speed 0 -row-mt 0 -threads 1 -tile-columns 0 -tile-rows 0 -cpu-used 0 -crf 15 -pass 1 -an -f null /dev/null && \
ffmpeg -i "image.jpg" -i "audio.mp3" -c:v libvpx-vp9 -g 300 -row-mt 1 -pix_fmt yuv420p -b:v 0 -speed 0 -row-mt 0 -threads 1 -tile-columns 0 -tile-rows 0 -cpu-used 0 -crf 15 -pass 2 -c:a libopus -b:a 128k "output.webm"

I tried to recreate such webm locally and load as an asset inside flutter and it still gave me empty video output, until the very end. Interestingly encoded image actually appears on screen, but somewhere near the end of the video.

At first, it seems like maybe whole video was really empty and the encoded image frame was actually only at the end, but if you play the video with the ffplay it will appear there from the beginning.

[wang-bin]

It's a known bug if a video stream contains only 1 frame. The frame output will be delayed for threaded decoding. For normal videos, the first frame will output after decoding a few packets. if only decode 1 packet, the frame will output at the end. ffplay works because it read the whole file and reach the end of file immediately. I don't have a good solution right now.

[skullchap]

@wang-bin hmm actually setting FFmpeg:threads=1 helped :D thanks for the hint!

[wang-bin]

There is a bug in my code and now this issue should be fixed. In this case the whole file will be read immediately like ffplay and mpv. Please run flutter pub cache clean and build again

[skullchap]

@wang-bin Can confirm, it works now. Thank you for the fix. I was curious what did you fix in code, but couldn’t find any version bumps, any recent commits.

[wang-bin]

in the dependency

[skullchap]

@wang-bin seeking feels weird, it's like it re-downloads already fetched parts when i scrub through.

[wang-bin]

yes, you have to prepend cache: to the url to enable cache, e.g. cache:https://i.4cdn.org/wsg/1712187049637653.webm

[skullchap]

@wang-bin first i did get : [FFmpeg:cache] Protocol 'cache' not on whitelist 'file,rtmp,http,https,tls,rtp,tcp,udp,crypto,httpproxy,data,concatf,concat,subfile'! in logs. I guess it stumbled to this. Anyway i fixed it by passing updated avio.protocol_whitelist to player at registerWith. Now i'm stuck at

[FFmpeg:TEMPFILE] ff_tempfile: Cannot open temporary file ./ffcacheEtRVva
[FFmpeg:cache] Failed to create tempfile
...
Error avrt::avformat_open_input(&fmtctx_, ffmpeg::from_file_uri(in.c_str()), fmt, &dict) @345 
/home/runner/work/mdk-sdk/mdk-sdk/mdk/ffmpeg/plugin/FFmpegPacketIO.cpp: 
(0xffffffe2) Read-only file system

I guess it's Android file storage permission related? How can i pass custom caching directory to it with help of path_provider's getApplicationCacheDirectory() for example?