Encoding into WebM with an unsupported audio stream panics

[Pyroglyph]

When encoding with FFmpeg, it will automatically re-encode unsupported channels to match the target container. This currently doesn't happen with ab-av1, and it panics instead. This appears to be because FFmpeg is being told to copy the audio stream rather than to let it decide on its own. I've only tested this with the WebM container, but I imagine this also applies when using other containers with limited support for other formats.

I'll be referring to this 1MB sample video as input.mp4. It's format is H264/AAC.

Running this will currently cause a panic.

ab-av1 encode -i input.mp4 --crf 20 -o output.webm

This line appears in FFmpeg's stderr:

[webm @ 000001e53518a3c0] Only VP8 or VP9 or AV1 video and Vorbis or Opus audio and WebVTT subtitles are supported for WebM.

This is because ab-av1 is telling FFmpeg to copy the AAC stream into the WebM, which won't work because WebM only supports Vorbis and Opus. Specifying --acodec libopus allows the video to encode normally, so this is definitely the cause.

Since libopus is already in ab-av1's requirements, I vote that when WebM is selected as the output container and the input audio stream is incompatible, it should be automatically converted to Opus so as not to cause the conversion to fail. This mirrors FFmpeg's current behaviour.

[alexheretic]

Yes I agree it does seem like this could be handled automatically.

One issue might be detecting the current audio stream(s) and determining if they are not free codecs. But perhaps it could detect this error and then retry using libopus?

[mr44er]

This will start a chain to more unforeseen cases. If the source file has multichannel-audio, it also stops. $anything-multichannel to opus still expects -af aformat=channel_layouts="7.1|5.1"

I don't know if you want to keep track on every case what a specific container can do or not. The next guy will cry that he wants vorbis as default because why not? :)

I would allow .mp4 and .mkv only, with .mkv as default, because it allows every combination and avoids errors/stops. If someone wants .webm in the end, he should use ffmpeg afterwards ffmpeg -i file.av1.mkv -c copy output.webm and deal with his specific error message per container.

Or maybe this could be split for convenience into two parts with a tempfile.av1.mkv that the encode will run 100%, after that it triggers the second command ffmpeg -i tempfile.av1.mkv -c copy output.webm If it works, good, if not the user has not lost any progress, he just needs to adjust his parameters for .webm

[veikk0]

I've run into multiple similar issues with WebM output, though not with audio.

Unsupported subtitle streams cause this issue as well. When encoding from a Blu-ray MKV that has PGS subtitle streams, going to a WebM target exhibits this issue as well. This can be mitigated by excluding subtitles with --enc sn.

I just ran into a rare case where an MP4 file claims to have "menus" (this isn't sourced from a Blu-ray or a DVD and no such menus are present, dunno what that's about), which seems to trigger this issue as well. I can't share the file but the last section of running MediaInfo on the file shows the following:

Menu #1
ID                                       : 3
Codec ID                                 : text
Duration                                 : 41 min 19 s
Language                                 : English
Menu For                                 : 1,2
00:00:00.000                             : Chapter 1

Menu #2
00:00:00.000                             : Chapter 1

I managed to solve this issue with --enc dn.

The biggest hassle for me as a user here is that crf-search runs fine but when it comes time to actually encode, the process just runs forever without exiting, just displaying the encode time elapsed and an empty progress bar. You come back to a machine that's been idling for hours instead of working on a queue of encodes. It would be good if ab-av1 recognized if the ffmpeg process exited with an error or a warning, printed FFmpeg's console output to the terminal, and then exited.

[mr44er]

Maybe for reference: https://github.com/alexheretic/ab-av1/issues/129#issuecomment-1516700175