Fails to index FLAC audio file

[rectalogic]

I generate a short FLAC audio file in NUT format using ffmpeg (file is attached):

ffmpeg -f lavfi -t 1 -i flite=text=hello:voice=rms -f nut -codec:a flac -sample_fmt s16 -ac 2 -ar 44100 -y audio.nut

audio.nut.zip

FFMS_DoIndexing2 fails on this with Audio decoding error Sample code:

#include <ffms.h>
#include <stdio.h>

int main(int argc, char* argv[])
{
    /* Initialize the library. */
    FFMS_Init(0, 0);

    char errmsg[1024];
    FFMS_ErrorInfo errinfo;
    errinfo.Buffer = errmsg;
    errinfo.BufferSize = sizeof(errmsg);
    errinfo.ErrorType = FFMS_ERROR_SUCCESS;
    errinfo.SubType = FFMS_ERROR_SUCCESS;
    const char* sourcefile = argv[1];

    FFMS_Indexer* indexer = FFMS_CreateIndexer(sourcefile, &errinfo);
    if (indexer == NULL) {
        fprintf(stderr, "FFMS_CreateIndexer error: %s\n", errinfo.Buffer);
        return 1;
    }
    FFMS_TrackTypeIndexSettings(indexer, FFMS_TYPE_VIDEO, 1, 0);
    FFMS_TrackTypeIndexSettings(indexer, FFMS_TYPE_AUDIO, 1, 0);

    FFMS_Index* index = FFMS_DoIndexing2(indexer, FFMS_IEH_ABORT, &errinfo);
    if (index == NULL) {
        fprintf(stderr, "FFMS_DoIndexing2 error: %s\n", errinfo.Buffer);
        return 1;
    }

    FFMS_Deinit();

    return 0;
}

ffprobe shows this, and ffplay is able to play it:

$ ffprobe -hide_banner audio.nut
Input #0, nut, from 'audio.nut':
  Metadata:
    encoder         : Lavf58.76.100
  Duration: 00:00:00.81, start: 0.000000, bitrate: 169 kb/s
  Stream #0:0: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16
    Metadata:
      encoder         : Lavc58.134.100 flac

(it would be useful if FFMS2 included the underlying AVERROR code in the FFMS_ErrorInfo for any errors originating in ffmpeg)

[rectalogic]

In lldb, avcodec_send_packet is returning -22.

* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 4.1
    frame #0: 0x0000000100511ae8 libffms2.4.dylib`FFMS_Indexer::IndexAudioPacket(this=0x0000600003826c70, Track=0, Packet=0x000000016fdfec50, Context=0x0000600000a28000, TrackIndices=0x0000600001f2c000) at indexing.cpp:270:13
   267      int Ret = avcodec_send_packet(CodecContext, Packet);
   268      if (Ret != 0) {
   269          if (ErrorHandling == FFMS_IEH_ABORT) {
-> 270              throw FFMS_Exception(FFMS_ERROR_CODEC, FFMS_ERROR_DECODING, "Audio decoding error");
   271          } else if (ErrorHandling == FFMS_IEH_CLEAR_TRACK) {
   272              TrackIndices[Track].clear();
   273              IndexMask.erase(Track);
Target 0: (ffaudio) stopped.
(lldb) p Ret
(int) -22
(lldb) p *Packet
(AVPacket) {
  buf = 0x0000600000a3c040
  pts = 35721
  dts = 35721
  data = 0x0000600001f34000 ""
  size = 0
  stream_index = 0
  flags = 1
  side_data = nullptr
  side_data_elems = 0
  duration = 0
  pos = 17102
  convergence_duration = 0
}
(lldb) p *Packet->buf
(AVBufferRef)  (buffer = 0x0000600000430030, data = "", size = 64)

Hmm, so maybe it's this: https://github.com/FFmpeg/FFmpeg/blob/71fb6132637a2a430375c24afc381fff8b854fe7/libavcodec/decode.c#L600-L601

We have AVPacket.data set but AVPacket.size is 0.

[qyot27]

The error is specifically FLAC-in-NUT, not FLAC (and not NUT on its own, either). Which would point to either

A) The mux is bad, and therefore the NUT muxer in FFmpeg needs fixing (and ffmpeg being able to detect it 'properly' could simply be in spite of the problem). B) The mux is actually fine, but there's some additional element of the demuxing step(s) that FFMS2 needs to account for.

If you take the generated NUT file and demux the audio stream, the unencapsulated FLAC file is completely fine. It indexes and plays back through FFMS2 without a problem.

[rectalogic]

Interesting, yes I get the same error using the ffmpeg demux/decode sample code:

...
audio_frame n:0 nb_samples:4608 pts:0
audio_frame n:1 nb_samples:4608 pts:0.10449
audio_frame n:2 nb_samples:4608 pts:0.20898
audio_frame n:3 nb_samples:4608 pts:0.313469
audio_frame n:4 nb_samples:4608 pts:0.417959
audio_frame n:5 nb_samples:4608 pts:0.522449
audio_frame n:6 nb_samples:4608 pts:0.626939
audio_frame n:7 nb_samples:3465 pts:0.731429
Error submitting a packet for decoding (Invalid argument)

https://github.com/FFmpeg/FFmpeg/blob/n4.4.4/doc/examples/demuxing_decoding.c

[rectalogic]

And even though ffplay can play it, ffmpeg gets an error transcoding it:

$ ffmpeg -hide_banner -i audio.nut -c:a flac -f nut -y audio-reencode.nut
Input #0, nut, from 'audio.nut':
  Metadata:
    encoder         : Lavf58.76.100
  Duration: 00:00:00.81, start: 0.000000, bitrate: 169 kb/s
  Stream #0:0: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16
    Metadata:
      encoder         : Lavc58.134.100 flac
Stream mapping:
  Stream #0:0 -> #0:0 (flac (native) -> flac (native))
Press [q] to stop, [?] for help
Error while decoding stream #0:0: Invalid argument
Output #0, nut, to 'audio-reencode.nut':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16, 128 kb/s
    Metadata:
      encoder         : Lavc58.134.100 flac
size=      17kB time=00:00:00.81 bitrate= 169.2kbits/s speed= 388x
video:0kB audio:16kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.085941%

[rectalogic]

Filed https://trac.ffmpeg.org/ticket/10849#ticket

[rectalogic]

I git bisected ffmpeg and these lines were added which is what fixes this issue: https://github.com/FFmpeg/FFmpeg/blob/748f2999eee82fd507da327fa57e47730f5effc1/fftools/ffmpeg_dec.c#L426-L430

It seems like this was kind of a hack to workaround an issue that wasn't understood, and a similar issue occurs with nut/flac files too. So I guess any demuxer/decoder would need a similar hack until/if the underlying issue is fixed in ffmpeg.

[myrsloik]

As you've figured out yourself it's an upstream issue. We don't plan on adding a workaround in FFMS2.