FLAC StreamDecoder unable to decode file when FileDecoder does

[dhensby]

I'm experiencing a problem with a set of FLAC files I'm attempting to decode using the StreamDecoder class. I am able successfully decode other FLAC files, so this does work in some circumstances.

Do you have any idea what might be going on to cause a Readable being piped to the StreamDecoder could result in an error when playing directly from a file would not?

The following code results in an error being emitted:

const { StreamDecoder } = require('flac-bindings');
const { createReadStream } = require('fs');

const filePath = '...';

const decoder = new StreamDecoder();
const fileStream = createReadStream(filePath);

decoder.on('error', (error) => {
  console.error(error);
  process.exit(1);
});

fileStream.pipe(decoder);
decoder.pipe(process.stdout);

Output:

Trace: Decoder stream got empty: that is an issue
    at FlacStreamDecoder._readCbk (./node_modules/flac-bindings/lib/decoder.js:131:17)
{ Error: FLAC__STREAM_DECODER_ABORTED
    at FlacStreamDecoder._emitDecoderError (./node_modules/flac-bindings/lib/decoder.js:157:26)
    at FlacStreamDecoder._transform (./node_modules/flac-bindings/lib/decoder.js:70:33) code: 7 }

Process finished with exit code 1

With the FileDecoder, however, it works as expected:

const { FileDecoder } = require('flac-bindings');

const filePath = '...';

const decoder = new FileDecoder({ file: filePath });

decoder.on('error', (error) => {
  console.error(error);
  process.exit(1);
});

decoder.pipe(process.stdout);

Output:

[binary data]

Process finished with exit code 0

[melchor629]

Hi, thanks for raising this issue. It is indeed something I did not expect to happen never, but it happened so I now must fix it, sorry for that :(

I will try to reproduce it somehow (I don't know yet). I will keep updated the issue with the progress.

[dhensby]

@melchor629 - I can send you a flac that breaks privately, it's copyrighted music so I can't share it publicly.

In the meantime I've refactored my code to use the FileDecoder instead.

Thanks a lot for the library!

[melchor629]

Hi, luckily, I found some flacs on my computer that also breaks everything using the same code as yours. They are flacs with 24bit 96KHz and 192KHz hehe.

I uploaded a quick fix for this and, for me, seems to be working properly but I want to test it a bit further. If not today, then tomorrow, I will create a new version with the fix.

Thanks to you by helping improving this library :D

[dhensby]

Great news - I also noticed the same files did actually crash the FileDecoder; some of the files just appear to be truncated (end early) but some also result in node crashing with this error:

FATAL ERROR: Error::New napi_get_last_error_info
1: 0x10003ae75 node::Abort() [~/.nvm/versions/node/v10.14.1/bin/node]
2: 0x10003b07f node::OnFatalError(char const*, char const*) [~/.nvm/versions/node/v10.14.1/bin/node]
3: 0x10003b035 node::OnFatalError(char const*, char const*) [~/.nvm/versions/node/v10.14.1/bin/node]
4: 0x1000433a3 napi_create_function [~/.nvm/versions/node/v10.14.1/bin/node]
5: 0x10a720e3a  [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
6: 0x10a726196 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
7: 0x10a724298 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
8: 0x10a721a1f flac_bindings::AsyncDecoderWork::onProgress(flac_bindings::DecoderWorkContext const*, Napi::Env&, flac_bindings::AsyncBackgroundTask<int, flac_bindings::DecoderWorkRequest*>::ExecutionProgress&, flac_bindings::DecoderWorkRequest* const*, unsigned long) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
9: 0x10a725cc7 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
10: 0x10a725ac2 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
11: 0x10a72814e Napi::ThreadSafeFunction::CallJS(napi_env__*, napi_value__*, void*, void*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
12: 0x10004b93c (anonymous namespace)::v8impl::ThreadSafeFunction::IdleCb(uv_idle_s*) [~/.nvm/versions/node/v10.14.1/bin/node]
13: 0x1009a683d uv__run_idle [~/.nvm/versions/node/v10.14.1/bin/node]
14: 0x1009a17eb uv_run [~/.nvm/versions/node/v10.14.1/bin/node]
15: 0x100041f5d node::Start(v8::Isolate*, node::IsolateData*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) [~/.nvm/versions/node/v10.14.1/bin/node]
16: 0x10004138a node::Start(uv_loop_s*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) [~/.nvm/versions/node/v10.14.1/bin/node]
17: 0x100040e6b node::Start(int, char**) [~/.nvm/versions/node/v10.14.1/bin/node]
18: 0x100001034 start [~/.nvm/versions/node/v10.14.1/bin/node]
19: 0x3 

[dhensby]

Another update; I installed your current dev branch (npm i melchor629/node-flac-bindings#dev) and I'm still seeing the same errors.

the log from my flac encoding is this (if it helps) - and yes, it's old:

Exact Audio Copy V1.0 beta 1 from 15. November 2010

EAC extraction logfile from 28. March 2011, 17:49

Used drive  : hp      BDDVDRW CA21N   Adapter: 0  ID: 1

Read mode               : Secure
Utilize accurate stream : Yes
Defeat audio cache      : Yes
Make use of C2 pointers : No

Read offset correction                      : 102
Overread into Lead-In and Lead-Out          : No
Fill up missing offset samples with silence : Yes
Delete leading and trailing silent blocks   : No
Null samples used in CRC calculations       : Yes
Used interface                              : Native Win32 interface for Win NT & 2000
Gap handling                                : Appended to previous track

Used output format              : User Defined Encoder
Selected bitrate                : 128 kBit/s
Quality                         : High
Add ID3 tag                     : No
Command line compressor         : C:\Program Files (x86)\FLAC\flac.exe
Additional command line options : -8 -V -T "ARTIST=%a" -T "TITLE=%t" -T "ALBUM=%g" -T "DATE=%y" -T "TRACKNUMBER=%n" -T "TOTALTRACKS=%x" -T "GENRE=%m" -T "ALBUMARTIST=%v" -T "ALBUM ARTIST=%v" %s

[melchor629]

Hello, the crash is something I found while testing this the last day, so I hope it won't happen anymore in the next version. For the other hand, as you already found, the fix did not fix the issue. To fix this I will need some time. The StreamDecoder implementation is the worst of all to properly implement so... Thanks for all the information :)

Hope I can find a proper solution for this soon.

[dhensby]

Thanks for your effort on this! 🙏

[melchor629]

Hello again, I uploaded an initial (and better fix) for this. I've tested on almost all my flac library and it seems to be pretty solid. BUT, there is one scenario where the this stream decoder could fail and I would like to investigate further (but it is difficult to reproduce - for me it is sometimes even in the same file). If you would like to try it, I pushed the change a couple minutes ago.

If I cannot manage to fix the other scenario soon, I will probably publish a new version with the known issue in the notes. At least better than before (I hope).

[dhensby]

OK - good news is that the stream decoder now starts playing the songs; before it errored before starting.

Bad news is I now get the same error as I do with the FileDecoder:

FATAL ERROR: Error::New napi_get_last_error_info
1: 0x10003ae75 node::Abort() [~/.nvm/versions/node/v10.14.1/bin/node]
2: 0x10003b07f node::OnFatalError(char const*, char const*) [~/.nvm/versions/node/v10.14.1/bin/node]
3: 0x10003b035 node::OnFatalError(char const*, char const*) [~/.nvm/versions/node/v10.14.1/bin/node]
4: 0x1000433a3 napi_create_function [~/.nvm/versions/node/v10.14.1/bin/node]
5: 0x10a5cde3a  [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
6: 0x10a5d3196 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
7: 0x10a5d1298 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
8: 0x10a5cea1f flac_bindings::AsyncDecoderWork::onProgress(flac_bindings::DecoderWorkContext const*, Napi::Env&, flac_bindings::AsyncBackgroundTask<int, flac_bindings::DecoderWorkRequest*>::ExecutionProgress&, flac_bindings::DecoderWorkRequest* const*, unsigned long) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
9: 0x10a5d2cc7 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
10: 0x10a5d2ac2 flac_bindings::AsyncDecoderWork::forInitOggFile(Napi::Object const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, flac_bindings::DecoderWorkContext*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
11: 0x10a5d514e Napi::ThreadSafeFunction::CallJS(napi_env__*, napi_value__*, void*, void*) [~/projects/dhensby/node-music/node_modules/flac-bindings/build/Release/flac-bindings.node]
12: 0x10004b93c (anonymous namespace)::v8impl::ThreadSafeFunction::IdleCb(uv_idle_s*) [~/.nvm/versions/node/v10.14.1/bin/node]
13: 0x1009a683d uv__run_idle [~/.nvm/versions/node/v10.14.1/bin/node]
14: 0x1009a17eb uv_run [~/.nvm/versions/node/v10.14.1/bin/node]
15: 0x100041f5d node::Start(v8::Isolate*, node::IsolateData*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) [~/.nvm/versions/node/v10.14.1/bin/node]
16: 0x10004138a node::Start(uv_loop_s*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) [~/.nvm/versions/node/v10.14.1/bin/node]
17: 0x100040e6b node::Start(int, char**) [~/.nvm/versions/node/v10.14.1/bin/node]
18: 0x100001034 start [~/.nvm/versions/node/v10.14.1/bin/node]
19: 0x3 

[melchor629]

That's weird... :/ Did you try to run something like npm rebuild flac-bindings (or npm rebuild), or reinstall the package (removing it and installing again). Maybe there is something cached. I remember to have fixed this already.

[dhensby]

I've just made sure and done a rebuild (which I probably should have done before).

Both the StreamDecoder and the FileDecoder crash with the same error.

Further, when I pipe to node-speaker (which is my real life use-case) I'm seeing a constantly repeated log of [../deps/mpg123/src/output/coreaudio.c:81] warning: Didn't have any audio data in callback (buffer underflow) for both FileDecoder and StreamDecoder. I appreciate this is an external library so not something you may be able to control.

[melchor629]

I've tried to replicate the issue on macOS and Windows with node 10.21.0 and 12.18.0 with no success (it never crashed) :(

If you could prepare a simple test scenario with a flac file and send it privately (email for example), would be really nice.

About node-speaker it seems this is something related to the package itself, but I can quickly checkout if this is "my fault".

[melchor629]

Quick update related to node-speaker: installed both speaker and flac-bindings (using dev branch) and played one song each run. No warnings printed, but sometimes speaker crashed :/

The script:

const { StreamDecoder } = require('flac-bindings')
const { createReadStream } = require('fs')
const Speaker = require('speaker')

const filePath = '...'

const inputFile = createReadStream(filePath)
const decodedFlac = new StreamDecoder()
const speaker = new Speaker({ channels: 2, bitDepth: 16, sampleRate: 44100 })

inputFile.pipe(decodedFlac)
decodedFlac.pipe(speaker)

package.json

{
  "name": "yeah",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "flac-bindings": "git+https://github.com/melchor629/node-flac-bindings.git#dev",
    "speaker": "^0.5.2"
  }
}

[dhensby]

Sorry for the delay, this is my test code which can be executed via cli like so: node index.js [path-to-flac]

const { createReadStream } = require('fs');
const { StreamDecoder, api: { format: { MetadataType: { STREAMINFO } } } } = require('flac-bindings');
const Speaker = require('speaker');

const [node, path, file] = process.argv;

(() => {
  const stream = createReadStream(file);
  const decoder = new StreamDecoder();
  decoder.on('metadata', (metadata) => {
    if (metadata.type === STREAMINFO) {
      speaker = new Speaker({
        channels: metadata.channels,
        sampleRate: metadata.sampleRate,
        bitDepth: metadata.bitsPerSample,
      });
      decoder.pipe(speaker);
    }
  });
  stream.pipe(decoder);
})();

I still get the error - I will email you the problematic file.

[melchor629]

Hi, I've been testing a bit on macOS and Windows (both node 12.18) and no problems so far (ignoring the corrupted files thing, which I detailed you in the email).

On macOS, node-speaker has some issues that can crash node (see https://github.com/TooTallNate/node-speaker/issues/92).

If there is nothing more to tackle around this issue, feel free to close the issue. I will publish a new version with this fix :)

Thanks for raising the issue and helping with everything to get it fixed.

[dhensby]

OK, great. Let's close this as the files I've got are corrupted so clearly the library shouldn't be expected to support them.

Thanks for all your help on this.

[melchor629]

Thanks to you providing all the information and feedback for fixing it :D

New version is on the go, in an hour or so will be published in npm.