Some tracks open just fine, but appear to yield no samples during decoding

[compenguy]

I wrote a pandora cli client, and recently Pandora seems to have stopped supplying mp3 URLs for tracks, so I switched to their AacPlus mp4 streams, which I'm using redlux with rodio to decode.

A fair number (maybe half?) of tracks I get from pandora work just fine, I initialize an MP4 decoder, no errors reported, and playback is great.

The other ~half open just fine with the mp4 decoder, but then immediately terminate - as far as I can tell never having played a sample.

To make sure it wasn't something I was doing wrong on my end, I cloned the redlux repo, copied one of these problem tracks into the tests/samples/ folder, updated the m4a test to point to my track instead (extended the sleep time to 2 seconds, and set the volume to 1.0), and ran cargo test. I hear 2 seconds of silence for my track, but when pointing it at your track I hear 2 seconds of audio.

This same track plays fine with vlc.

Comparing track metadata, all the failing ones have this metadata attached to them: ©too=Lavf58.20.100 or ©too=Lavf58.45.100.

This suggests to me that the failing tracks were (trans/en)coded using libavformat versions 58.20.100 (ffmpeg 4.1.6) or 58.45.100 (ffmpeg 4.3.2). The encoder for the working tracks is not known, but isn't listed.

Re-encoding your original test sample using ffmpeg -i Simbai\ \&\ Elke\ Bay\ -\ Energy.m4a Simbai\ \&\ Elke\ Bay\ -\ Energy_2.m4a did not render it unplayable, and following similar steps to re-encode my failing track made it work, so I'm guessing there's a particular encoding option that impacting this.

Beyond this point, I don't know what to look for or how to look for it, but I'm happy to help however I can.

[probablykasper]

Hmm, that's pretty odd. Could you provide the full metadata of the file?

If it's an issue with how it's encoded, I'm guessing it's an issue with fdk-aac, which redlux uses to decode audio samples. To be sure, it would make sense to try to somehow use the fdk-aac bindings directly.

Might be useful to know if it's actually playing any samples. Inside the test, you could try the Sink's sleep_until_end method instead of thread::sleep. If it's not playing any samples, I assume it ends instantly.

[compenguy]

I added the sleep_until_end - it's definitely dropping it instantly.

I added some logging statements to the decode_next_sample() function, and picked up this:

2021-04-03 13:16:38:018646175 [ERROR] <redlux:156>:decode frame error: DecoderError { code: 4098, message: "The input buffer ran out of bits." }
2021-04-03 13:16:38:037397975 [ERROR] <redlux:156>:decode frame error: DecoderError { code: 4098, message: "The input buffer ran out of bits." }
2021-04-03 13:16:38:037502986 [ERROR] <redlux:240>:Decoder error: TrackReadingError

I added a bit more instrumentation, and here's the line that it's erroring out on:

let object_type = track.audio_profile().or(Err(Error::TrackReadingError))?;

Un-squelching the error from mp4 yields:

Err(InvalidData("invalid audio object type"))

I patched mp4-rust to be aware of the audio profile value that wasn't supported, and it's the value for SBR, so I did some local modifications to patch mp4-rust and redlux's adts.rs to make it support that value as well.

Now I can trace the execution just past the call to self.aac_decoder.fill() to self.aac_decoder.decode_frame(), and the input buffer is still running out of bits.

[probablykasper]

Nice find. My first thought was that SBR wasn't supported by fdk-aac, but I believe it is.

From the fdk-aac decoder PDF, these seem to be the supported audio formats:

• AAC-LC Low-Complexity
• HE-AAC v1 High-Efficiency
• HE-AAC v2 High-Efficiency
• AAC-LD Low-Delay
• AAC-ELD v2 Enhanced Low-Delay

Based on the MPEG-4 Wikipedia article's list of audio object types, SBR seems to count as HE-AAC v1, so I don't know why your solution wouldn't work.

Either way, out of the formats supported by fdk-aac, rust-mp4 (and consequently redlux) only supports AAC-LC. Opened issue https://github.com/alfg/mp4-rust/issues/51 for that.

[compenguy]

Alright - I found was I was doing wrong. I was coercing SBR (5) and PS (29) object types to LTP (4) for the ADTS header instead of AAC LC (2). Once I did that everything worked.

You can see my patch here: https://github.com/probablykasper/redlux/compare/master...compenguy:sbr-support

I have validated using my own test file (not added to the repo) on your tests that it decodes the frames, and I've validated by pointing my pandora client to build against this branch, and it has played every track so far just fine.

I only added support for SBR and PS types because I don't know if there are valid type coercions for the ADTS header, and if there are, what they would be.

As soon as there's an mp4-rust release with his fix, I'll send my work as a pull request.

[probablykasper]

The fix is out in v0.3.0