processing / processing-sound

Audio library for Processing built with JSyn
https://processing.org/reference/libraries/sound/
GNU Lesser General Public License v2.1
149 stars 50 forks source link

Most MP3 files with meta-information fail to load #32

Closed MLanghof closed 1 year ago

MLanghof commented 5 years ago

14 describes much of the same issue, but the fix for that apparently didn't resolve it completely.

This is what I get when trying to open the following files as SoundFile:

https://drive.google.com/file/d/1cNWH7bISxTRyLBPKQEA8jmgoWJSiHxSM/view?usp=sharing

london.mp3:

Caused by: java.lang.NullPointerException
    at fr.delthas.javamp3.Decoder.samples_III(Decoder.java:625)
    at fr.delthas.javamp3.Decoder.decodeFrame(Decoder.java:546)
    at fr.delthas.javamp3.Sound.decodeFullyInto(Sound.java:228)
    at processing.sound.SoundFile.<init>(Unknown Source)

close.mp3:

Caused by: java.lang.ArrayIndexOutOfBoundsException: 16
    at fr.delthas.javamp3.Decoder.samples_I(Decoder.java:1506)
    at fr.delthas.javamp3.Decoder.decodeFrame(Decoder.java:517)
    at fr.delthas.javamp3.Decoder.init(Decoder.java:431)
    at fr.delthas.javamp3.Sound.<init>(Sound.java:49)
    at processing.sound.SoundFile.<init>(Unknown Source)

noLove.mp3:

Caused by: java.lang.ArrayIndexOutOfBoundsException: 3
    at fr.delthas.javamp3.Decoder.decodeFrame(Decoder.java:474)
    at fr.delthas.javamp3.Decoder.init(Decoder.java:431)
    at fr.delthas.javamp3.Sound.<init>(Sound.java:49)
    at processing.sound.SoundFile.<init>(Unknown Source)

The last two look suspiciously like lookups in some of the MPEG header tables (where usually the last value is illegal). All of the files have some sort of meta-information (I presume ID3 tags) preceding the real MP3 data.

The files without meta-information that I've tried seem to load fine. There are also some with meta-information that do load (presumably those where we are lucky to have no "bad" bit patterns that look like [invalid] headers in the ID3 tags).

Overall, about 80% of the files in my music library run into one of the above errors when trying to load them. I don't know whether the bug is with sound or some underlying library, but the frame parsing needs to be way more robust.

FWIW, I had extremely similar problems when I assumed that every 4 byte chunk of the byte stream that starts with the MP3 sync word (12 set bits) will be a valid header - sometimes they're not, and sometimes the bit pattern happens to indicate an invalid index into the MP3 info lookup tables. Once you "lock into" the series of headers/frames that shouldn't happen any longer but there is 0 guarantee about what is contained in the meta-info surrounding the real MP3 frames.

And for completeness: Sometimes a file may simply stop inside an MP3 frame (or there may be ID3 tag garbage after all the frames, causing similar issues to above). This file is missing a byte at the end (I didn't craft it that way, again just another file from my music library) and crashes the decoder because it's trying to read past the end:

toccata.zip

MLanghof commented 5 years ago

For the record, this happens both with the 2.1.0 version that comes with processing and the 2.2.0 test version.

kevinstadler commented 5 years ago

This is most likely a constraint of the underlying mp3 library that currently lacks support for some container formats, see https://github.com/kevinstadler/javamp3#Status -- I've only made minor changes to the fork and don't actually understand the subject matter enough to expand mp3 support myself. With the new version of the sound library we were a bit hard pressed to find freely available pure-Java mp3 decoder libraries, if you are aware of alternative libraries we could probably integrate them quite easily.

In the meantime our general recommendation is to use WAV files, since MP3 decoding is slow and in the end all samples are stored as raw uncompressed data in RAM anyway, the decrease in disk storage is a minor advantage in comparison.

zach-v commented 4 years ago

Ah, this would explain why I've been having issues with some of the mp3 files I have trying to load. I've been able to recreate the problems of loading with entirely new files exported from FL Studio specifically. When I export with audacity lame3 it works just fine.

Here's the error from the FL Studio exported file: Dec 12, 2019 4:37:19 AM com.jsyn.devices.javasound.JavaSoundAudioDevice <init> INFO: JSyn: default output latency set to 80 msec for Windows 10

Sound library error: unable to decode sound file <insert arbitrary file location>\sound.mp3 Action happens to play sound. Sound library error: rate needs to be positive java.lang.NullPointerException at processing.sound.AudioSample.frames(Unknown Source) at processing.sound.SoundFile.frames(Unknown Source) at processing.sound.AudioSample.playInternal(Unknown Source) at processing.sound.AudioSample.play(Unknown Source) at processing.sound.SoundFile.play(Unknown Source) at processing.sound.AudioSample.play(Unknown Source) at processing.sound.SoundFile.play(Unknown Source)

This is obviously if I go through and try to play the sound file that was not loaded. Like I said earlier that everything works fine with audacity rendered files with lame. Any fl studio rendered files do not work no matter the settings used (I tried various bit rates, and compression samples).

Im not the strongest with audio files, but I am willing to help resolve this error any way that I can if possible.

kevinstadler commented 4 years ago

As described in my comment above, the problem is that the mp3 decoder library we are using will sometimes stumble over files that have large header metadata (particularly when binary data such as a cover photo are encoded in the file). Switching to a different decoder library would solve the problem, the only limitation is that it should be a Java-based library (to keep the library portable across platforms) and its license should be compatible with the Processing one. Any suggestions are welcome.

LichtMarv commented 4 years ago

I had the same issue. I noticed the only difference between a working file and a broken one was, that one was stereo and the other mono. After converting the mono one to stereo it worked just fine.