Add mp3 encoding and/or decoding support

[Wallacoloo]

edited @tresf, added checklist

The MP3 patents have finally expired. Since LAME is available on all platforms, LAME will be used and bundled to provide this functionality.

• [x] Export song to mp3 via GUI (#3615)
• [x] Export song to mp3 via CLI(#3641)
• [x] Import tracks as mp3 via SampleTrack, AudioFileProcessor(#6750)

---

The discussion on #1471 recently led to some talk over the possibility of supporting the mp3 file format (specifically in decoding samples, but it also has use as an export option).

For those that don't know, there are a variety of patents associated with encoding and decoding mp3's. In order to do either option, you must obtain a license. These patents are all scheduled to expire in the U.S. sometime before 2018 (the last one expires 30/Dec/2017), according to this source. Other sources (liftarn's comment) indicate that September 2015 is the month that mp3-related patents expire in the U.S. However, the patents have already expired in most other countries, and may have never applied to some countries.

LAME

Now, there is a non-commercial (LGPL-licensed) mp3 encoder/decoder, LAME. According to Wikipedia, distributing LAME in compiled form along with LMMS would likely be in violation of the US patents. A popular open source tool, Audacity, is able to provide mp3 encoding through LAME by detecting its presence on the system at runtime instead of shipping precompiled binaries. They explain the reasoning briefly at the top of this page. This appears to remove responsibility from the developers of Audacity over patent licensing, instead pushing that responsibility to the users that are affected (i.e. U.S. users who download LAME). Additionally, that same page says that Thomson (the patent enforcer) permits mp3 encoding for "personal" use, which a decent portion of LMMS users do.

So if we want mp3 support before the patents expire, we could go with a similar approach to Audacity if all the developers are comfortable with it. I sure am not a lawyer, so I can only judge based upon prior projects, of which Audacity is a relevant one to reference.

Gstreamer

Another option to supporting mp3's is through gstreamer. gstreamer is essentially a plugin-based audio routing library. As such, it provides plugins for both encoding and decoding a variety of audio formats, including mp3.

Historically, I think Gstreamer has been more linux-oriented, as it was adopted by the gnome project fairly early. But it claims to support OS X and Windows plus mobile.

I wish I knew more about this option. All I can say is that it seems like a more generalized approach to encoding and decoding that happens to support mp3's.

---

Anyways, feel free to discuss your opinions or concerns about mp3 support in lmms, whether you think it's worth adding now or best to wait for the patents to completely expire, and which implementation approach might be the best. Also, clarifications on the expiration dates or legal issues are much appreciated if you have that knowledge.

[tresf]

Some general questions....

Since the majority of proprietary OSs already have mp3 playback and encoding support (i.e. ripping mp3s with iTunes, Windows Media Player, playback) is licensing an issue on those platforms that already have the capabilities?

Furthermore are there any methods that provide fallback on an existing API?

Last, how much is unlimited licensing? Could we kickstart the licensing costs to cover ourself legally? (Does the paid license allow the use of a 3rd party implementation such as LAME?)

[softrabbit]

http://gstreamer.freedesktop.org/documentation/plugins.html, looks like Gstreamer uses LAME for encoding mp3, and libmad for decoding. Both listed as "ugly", i.e. distribution might pose problems, license- or patentwise. So, adding gstreamer would be quite a lot of stuff just to end up using LAME.

Last, how much is unlimited licensing?

US$ 15k/year minimum, $0.75/unit for decoding, $2.50 to have it both ways. http://mp3licensing.com/royalty/software.html

[musikBear]

Honestly.. is it worth the hazzle /hazard afaik ogg has better sound quality than mp3 -right? Anyone that want to use mp3, can just export to wave, and then use like.. audacity to convert. Imho lmms could use founds much more creative, and gain more quality, than pouring mulahh in fraunhofers money-bin - (not that Fraunhofer is a bad thing, isent, and that is also worth a thought, before 'circumfising' there rights

[curlymorphic]

15k a year minimum, is alot, If we added this feature, we would then have the added stress or raising this money every year,

I am not sure kickstarter would raise that sort of money just to add mp3 to an already existing project. I would not expect kickstarter to raise this year after year for an already implemented feature.

A thought that I just had, was to offer an option on export to open the .wav file in a 3rd party program (ie audacity) where the user can do the encoding. Ive not looked into it, but maybe it's possible to do this on import aswell.

As for importing mp3's into a Daw, I personally feel this is not good practice to encourage the use of lossy compressed audio, the encoding artifacts seem to stack up when multiple samples are used, maybe its good to make users think "why have I got to convert this this mp3 before i can use it". I know it is possible to get decent quality mp3 at 320K, but most are not, generally 320k.

I guess what I am saying is, thats a lot of money we have to find.

The 3rd party route may be possible, but I would like to see some real legal advise ( not something someone once said in a forum etc). LMMS afaik is not a limited company, So it could come down to individual responsibility. I would hate to see anyone here having to go thru a costly legal battle.

With 1500 download per day, we are far to big a project to "chance it". From what I have seen we are one of the larger open source audio projects, we are not under the radar and would be a prime target to make example of.

On a more positive note, If we can do this properly, I feel it would be great to have as mp3 is a very common and popular format, and we are somewhat lagging behind on this.

[tresf]

Thanks for the licensing research... I think the vast majority of our proprietary OS users are already licensed for decoding and encoding as the licensing costs are covered by Microsoft, Apple.

libaudiodecoder

Here's a project which claims to leverage the licensed versions of the libraries already on the OS (originally part of the mixxx project):

https://github.com/asantoni/libaudiodecoder/

The license is MIT, so I think we'd likely need special permission to re-release it as GPL.

Allegro

Here's another project which may have hooks to the correct DirectShow libraries on Linux, created primarily for game development:

http://alleg.sourceforge.net/

The license is a "do what you want" license, so we should have the ability to re-license it. They recommend a "pay it forward" or "giftware" usage, which we're not bound to, but we'd likely comply with by patching stuff upstream if we were to use it.

Windows DirectShow

Or we roll our own GPL version of the same libary... Here's a C++ example for Windows to use the built-in MP3 decoder library, assuming mingw has support (there's a good chance mingw does not have DirectShow support)

using dshow.h http://stackoverflow.com/questions/8121570/use-windows-built-in-mp3-decoder-to-play-audio/8121768#8121768

irrKlang

This one is just bazaar. It is very cheap to use and they have options for "free", "cheap" and "hobbyist" projects, but I highly doubt they'd offer up a GPL version of the code.

http://www.ambiera.com/irrklang/irrklang_pro.html

[Wallacoloo]

The option of opening a wave in audacity (or another program) sounds relatively easy to implement & could be useful for more than just conversions. I think licensing mp3 for lmms is definitely out of the question and frankly just not worth anywhere near that amount of money.

But I really like @tresf's idea of using the OS licenses if that turns out to be possible. MIT is far more permissive than GPL, so I do believe it's ok to use it in a GPL codebase (would have to doublecheck this though). IMO, it's a shame that lmms doesn't have a more permissive license, but that's a different discussion. On Apr 22, 2015 5:38 AM, "Tres Finocchiaro" notifications@github.com wrote:

Thanks for the licensing research... I think the vast majority of our proprietary OS users are already licensed for decoding and encoding as the licensing costs are covered by Microsoft, Apple.

libaudiodecoder

Here's a project which claims to leverage the licensed versions of the libraries already on the OS (originally part of the mixxx project):

https://github.com/asantoni/libaudiodecoder/

The license is MIT, so I think we'd likely need special permission to re-release it as GPL.

Allegro

Here's another project which may have hooks to the correct DirectShow libraries on Linux, created primarily for game development:

http://alleg.sourceforge.net/

The license is a "do what you want" license, so we should have the ability to re-license it. They recommend a "pay it forward" or "giftware" usage, which we're not bound to, but we'd likely comply with by patching stuff upstream if we were to use it.

Windows DirectShow

Or we roll our own GPL version of the same libary... Here's a C++ example for Windows to use the built-in MP3 decoder library, assuming mingw has support

using dshow.h

http://stackoverflow.com/questions/8121570/use-windows-built-in-mp3-decoder-to-play-audio/8121768#8121768

irrKlang

This one is just bazaar. It is very cheap to use and they have options for "free", "cheap" and "hobbyist" projects, but I highly doubt they'd offer up a GPL version of the code.

http://www.ambiera.com/irrklang/irrklang_pro.html

— Reply to this email directly or view it on GitHub https://github.com/LMMS/lmms/issues/2000#issuecomment-95159024.

[mikobuntu]

How about a command line equivalent instead of having to use a GUI program such as Audacity. Openshot video editor uses Blender in the background ( i think through python scripts ) for rendering of animated titles. The problem would perhaps be making this cross platform?

http://sox.sourceforge.net/soxformat.html << Sox looks interesting, but the original problem of damn mp3 patent is obviously still there. Same deps tho as many others.

[Wallacoloo]

@mikobuntu great idea. It's pretty easy to execute something of the form " [args ...]" in a cross platform manner, and we can use % operators to let the user substitute the file name, etc. We could also do the same for decoding mp3's. Though it does limit that functionality to more advanced users. We may be able to provide guides in our wiki like audacity does in its wiki though (I think that's safe since mp3 decoding is legal in most countries) On Apr 22, 2015 9:53 AM, "Mikobuntu" notifications@github.com wrote:

How about a command line equivalent instead of having to use a GUI program such as Audacity. Openshot video editor uses Blender in the background ( i think through python scripts ) for rendering of animated titles. The problem would perhaps be making this cross platform?

— Reply to this email directly or view it on GitHub https://github.com/LMMS/lmms/issues/2000#issuecomment-95262780.

[tresf]

@mikobuntu great idea. It's pretty easy to execute something of the form " [args ...]" in a cross platform manner, and we can use % operators to let the user substitute the file name, etc.

Unless we're doing batch conversion, I strongly disagree with this approach, at least for MP3 features topic... Here's why:

1. There's no easy cross-platform way of knowing where the software is installed, so we'll get into a mess when locating and launching.
2. There's no common command-line utility on these OSs for MP3 encoding/decoding.
3. Audacity only has support for MP3 encoding when LAME is installed (in that case, why wouldn't we just use LAME directly and avoid the unnecessary dependency?)
4. Would we constantly be doing on-the-fly MP3 --> WAV rendering for sample playback? Would we just write to the temp folder?

I just think it's a last-ditch effort to use the CLI to render something and I don't think this approach should be considered as it sounds like a Rube-Goldberg approach to a very simple problem.

But I really like @tresf's idea of using the OS licenses if that turns out to be possible. MIT is far more permissive than GPL, so I do believe it's ok to use it in a GPL codebase (would have to doublecheck this though).

I looked into this a bit more and it appears you are right @Wallacoloo and they are compatible, that the overwhelming response I'm reading. :+1:

I do fear that the code links against Microsoft Headers (rather than MingW headers) which would be a snag when building windows binaries on Ubuntu.

[softrabbit]

@tresf, keep digging :+1:

libaudiodecoder: Only Windows and OSX, not Linux. A solution covering at least those 3 would be better. Still probably the best out of this bunch.

Allegro: One mention of mp3 on their site, in a code snippet demonstrating how to add your own formats... not very promising. http://alleg.sourceforge.net/stabledocs/en/alleg026.html#register_sample_file_type

Windows DirectShow: Even less cross-platform than libaudiodecoder, right?

irrKlang, http://www.ambiera.com/irrklang/features.html:

MP3 playback is done via a plugin which comes along with irrKlang (and its full source code) so that it is easily possible to remove mp3 support if wished. If you are using mp3 playback in your application, you might need to purchase an mp3 license from Thompson Multimedia

[tresf]

@softrabbit here's something I read from a libaudiodecoder dev per https://plus.google.com/+RJRyan/posts/5QgqTsjJqXA.

patches welcome :). For +Mixxx DJ Software at least, on Linux the situation is easier because we can just depend directly on the actual decoders and since the distro ships the decoder we don't have liability. Windows / OS X are where the patent nightmares lie for us.

Although I don't have the capacity to code this all in, I agree with this statement. Linux MP3 isn't really the problem as the user has to intentionally install the Bad/Ugly codecs which the OSs put up plenty of warnings against and it would be something we offer support for, but don't ship with in terms of potentially licensed binaries.

[liftarn]

What about making a US version (without MP3) and a rest-of-the-world version (with MP3)?

[Wallacoloo]

@tresf

@mikobuntu great idea. It's pretty easy to execute something of the form " [args ...]" in a cross platform manner, and we can use % operators to let the user substitute the file name, etc.

Unless we're doing batch conversion, I strongly disagree with this approach, at least for MP3 features topic...

The advantage of the "cli" approach is that it frees the developers of most responsibility for users breaking their local laws. That is, since such a feature wouldn't be targeted specifically at mp3 encoders or decoders, we are less liable if it gets used for illegal mp3 encoding.

It's kind of like if a c++ compiler compiles a binary that is used to bypass DRM, the compiler writers aren't held accountable. But if the compiler specifically exposes functions to bypass DRM to programmers, then the compiler writers could be accountable. Similarly, if we just launch a user-defined program with user-defined arguments, we probably wouldn't be held accountable if the user uses that feature to aid them in encoding mp3s illegally. OTOH, if we specifically support LAME as a plugin, we might have some accountability. But of course, I am not a lawyer.

[Wallacoloo]

libaudiodecoder looks like a good solution in terms of avoiding liabilities to me though.

[Reaper10]

My vote is for Gstreamer.

[zonkmachine]

FL Studio use LAME!?

[jasp00]

How about libmad or libmpeg3 as decoder and LAME as encoder?

[zonkmachine]

How about libmad...

Cooler name = My vote.

[Umcaruje]

I think this issue can be worked on now. The patents for mp3 have expired: https://www.iis.fraunhofer.de/en/ff/amm/prod/audiocodec/audiocodecs/mp3.html

On April 23, 2017, Technicolor's mp3 licensing program for certain mp3 related patents and software of Technicolor and Fraunhofer IIS has been terminated.

So we're now completely free to use mp3 in LMMS

[tresf]

Well... technically we've always been free to use mp3 in LMMS. Now we are potentially free to bundle mp3 libraries too.

From wikipedia:

"Technicolor claims to control MP3 licensing of the Layer 3 patents in many countries, including the United States, Japan, Canada and EU countries."

"The basic MP3 decoding and encoding technology is patent-free in the European Union, all patents having expired there by 2012 at the latest. "

"In the United States, the technology became substantially patent-free on 16 April 2017"

This is all good news. We should be able to offer LAME support on all platforms quite easily.

[michaelgregorius]

I have found a CMake find script for lame here:

# - Try to find LAME
# Once done this will define
#
# LAME_FOUND - system has liblame
# LAME_INCLUDE_DIRS - the liblame include directory
# LAME_LIBRARIES - The liblame libraries

find_path(LAME_INCLUDE_DIRS lame/lame.h)
find_library(LAME_LIBRARIES mp3lame)

include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Lame DEFAULT_MSG LAME_INCLUDE_DIRS LAME_LIBRARIES)

list(APPEND LAME_DEFINITIONS -DHAVE_LIBMP3LAME=1)

mark_as_advanced(LAME_INCLUDE_DIRS LAME_LIBRARIES LAME_DEFINITIONS)

I guess it would be integrated quite similar to the Ogg Vorbis support.

[michaelgregorius]

And here's a simple example on how to use the library: https://stackoverflow.com/questions/2495420/is-there-any-lame-c-wraper-simplifier-working-on-linux-mac-and-win-from-pure/2496831#2496831.

[michaelgregorius]

Implemented with #3615 for the export via the GUI. Export via the command line is not supported yet and might be covered by a new issue once #3615 is merged. MP3 import is also not covered by #3615 and should also go into a new issue.

For now MP3 support is disabled for the Windows and OS X build because I assume that Lame will first have to be added to these environments.

Edit: Added the info about the Windows and OS X build.

[tresf]

For now MP3 support is disabled for the Windows and OS X build because I assume that Lame will first have to be added to these environments.

brew install lame should do the trick for Mac. If we keep consistent with Windows, we'll need @tobydox to add lame to his ppa for mingw to fulfil the dependency on Windows. Regardless, it shouldn't be disabled, just built without support, similar to if libogg was missing, etc.

[michaelgregorius]

@tresf The windows build is not disabled completely, it's just always building without MP3 support. I guess once @tobydox has added lame to the PPA we can simply adjust the CMakeLists.txt and the corresponding Travis script for Windows as was done for OS X (969ade26b9652b71d4435c2d78deb0c53e526d8c) and then all three platforms will support MP3 encoding via Lame.

[tresf]

@tresf The windows build is not disabled completely, it's just always building without MP3 support.

I know, I'm asking you to stop doing that. Cmake is smart enough to know if it's there. Forcing off is pointless.

[tobydox]

mingw[32|64]-x-lame is now available at https://launchpad.net/~tobydox/+archive/ubuntu/mingw-x-trusty - might take a few minutes until package has been published

[michaelgregorius]

Command line export is implemented via #3641.

[michaelgregorius]

I have checked the second box for "Export song to mp3 via CLI" because #3641 has been integrated into stable-1.2.

[gi0e5b06]

Done. Using libmpg123. I chose that lib because it offers 32f output, contrary to libmad (and because LAME is somehow obsolete for decoding).

[musikBear]

@gi0e5b06 when you say 'done', you are referring to export -right? Because if import is done, mp3 need to be added to selectable files in the select for import in both AFP and Sample-track, witch both look like this now:

[PhysSong]

@musikBear What @gi0e5b06 says is he/she have implemented the import feature on the fork(gi0e5b06/lmms). The commit isn't in original repository(LMMS/lmms) now and it won't be unless someone(maybe I?) cleanup the code and open a pull request.

FYI, I'm willing to write my own code for importing mp3, if @gi0e5b06's one turns out to be inappropriate.

[PhysSong]

https://github.com/lieff/minimp3 looks like another good candidate.

[PhysSong]

https://github.com/lieff/minimp3 looks like another good candidate.

It looks the library is suitable for decoding, but it can't parse ID3 metadata tags. It supports skipping ID3 tags while detecting MP3 frames, though.

One more news: libsndfile (optionally)supports MP3 encoding/decoding as of 1.1.0beta1 via libmp3lame for encoding and libmpg123 for decoding.

[indigotechtutorials]

It would be great to get import support for mp3 nowadays most audio files are in mp3 format and it would make it a much better experience

[musikBear]

It would be great to get import support for mp3 nowadays most audio files are in mp3 format and it would make it a much better experience

No! MP3 is a sc Lossy format. It means that if you use MP3 as sample, you do not get the full dynamic sound, you get a 'subset' of that sound, because MP3 is compressed! Then when you export in MP3, you have a compression of a compression, and then you actually can hear the quality-drop. MP3 is fine for export. Especially if you upload on one of the popular hosts, like youTube, soundcloud or Spotify /other soMe' -because all those type of portals will always compress your upload to MP3 ..128bit afair..., so there a high-quality render in wave-format wont help, but for real master render, wave or ogg is the one to use, and always wave or ogg files for samples used in your projects. MP3 is not suited at all!

[allejok96]

It looks the library is suitable for decoding, but it can't parse ID3 metadata tags

@PhysSong why do we need ID3 parsing for import?

[indigotechtutorials]

I understand that other file formats are better but mp3 is the most used format and supporting it in lmms would make it easier to use and help new producers have better experiences. When I first was making beats on LMMS I would get samples from packs and alot of them are in mp3 and also downloading samples from YouTube.

[Rossmaxx]

could you add #6750 here?

[pozhiloy-enotik]

Hello. This thingy with 7k stars still has no mp3?

[zonkmachine]

Hello. This thingy with 7k stars still has no mp3?

It's in the works. https://github.com/LMMS/lmms/pull/6750