Support for AAC in M4A (MP4) container

[thwaller]

This is a feature request.... could support for the M4A file (commonly called an iTunes audio file) be added? The format is not a proprietary Apple format, so at least that is not a concern.

[d3592182]

I'm familiar with m4a from audiobooks, the format has a chapterization feature, which works well in VLC and "Voice" (open source android audiobook app).

M4A appears to be MP4 that only contains audio: https://en.wikipedia.org/wiki/MPEG-4_Part_14#.MP4_versus_.M4A

According to https://en.wikipedia.org/wiki/Comparison_of_video_container_formats , the MP4 format has patent issues:

I'm afraid I don't know the details on that. Perhaps/hopefully the Wikipedia page is wrong.

[thwaller]

The m4b file is the "proper" extension for audio books. The m4a, yes, is nothing but a mp4 file containing only audio. In my experiences, you can name it m4a or mp4 and have no consequence. It is my understand that, in general, Apple is behind all of the m4x extensions, which are all essentially mp4 files.

Regarding patents, I am unsure of the impact of that, so it is hard to respond. There are plenty of softwares that are open source that encode AAC, save to the MP4 container, read and write MP4 metadata. The MP4 is an open format, which I assume is also important.

[peripatetes]

Here's a guess at the patent situation as of 2019. More detail would require an actual thorough patent search.

• Patents on AAC-LC ought to have expired by now, since it was published in 1997.
• I'm unsure about the MP4 container format. I don't see any patents filed after the original 1999 Apple spec, and I don't know that there has ever been a royalty/licensing program for just the container. But the revision of the container in use wasn't published as a standard until 2003, so there could be issues.
• HE-AAC and HE-AACv2 are definitely still under patent for the next 4+ years, and VIA Licensing is actively collecting royalties for these.
• The latest version of AAC, xHE-AAC, is heavily patented and those patents won't expire for 12+ years.

Unlike e.g. ffmpeg, I think people have relied on libsndfile being fully legal to use without a patent license. Supporting LC-AAC in MP4 could be legal in the near term, but would inevitably lead to people wondering why their AAC files won't decode (answer: because they're HE-AAC v1 or v2).

[peripatetes]

An update to this feature request since it's been four years:

• AAC-LC, which is the most common variety, has been patent-free for six years now. (AAC-LC has also lost less ground to newer formats than HE-AACv1 and v2, which were always only useful at low bitrates.)
• The MP4 container format version 2, standardized 2003, is also patent-free now.
• Red Hat shipped an AAC-LC encoder and encoder they deemed to be patent-free, fdk-aac-free, in 2017, and no negative fallout occurred. MP4 container support has also been shipping in free software.
• HE-AAC v1 decoding is also patent free as of this year (IANAL but standards were published 2003 and published patent lists show things expiring earlier this year). Encoding HE-AACv1 is more likely to still be encumbered right now, but it looks to me like the last patent expires early next year.
• HE-AACv2 is likely patent encumbered till 2026, and xHE-AAC is likely encumbered till ~2031. But again, these files are much much less common, and anyone encoding a file today is not well served by HE-AAC v1 or v2 (use either Opus or AAC-LC or xHE-AAC).
• AAC-LC has solidified its place as the defacto baseline for lossy audio, displacing MP3. Yes, Opus is superior and patent-unencumbered, and in 2017 it was still new-ish. But Apple refused to add support for Opus file playback, and patent trolls are causing some things that used Opus to drop their support.
• Getting the right library for encoding may still be an obstacle. The Android library FDK-AAC is definitely superior to other open source AAC encoders, but it includes encoding support for versions of AAC which are still patented. Red Hat stripped out everything but AAC-LC for fdk-aac-free. FAAC 1.3 may be OK to use; it should at least not be worse than MP3, while libavcodec's aacenc may be worse.

[ProgramComputer]

https://stackoverflow.com/questions/29565068/mp4-file-format-specification - a starter thread to read on the mp4 container format Bento4 or mp42 could be used as a library.

@arthurt Can mpeg.c be used for mp4?

[pjg864]

An update to this feature request since it's been four years:

• AAC-LC, which is the most common variety, has been patent-free for six years now. (AAC-LC has also lost less ground to newer formats than HE-AACv1 and v2, which were always only useful at low bitrates.)
• The MP4 container format version 2, standardized 2003, is also patent-free now.
• Red Hat shipped an AAC-LC encoder and encoder they deemed to be patent-free, fdk-aac-free, in 2017, and no negative fallout occurred. MP4 container support has also been shipping in free software.
• HE-AAC v1 decoding is also patent free as of this year (IANAL but standards were published 2003 and published patent lists show things expiring earlier this year). Encoding HE-AACv1 is more likely to still be encumbered right now, but it looks to me like the last patent expires early next year.
• HE-AACv2 is likely patent encumbered till 2026, and xHE-AAC is likely encumbered till ~2031. But again, these files are much much less common, and anyone encoding a file today is not well served by HE-AAC v1 or v2 (use either Opus or AAC-LC or xHE-AAC).
• AAC-LC has solidified its place as the defacto baseline for lossy audio, displacing MP3. Yes, Opus is superior and patent-unencumbered, and in 2017 it was still new-ish. But Apple refused to add support for Opus file playback, and patent trolls are causing some things that used Opus to drop their support.
• Getting the right library for encoding may still be an obstacle. The Android library FDK-AAC is definitely superior to other open source AAC encoders, but it includes encoding support for versions of AAC which are still patented. Red Hat stripped out everything but AAC-LC for fdk-aac-free. FAAC 1.3 may be OK to use; it should at least not be worse than MP3, while libavcodec's aacenc may be worse.

@peripatetes Could you help me to check if AAC-LC is patent free to use in TS(transportStream) and RTMP stream?

[peripatetes]

@pjg864 sorry for the late response. I'm not a lawyer and my knowledge about these things basically comes from a few minutes' web search rather than any IP law expertise. But I don't see a reason why the patent situation for AAC-in-TS or AAC-in-RTMP would differ from simply the patents for the codec plus the patents for the container/protocol; combining the existing two things is likely not novel/patentable in these cases. TS-related patents have expired; the last ones to expire may well have been in 2018, and it should certainly be safe by now. Wikipedia suggests that RTMP is still patented until at least April 2025; Adobe grants a patent license to all implementations, but it comes with some restrictions and you'd need to figure out what that means for your case.

[sunflsks]

Would it be feasible to add support just for the AAC-LC codec (as that is both the most-used AAC version and the one which is certainly patent-free)? If so, it might be a good idea to look into the ffmpeg libavcodec encoder (it seems to be the best free one). If not, maybe we could pull a Red Hat and use the Fraunhofer library but only with support for the codecs that are no longer patented? I'd be willing to take this forward.