Closed 1kilogram closed 1 year ago
the MP4Box.js library should work out of the box
That's always a goal, but different people have different use cases and the library may not work out of the box for all use cases. On top of that, bugs can make it harder to use.
the processing of .mp4 files within the MP4Box concept must work without dependence on modern API which in some cases instead of possibilities add limitations
MP4Box.js is independent of the APIs you listed. It can be used with them or not.
I'm closing this issue as I don't see any concrete actions from it. If you have specific bugs or suggestions to improve the code, feel free to submit them and submit patches (this is open source software).
Modification of MP4Box.js development policy
The MP4Box.js library is required to work with binary media data without using (e.g. after segmentation as specified in README) the following time consuming browser APIs: NO
Web Codecs API
, NOMedia Recorder API
, NOMedia Source Extensions API
(never MSE). This development policy must follow because of a commercial mistake by Apple inc. and the cretinism (this is not a nickname, but a diagnosis) of Apple inc. executives.The main problem is that Apple to keep App Store application market, programmatically blocks some features of different API, deliberately breaking W3C specifications.
Take the great example that led me to ffmpeg.js, and later to MP4Box.js:
So we have:
Web Audio API
, which is available fromiOS6+
,cHr14+
,fFox25+
that is for devices from 2011 - 2013 onwards (updates). The idea is: the ability to send voice messages without usinggetUserMedia API
(available only withiOS 11+
,cHr53+
,fFox36+
starting from 2017 release/update)How to do it:
So according to the Web Audio API specification this is quite a simple procedure, which is done with the function
audioContext.decodeAudioData()
. This works everywhere exceptiOS16+
and will never work because of the cretinism of Apple inc management. I'm not saying Apple's management is bad, as as there are things to learn from Google (Chrome, Firefox), for example unlike Google, Apple follows backward compatibility principles.For example Application Cache (iOS 3.2+) is available on modern devices iOS16+, not only via HTTPS, but also via HTTP protocol! Which is very convenient, for example, for artists, who have nothing to hide and no time to do programming (add the file "map-files.cache" and everything works).
So,
audioContext.decodeAudioData()
on iOS is programmatically blocked, and only for video files. Safari handles audio files normally, just like Chrome or FireFox.We look for a workaround and find it:
audioContext.createScriptProcessor()
, which, in the first tests immediately proved itself and recorded an "oscillator"audioContext.createOscillator()
via "offline Audio Context". And it all worked on iOS! But... When trying to record audio with video: nothing happens.I'm sure there's a trigger on iOS that blocks this kind of action with video, otherwise it opened up a huge opportunity, which would have made a lot of apps just not have the App Store, as they would have been based in the public domain for all devices a long time ago.
Brilliant solution:
FFmpeg.js
andMP4Box.js
The Web Audio API study took place between October 22 and October 31 of this year. And then on November 6, I came across ffmpeg.js. Imagine my surprise when I experimented with ffmpeg.js and found out that the task of separating audio track from 10-minute FullHD video amounts to less than 2 seconds!
Yes, we go out of our way to save the audio in a
.mp4
container, but we have an advantage in this action:mp4
container is already compressed and doesn't require any transcoding to compress tomp3
and the difference in their sizes is negligible.iOS
which does not read all kinds of audio, can read the .mp4 container (iOS 4+
,cHr 4+
,fFox 3.5+
)This is the solution. Brilliant solution, but there is only one catch: ffmpeg.js library takes 26 mb and will take very, very long to download on a 3g connection. This is a powerful tool that takes a lot of memory, and despite running even on
iOS9+
does not perform its tasks, asSafari
just collapsed from the attached file of 26 mb! But oniOS11+
everything works fine. WithWeb Worker
you can unload a worker process, which allows you to free up memory if the user doesn't need ffmpeg.js features anymore.Let's look at MSE (
Media Source Extensions API
) yes, following Google's policy - everything works fine, but the cretins (diagnosis, not calling them names!) from Apple have distinguished themselves again, quote:The elementary mass of tasks that iOS does in ffmpeg.js in a matter of minutes and literally seconds (if the solution is intelligently put) all of this is simply not available to the iOS MSE API! How is that possible? Obviously, this is a commercial mistake of Apple inc. which can be bypassed after all...
And the best option is MP4Box.js! library which I accidentally came across only on November 10! MP4Box.js is a mechanism which has huge advantages over ffmpeg.js:
Regarding iOS specifics: translating
.MOV
container to.MP4
container is a lightweight procedure which takes less than 2kb of code and needsFileReader API
andBlob
andUint8Array
(iOS6+
). This is a feature ofiOS
and does not apply to other systems, for example Android usually records camera video to anmp4
file and we do not need to translate it because we assume that the media file from which you want to remove video and leave the audio will be in a.mov
or.mp4
container in this case.Now I hope I've convinced the MP4Box.js community and people involved in development that
we need to make the MP4Box.js development policy progressive:
out of the box
[ ] the processing of
.mp4
files within the MP4Box concept must work without dependence on modern API which in some cases instead of possibilities add limitations:Regards, @1kilogram.