Enable platform decoder for proprietary_codecs flag on Linux

[uazo]

from https://groups.google.com/a/chromium.org/g/chromium-dev/c/uI5knHDhLjE

---------------------------------------
Proprietary codecs
In addition, you may want to include support for proprietary audio and video codecs, 
as Google's WebView does. These codecs may be covered by patents or licensing
agreements, and you should seek legal advice before distributing a build of WebView
which includes them. You can enable them with the following GN arguments:

ffmpeg_branding = "Chrome"
proprietary_codecs = true
---------------------------------------

in cromite is active by default because it is derived from bromite. it is a question of whether the enabling is correct or I might run into legal problems.

[Universalizer]

plus there is one thing I just don't understand: is it possible that I am the one making all this trouble? and everyone else?

Uazo, you are awareful, well-alert (not ignorant) and on ethical / moral path.

[uazo]

@PF4Public (OT) you who know a lot about it, but LGPL 2.1+ (=ffmpeg, statically linked) is compatible with BSD 3 (=chromium)? searching on the subject I find different opinions.

[PF4Public]

@PF4Public (OT) you who know a lot about it, but LGPL 2.1+ (=ffmpeg, statically linked) is compatible with BSD 3 (=chromium)? searching on the subject I find different opinions.

Not surprising that you found differing opinions :) Different components of ffmpeg have different licenses, see their readme: https://github.com/FFmpeg/FFmpeg/blob/master/LICENSE.md. Should you use only LGPL parts, you are free to link even to proprietary binaries. But with GPL parts you should also maintain GPL terms. Your whole resulting binary should be able to be covered under GPL terms. BSD-licensed code fits here just fine. Among GPL requirements are license notices, open access to every bit necessary for the end-user to rebuild your binary: sources, building scripts/procedures etc. All these issues go away if you link dynamically instead (although there might still be caveats and edge-cases).

[Universalizer]

Uazo, curious technical questions :- Exact size of ffmpeg binary library in Chromium ? Which branch :- stable or developer (dev) ?

[uazo]

Exact size of ffmpeg binary library in Chromium ?

Details


Which branch

try searching ffmpeg_revision in https://source.chromium.org/chromium/chromium/src/+/main:DEPS for example: https://source.chromium.org/chromium/chromium/src/+/refs/tags/121.0.6167.164:DEPS;l=492

[uazo]

a few updates. I'm focusing on android for now.

to my surprise all the code needed to exploit the mediacodecs of the s.o. is already present: it is a great luck because all the necessary mojo calls between the rendering process (in sandbox) and the gpu process are already present. I was also able to reroute the code path in such a way that ffmpeg was left to parse the audio config but turn the actual decoding over to java. by the way, there is no control over the android version, so I assume that decoding of aac is always assumed to be present in the various s.o. distributions.

the problem is that it does not work, yet :)

in addition, it seems to me that the decoding of encrypted streams does not work as i imagined it would. i thought that the cdm communicates directly with the output device. however, this does not seem to be the case. i.e., the encrypted stream arrives in the browser, the cdm is asked to decrypt it, and the decrypted stream arrives at the browser, which sends it to the parser and the audio/video decoder. Who knows, I am probably reading the code wrong.

[Universalizer]

ffmpeg was left to parse the audio config but turn the actual decoding over to java.

Logically, may be or due to, Java as a cross-platform language.

---

i.e., the encrypted stream arrives in the browser, the cdm is asked to decrypt it,

Probably, exo player ? Or any other algorithmic communication.

[Universalizer]

Hypothesis ; whether logically transcoding possible ?

Suppose an example, if user is surfing the website through Andoid's Cromite, the example.com site having some .aac & H.264/AVC files inside .mp4 Container, BUT Cromite be code (Instructions) it in such a way, that any closed-sourced video & audio stream further Transcode to any other open-source video & audio streams & open-source Container.

Will these be really possible ?

[uazo]

Will these be really possible ?

no. The problem is that the decoding software is opensource but to distribute it requires a licence for the codec. so you are not allowed to decode that audio if you do not have a licence and therefore cannot transcode it to another format.

[uazo]

but the problem is outdated. I managed to activate the s.o. libraries for decoding aac, for now in android. I have to see if the same logic is also applicable in desktops, by the way, which is quite simple because everything is already there (and it surprised me) once you understand how that very complex code runs.

I am left with the question of availability in the device, although it is written that every device has it.

@romain-hunault I apologise for disturbing you. is this true? as manufacturers of android hardware do you have any obligations? are you obliged to purchase the aac licence?

[Universalizer]

Will these be really possible ?

no. The problem is that the decoding software is opensource but to distribute it requires a licence for the codec. so you are not allowed to decode that audio if you do not have a licence and therefore cannot transcode it to another format.

Ok understood, well explain by you, thanks.

[uazo]

UPDATE: I was able to enable management through the s.o. libraries in windows as well.

there is only one caveat: in windows there is no support for h264 HIGH422PROFILE (h264 high 4:2:2) (example url) Or rather, it is supported (in windows) but does not work (in browser). :( pity. should not be a problem for fingerprinting because

• it would not be supported by any windows device
• it is possible to understand that it lacks support only after play, and since autoplay is disabled the site cannot try it without user intervention.

I also checked the other popular browsers (chrome, vivaldi, brave) and cromite would be the only one using the s.o. aac decoder.

now it's linux's turn. I need also to make a summary table with all codecs active and not, and if it is possible to retrieve some device information. EDIT: ah I also need to see the encoders

[uazo]

ah, other thing. I decided not to use the cisco dll, too dangerous to load libraries not under my control among other things in the gpu process. in fact, i really don't know how to do it for linux, but I am starting to try the solution indicated in https://github.com/uazo/cromite/issues/710#issuecomment-1896579823. @PF4Public is there any way in linux to verify the source of a .so library? Is the path enough?

[uazo]

@PF4Public when you have time and willing, would you do this check for me?

• add "is_component_ffmpeg = true" to gn gen
• do the build

at this point you should get "libffmpeg.so" in the out. Now you should try if, by deleting it, the browser use the libffmpeg.so of the s.o. (as written in https://groups.google.com/a/chromium.org/g/chromium-packagers/c/R5rcZXWxBEQ/m/XzfC2kFQH2gJ) I'll give it a try, too.

by the way, no problem with device fingerpriting, since internally chromium sets an allowlist of ffmpeg codecs to use, so even if you have a library with all codecs, only those officially supported by chromium are currently supported. possibly, the only problem is the security of the library source.

[uazo]

I confirm that it works. I got the .so from https://github.com/nwjs-ffmpeg-prebuilt/nwjs-ffmpeg-prebuilt

alternatively, what if I made a repo that was only used to produce the .so and that the user has to fork and launch the action? EDIT: just have to launch ninja -C out/lin64/ libffmpeg.so

[PF4Public]

@PF4Public is there any way in linux to verify the source of a .so library? Is the path enough?

Sorry, but I don't understand your question. Could you please rephrase it?

@PF4Public when you have time and willing, would you do this check for me?

• add "is_component_ffmpeg = true" to gn gen
• do the build

at this point you should get "libffmpeg.so" in the out. Now you should try if, by deleting it, the browser use the libffmpeg.so of the s.o. (as written in https://groups.google.com/a/chromium.org/g/chromium-packagers/c/R5rcZXWxBEQ/m/XzfC2kFQH2gJ)

Oh, by the way I forgot to tell you that usage of system ffmpeg was made impossible in recent versions of Chromium. They use a modified version of ffmpeg and as such system ffmpeg should be build accordingly. AFAIK only Arch linux does that in official repository. I do too, but it is "unofficial" :) All that means that you cannot rely on system ffmpeg on Linux any longer.

[uazo]

is there any way in linux to verify the source of a .so library? Is the path enough?

Sorry, but I don't understand your question. Could you please rephrase it?

certainly. In Windows, I can enable a sandbox flag that does not allow non-Microsoft libraries to be loaded, or I can check the origin of the dll by checking the digital signature of the file. so, assuming that i do not distribute the libffmpeg.so file but leave it to the user to download (or better, build), what do you think is the best way to ensure that a malicious library is not loaded in linux?

Oh, by the way I forgot to tell you that usage of system ffmpeg was made impossible in recent versions of Chromium

In my opinion, you can bring this to the attention of the chromium team.

[uazo]

another update. Just to complicate matters, I found a new possibility for fingerprinting the device using the media API, but it is still not exploited by tools such as creepjs... I need users' logs... carl absolutely disagreed with this, and he is actually right, but to do my job better I should have them.

[uazo]

They use a modified version of ffmpeg and as such system ffmpeg should be build accordingly.

I checked for linux, it is not true that they use a particular ffmpeg modified version. to what are you referring? This is their ./configure for linux x64

--disable-everything --disable-all --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-static --enable-avcodec --enable-avformat --enable-avutil --enable-static --enable-libopus --disable-debug --disable-bzlib --disable-error-resilience --disable-iconv --disable-network --disable-schannel --disable-sdl2 --disable-symver --disable-xlib --disable-zlib --disable-securetransport --disable-faan --disable-alsa --disable-autodetect --enable-decoder='vorbis,libopus,flac' --enable-decoder='pcm_u8,pcm_s16le,pcm_s24le,pcm_s32le,pcm_f32le,mp3' --enable-decoder='pcm_s16be,pcm_s24be,pcm_mulaw,pcm_alaw' --enable-demuxer='ogg,matroska,wav,flac,mp3,mov' --enable-parser='opus,vorbis,flac,mpegaudio,vp9' --extra-cflags=-I/usr/local/google/home/dalecurtis/code/chrome/src/third_party/opus/src/include --disable-linux-perf --x86asmexe=nasm --optflags='\"-O2\"' --enable-decoder='theora,vp8' --enable-parser='vp3,vp8' --enable-lto --arch=x86_64 --target-os=linux --enable-pic --cc=clang --cxx=clang++ --ld=clang --extra-ldflags='-fuse-ld=lld' --enable-decoder='aac,h264' --enable-demuxer=aac --enable-parser='aac,h264'

I do too, but it is "unofficial"

so do you distribute a version of ffmpeg without the correct licences? or is it only for your internal use? is that correct, can you do it? I ask because if there is a legal way I would do it too. I'm interested because there is no other way to decode aac since there is no hardware decoder for audio in (chromium) linux, so, I must also release that .so file.

[PF4Public]

download (or better, build), what do you think is the best way to ensure that a malicious library is not loaded in linux?

If your library is loaded in runtime (on demand, like LoadLibraryEx and GetProcAddress in Windows), then you could download it on your build machine, checksum it, hardcode it into the binary, and then compare the checksum before each load of the library.

Checking the manually built library is a big problem in it's own. Simplest would be to verify the archive of the sources from which it should be built because making reproducible builds so that you could verify the resulting binary everywhere is a daunting task (dependent on the size of a library of course)

a sandbox flag that does not allow non-Microsoft libraries to be loaded, or I can check the origin of the dll by checking the digital signature of the file

I don't think such a thing exists in Linux, but I'll do a research on the topic and let you know if I can find something.

In my opinion, you can bring this to the attention of the chromium team. I checked for linux, it is not true that they use a particular ffmpeg modified version. to what are you referring?

Compare /libavformat/avformat.h in ffmpeg project and in Chromium: https://github.com/FFmpeg/FFmpeg/blob/3635c1ee97faeb2fb0757a0b158e7de959dfa298/libavformat/avformat.h#L1277-L1281

attribute_deprecated
int64_t    av_stream_get_end_pts(const AVStream *st);
#endif

#define AV_PROGRAM_RUNNING 1

https://source.chromium.org/chromium/chromium/src/+/main:third_party/ffmpeg/libavformat/avformat.h;l=1116-1124?q=av_stream_get_first_dts&ss=chromium%2Fchromium%2Fsrc%3Athird_party%2Fffmpeg%2F

attribute_deprecated
int64_t    av_stream_get_end_pts(const AVStream *st);
#endif

// Chromium: We use the internal field first_dts vvv
int64_t    av_stream_get_first_dts(const AVStream *st);
// Chromium: We use the internal field first_dts ^^^

#define AV_PROGRAM_RUNNING 1

They use internal field, which means that if system library does not provide this field it leads to a runtime failure.

so do you distribute a version of ffmpeg without the correct licences? or is it only for your internal use? is that correct, can you do it?

No, I build this library locally and anyone building ungoogled-chromium or Cromite using my overlay should also do the same locally. In this case I do not distribute anything at all. I also provide pre-compiled ungoogled-chromium, which does not require system ffmpeg, but it has proprietary codecs enabled. I'm not a lawyer and didn't investigate how unlawful/lawful that might be :)

I need users' logs... carl absolutely disagreed with this, and he is actually right, but to do my job better I should have them.

Would creating several virtual machines and gathering logs there suffice? I could do this if it might help you.

[uazo]

checksum it, hardcode it into the binary, and then compare the checksum before each load of the library.

I like this solution, I don't know if I can do it, but I like it: I first have to check whether the build is reproducible so I could put the .so without the proprietary codecs in the official build, and allow the user to replace the library. i think i'll proceed in steps, for linux for now i'll continue as before, today, if i can, i'll publish this patch. yesterday i tested it on browserstack with real devices and it seems to work. I would like to propose adoption to other browsers as well, once I am sure it works.

They use internal field, which means that if system library does not provide this field it leads to a runtime failure.

you're right, I checked it wrong.

I'm not a lawyer and didn't investigate how unlawful/lawful that might be :)

Yeah, same problem as mine.

Would creating several virtual machines and gathering logs there suffice?

unfortunately i think i need physical machines, i.e. not virtual ones: I don't want to make public yet the fingerpriting flaw i found until i solve it, we can discuss it later.

I could do this if it might help you.

thanks, we'll talk again, actually you're already helping me a lot.

[uazo]

for those who want to try, release: https://github.com/uazo/cromite/releases/tag/v122.0.6261.111-6144adc65250e66b02fbd658839dd786fa5bf1ea is the same version of https://github.com/uazo/cromite/releases/tag/v122.0.6261.111-aeb7dce1db09d042991073d6352a1363efeeceee but with the patch of https://github.com/uazo/cromite/pull/856 as a check, you can open chrome://media-internals: kAudioDecoderName and kVideoDecoderName must be not ffmpeg for the aac and h264 codecs. all youtube live videos are in aac format, h264 and HEVC videos can be found, for example, at https://tools.woolyss.com/html5-audio-video-tester

[macchrome]

Platform audio-visual (AV) decoding under Linux including xHE_ACC

https://github.com/macchrome/chromium/releases/tag/v123.6312.127-M123.0.6312.127-r1262506-portable-ungoogled-Lin64

[uazo]

you must supply your own Chromium compatible dynamically loaded FFMpeg libraries

that is exactly the problem. How do you allow the download of a component for which you do not have a licence?

[macchrome]

We have to be pragmatic, unless the desire is that each indiviual user of Chromium compiles it from source,

[uazo]

unless the desire is that each indiviual user of Chromium compiles it from source,

well, that would be nice. the use of my docker images would already allow this. in fact, the idea is to create a docker image for 'in house' build of ffmpeg for chromium. is that there should be 10 of me to do everything! :( but slowly I will get there!

[lpslp]

cromite is definitely missing some codecs.

I didn't create a separate ticket because I can't even name which codecs exactly - but cromite behaves very strangely. Sometimes I get this and this same video opens NORMALLY in chrome and firefox. And even! in palemoon video opens normally, but in cromite - doesn't. (I can't give links to video examples)

but the very fact that even palemoon opens video what cromite can't - is very sad

[PF4Public]

@lpslp If you cannot publish links with failing videos, you can examine the "Media" tab in devtools, to see which codec was expected and wasn't found.

[lpslp]

thanks for advice PF4Public in latest Google Chrome it show it like this

upd i want add some screen from log, maybe it helps to find "what's wrong"

[PF4Public]

It looks like they're trying to use the codec, that is for some reason disabled in Cromite. The "Track" tab should show the actual codec that is used there, for example AV1 or h264.

[lpslp]

The "Track" tab should show

sorry, but i don't know where is it... if you can, describe, where exactly "this" tab located?

[PF4Public]

[lpslp]

those was from Google Chrome...

[PF4Public]

h264 is unlikely to be a problem, but aac… @uazo, is it possible that aac is disabled in Cromite?

[uazo]

chrome://media-internals please use this and send the log.