Latest Widevine 4.10.2252.0 fails to load

[wagnerch]

It appears Google may have changed the latest Widevine libraries, the size has changed by about 1M, and it is not linking all of the same libraries as it used to (perhaps some stuff is statically linked now). In any case these are the errors, but there isn't enough logging to know the issue. Also the dlerror() message is squashed and never bubbles up.

2021-05-09 20:29:53.661 T:4807 ERROR : AddOnLog: inputstream.adaptive: Unable to load widevine shared library (/storage/.kodi/cdm/libwidevinecdm.so) 2021-05-09 20:29:53.661 T:4807 ERROR : AddOnLog: inputstream.adaptive: OpenDRMSystem failed

Related issues: https://github.com/CastagnaIT/plugin.video.netflix/issues/1154

Working CDM:

13505.111.0/libwidevinecdm.so: linux-vdso.so.1 (0xbefbd000) /usr/lib/libarmmem-v7l.so (0xb6808000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0xb67df000) libm.so.6 => /usr/lib/libm.so.6 (0xb6776000) libdl.so.2 => /usr/lib/libdl.so.2 (0xb6763000) librt.so.1 => /usr/lib/librt.so.1 (0xb674c000) libnss3.so => /usr/lib/libnss3.so (0xb664f000) libnssutil3.so => /usr/lib/libnssutil3.so (0xb661c000) libnspr4.so => /usr/lib/libnspr4.so (0xb65e5000) libc.so.6 => /usr/lib/libc.so.6 (0xb64a6000) /usr/lib/ld-linux-armhf.so.3 (0xb6f66000) libplc4.so => /usr/lib/libplc4.so (0xb6f8f000) libplds4.so => /usr/lib/libplds4.so (0xb6f8b000)

Non-working CDM:

13816.64.0/libwidevinecdm.so: linux-vdso.so.1 (0xbedb9000) /usr/lib/libarmmem-v7l.so (0xb5dd7000) libdl.so.2 => /usr/lib/libdl.so.2 (0xb5dc4000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0xb5d9b000) librt.so.1 => /usr/lib/librt.so.1 (0xb5d84000) libm.so.6 => /usr/lib/libm.so.6 (0xb5d1b000) libc.so.6 => /usr/lib/libc.so.6 (0xb5bdc000) /usr/lib/ld-linux-armhf.so.3 (0xb6f78000)

[wagnerch]

ARM, LibreELEC

[dagwieers]

This affects all ARM-based Linux systems (not Android systems). And what is worse, it appears Google will be revoking older Widevine CDM versions as of 2021-05-31, so we have 3 weeks to get a fix out, or all Widevine-encoded content will stop working. https://www.widevine.com/news

[wagnerch]

This appears to be the dlerror: cannot allocate memory in static TLS block

At least with testing a simple C program to just do a dlopen:

#include <stdio.h>
#include <dlfcn.h>

int main(int argc, char **argv)
{
    void *dl = dlopen("libwidevinecdm.so", RTLD_LAZY);
    if (!dl) {
        char *msg = dlerror();
        fprintf(stderr, "ERROR: %s\n", msg);
    }
}

[matthuisman]

Do we have access to 4.10.2252.0 for other platforms? I know 4.10.2209.0 works fine on Windows and Linux x86_64 for example. See Linux and Chrome OS list different versions... So maybe 4.10.2252.0 is just for Chrome OS and is now more closely linked - that is where IA helper extracts it from after all.

https://dl.google.com/widevine-cdm/versions.txt also lists 4.10.2209.0/1 as latest

If thats the case, it would mean we need to find a new source for a Linux ARM build of Widevine blob. Raspberry Pi has a version, so be interested to see if that gets updated... Did anyone manage to get more info from them where they were getting it from?

[wagnerch]

I think libwidevinecdm0 comes from Ventz. It's not fully clear whether it is an official Google agreement. My guess is they may be pulling it from the same sources.

https://blog.vpetkov.net/2020/03/30/raspberry-pi-netflix-one-line-easy-install-along-with-hulu-amazon-prime-disney-plus-hbo-spotify-pandora-and-many-others/

[matthuisman]

i dont think they would take it from there to be honest. They are quite big org and seem to do things "by the book". Ventz was probably pulling it from Chrome OS as well - that seems to be the only actual source.

Just tried using my Widevine pull script, and still no Linux ARM builds - only Darwin ARM64 and the others.

I wonder if those Chrome blobs are using newer version of one of the linked libs or something like that?

[wagnerch]

I know it's not a special build for Raspberry Pi, it has the same sha1 as the one pulled from ChromeOS. But that may not really indicate much of anything.

[matthuisman]

just had a thought over herE: https://github.com/xbmc/inputstream.adaptive/pull/679#issuecomment-837356073

We are sure the blobs aren't AARCH64 now? Is the HW they pulling from 32 or 64 bit? Maybe latest Chrome OS has switched fully to 64 bit. We did just get a ARM64 build for Darwin / MAC... so wouldn't be huge surprise Chrome OS followed

[wagnerch]

They are not, it's still armhf. You can tell because it has a dependency on the armhf dynamic linker: /usr/lib/ld-linux-armhf.so.3 (0xb6fb3000)

If you compile with: arm-linux-gnueabihf-gcc-8 -Wall -o wvdl wvdl.c -ldl -Wl,--no-as-needed ~/libtcmalloc.so.4.5.3

The last 2 arguments are forcing a link to libtcmalloc.so.4 even though it is not needed by the C program, and run with LD_DEBUG=all you see it gets past linking and calls init in libwidevinecdm and results in a segmentation fault. My guess is Chromium is linking libtcmalloc.so.4, so the symbols are already loaded and TLS is already allocated, and malloc() has been replaced at this point before they dlopen(libwidevinecdm.so). Don't know though. I noticed there are 4 new global symbols: 0080efc0 g D google_malloc 00000000 Base .protected start_google_malloc 0080e688 g D malloc_hook 00000000 Base .protected __start_malloc_hook 00812c70 g D google_malloc 00000000 Base .protected stop_google_malloc 0080efbc g D malloc_hook 00000000 Base .protected __stop_malloc_hook

These are related gperftools (which includes tcmalloc). Not really an expert on all of this dynamic linker & ELF binary stuff. But could't really go much further than that.

LD_DEBUG=all LD_LIBRARY_PATH=/storage:/storage/.kodi/userdata/addon_data/script.module.inputstreamhelper/backup/13816.64.0/ ./wvdl

      3705:     calling init: /usr/lib/librt.so.1
      3705:
      3705:
      3705:     calling init: /storage/.kodi/userdata/addon_data/script.module.inputstreamhelper/backup/13816.64.0/libwidevinecdm.so
      3705:
Segmentation fault

[matthuisman]

thanks for confirming :)

[wagnerch]

BTW, this is what 1679 looks like through the same program (through the point where the segfault happens):

      3715:
      3715:     calling init: /usr/lib/librt.so.1
      3715:
      3715:
      3715:     calling init: /storage/.kodi/userdata/addon_data/script.module.i
nputstreamhelper/backup/13505.111.0/libwidevinecdm.so
      3715:
      3715:     opening file=/storage/.kodi/userdata/addon_data/script.module.in
putstreamhelper/backup/13505.111.0/libwidevinecdm.so [0]; direct_opencount=1
      3715:
      3715:
      3715:     calling fini: /storage/.kodi/userdata/addon_data/script.module.i
nputstreamhelper/backup/13505.111.0/libwidevinecdm.so [0]
      3715:
      3715:
      3715:     calling fini: /usr/lib/libnss3.so [0]

[matthuisman]

anyone got a copy of the new blob that they can upload and send me? Nevermind - i've just downloaded the chrome os :)

[matthuisman]

so, opening that new blob in text editor, there is all the below text in it near the top that older versions dont have are we sure this blob is good?? has anyone got a chrome book to try? Has google messed up?

Error text from top of blob (just opening with notepad): https://pastebin.com/raw/KDHfY8QL

[wagnerch]

Did you get this sha1?

4d129768d3753328a837a940e6ee903830c791c4 libwidevinecdm.so

I downloaded 2 separate ChromeOS images and got the same sha1, and it does dynamically link if the binary doing the dlopen is linked with tcmalloc, just crashes.

No Chromebook here.

[matthuisman]

did you see all the error text... pretty sure that shouldnt be inside the blob... https://pastebin.com/raw/KDHfY8QL

I think it's a bad blob... maybe Google is pushing updates via Chrome once it's booted? Or they have messed up?

We need to confirm with a chrome OS user if it works... If yes - get them to pull the blob and we can compare it... if its different then chrome are pushing an update to it

[wagnerch]

Yeah, possibly. Or it could be what you said earlier, "maybe 4.10.2252.0 is just for Chrome OS and is now more closely linked".

bits of that blob are from: https://github.com/webrtc-uwp/chromium-base/blob/master/trace_event/memory_infra_background_whitelist.cc

[wagnerch]

Well, this is interesting, 2252 has a .init section (which is where I am crashing), 1679 does not. In that .init section it calls InitializeCdmModule_4, which is problematic because CdmAdapter normally does that.

Disassembly of section .init:

0080e674 <_init@@Base>:
  80e674:       e92d4008        push    {r3, lr}
  80e678:       ebefbc00        bl      3fd680 <InitializeCdmModule_4@@Base-0xe0>
  80e67c:       e8bd8008        pop     {r3, pc}

Disassembly of section .fini:

0080e680 <_fini@@Base>:
  80e680:       e92d4008        push    {r3, lr}
  80e684:       e8bd8008        pop     {r3, pc}

[mediaminister]

2252 has a .init section (which is where I am crashing), 1679 does not.

This doesn't make sense to me:

BTW, this is what 1679 looks like through the same program (through the point where the segfault happens):

  3715:
  3715:     calling init: /usr/lib/librt.so.1

[matthuisman]

below posted by Fuzzard on Slack

" i think your libwidevine.so issue is going to be because we dont use (and therefore link) some new/changed libs namely boringssl and tcmalloc

    81: 00000000009d4600     0 NOTYPE  GLOBAL PROTECTED   17 __start_google_malloc
    82: 00000000009d3e70     0 NOTYPE  GLOBAL PROTECTED   16 __start_malloc_hook
    83: 00000000009d8730     0 NOTYPE  GLOBAL PROTECTED   17 __stop_google_malloc
    84: 00000000009d45d3     0 NOTYPE  GLOBAL PROTECTED   16 __stop_malloc_hook

   152: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND OPENSSL_memory_alloc
   153: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND OPENSSL_memory_free
   154: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND OPENSSL_memory_get_size
   155: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND sdallocx"

"

below is me again:

The error we get when trying to load the lib is "cannot allocate memory in static TLS block" after a google, seems similar fix for other issues is export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libX.so before executing. so it does seem that error is related to not having the libraries linked. I assume both boringssl and tcmalloc are available and loaded in Chrome OS

@CastagnaIT on slack suggested trying the widevine blob with chromium browser on RPI and see if that works. seems like a good idea and be interested in the results. I can try that tomorrow of no-one beats me to do.

[wagnerch]

Here is the chrome dependencies from the 138614 image:

$ readelf -d chrome

Dynamic section at offset 0x87e2ee8 contains 55 entries:
  Tag        Type                         Name/Value
 0x0000000f (RPATH)                      Library rpath: [$ORIGIN]
 0x00000001 (NEEDED)                     Shared library: [libdl.so.2]
 0x00000001 (NEEDED)                     Shared library: [libpthread.so.0]
 0x00000001 (NEEDED)                     Shared library: [librt.so.1]
 0x00000001 (NEEDED)                     Shared library: [libsmime3.so]
 0x00000001 (NEEDED)                     Shared library: [libnss3.so]
 0x00000001 (NEEDED)                     Shared library: [libnssutil3.so]
 0x00000001 (NEEDED)                     Shared library: [libnspr4.so]
 0x00000001 (NEEDED)                     Shared library: [libcups.so.2]
 0x00000001 (NEEDED)                     Shared library: [libdbus-1.so.3]
 0x00000001 (NEEDED)                     Shared library: [libfontconfig.so.1]
 0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 0x00000001 (NEEDED)                     Shared library: [libexpat.so.1]
 0x00000001 (NEEDED)                     Shared library: [libdrm.so.2]
 0x00000001 (NEEDED)                     Shared library: [libsync.so]
 0x00000001 (NEEDED)                     Shared library: [libxkbcommon.so.0]
 0x00000001 (NEEDED)                     Shared library: [libgbm.so.1]
 0x00000001 (NEEDED)                     Shared library: [libevdev.so]
 0x00000001 (NEEDED)                     Shared library: [libgestures.so.0]
 0x00000001 (NEEDED)                     Shared library: [libasound.so.2]
 0x00000001 (NEEDED)                     Shared library: [libcras.so.0]
 0x00000001 (NEEDED)                     Shared library: [libassistant.so]
 0x00000001 (NEEDED)                     Shared library: [libffi.so.6]
 0x00000001 (NEEDED)                     Shared library: [libc++.so.1]
 0x00000001 (NEEDED)                     Shared library: [libc++abi.so.1]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
 0x00000001 (NEEDED)                     Shared library: [ld-linux-armhf.so.3]

Wondering if boringssl & tcmalloc is statically linked in chrome?

[dagwieers]

I can confirm that the PHASER version 13816.64.0 image (also reported as 90.13816.0 or ChromeOS 90.0.4430.100) works fine on a Chromebook using Widevine on Netflix.

[dagwieers]

@matthuisman If I understand the error correctly, it means some loaded library is deplating the preallocated TLS area. So that must be one of the libraries loaded prior to the error.

There could be a few reasons for this:

• The preallocated TLS area is now smaller than before (not likely)
• Other stuff is using the TLS area that wasn't using it before
  • Could be an external library that is now loaded and wasn't before (not likely)
  • Could be the Widevine CDM itself that is using more
  • Could be that due to threading, a larger number of threads are using the same size, thus more

The solution could be to:

• Increase the preallocated size of the TLS area (is that possible?)
• Reduce the existing use of the TLS area (this is what is done in similar cases on Google)

And a workaround that was mentioned is to preload the library complaining so it has sufficient TLS area when loaded.

Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1722181#c20

[dagwieers]

On my x64 laptop, I can see libwidevinecdm.so itself is using a much larger TLS area now (96 bytes instead of 24 bytes).

[dag@moria ~]$ readelf -Wl libwidevinecdm.so.x64.old.working | grep TLS
  TLS            0x321020 0x0000000000322020 0x0000000000322020 0x000010 0x000018 R   0x8
[dag@moria ~]$ readelf -Wl libwidevinecdm.so.x64.new.working | grep TLS
  TLS            0x9d8080 0x00000000009da080 0x00000000009da080 0x000014 0x000060 R   0x40

[wagnerch]

@dagwieers Since you have a Chromebook that runs with this image, perhaps you can run Chrome with LD_DEBUG?

Something like this on the command line: LD_DEBUG=bindings LD_DEBUG_OUTPUT=/tmp/ld /opt/google/chrome/chrome

Should output a /tmp/ld.pid file reporting dynamic linkers bindings (searching for symbols and where they are found). That could at least validate whether libwidevinecdm is looking for the 8 symbols noted earlier.

Example: 381634: binding file ls [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `getenv' [GLIBC_2.2.5]

basically this is running LD_DEBUG on the "ls" command, and you can see ls is resolving getenv@GLIBC_2.2.5 from libc.so.6.

[dagwieers]

@wagnerch No, I cannot. I don't have a console on these. And since they are being used, I cannot just go and scratch them :-/

Update: ChromeOS v91 can now give console access without the need to reinstall the laptop 👍🏻

[wagnerch]

@matthuisman @CastagnaIT If you haven't gotten to it, using this libwidevinecdm on Raspberry Pi OS results in the following errors on the terminal when running chromium-browser and accessing Netflix:

[1423:1:0511/123011.891141:ERROR:cdm_module.cc(139)] CDM at /opt/WidevineCdm/_platform_specific/linux_arm/libwidevinecdm.so could not be loaded. [1423:1:0511/123011.892512:ERROR:cdm_module.cc(140)] Error: /opt/WidevineCdm/_platform_specific/linux_arm/libwidevinecdm.so: cannot open shared object file: Operation not permitted

[wagnerch]

I posted this on https://github.com/emilsvennesson/script.module.inputstreamhelper/issues/437#issuecomment-839226900

There are some other things bothering me:

$ readelf -d libwidevinecdm-4.10.2252.0.so
Dynamic section at offset 0x816c58 contains 34 entries:
  Tag        Type                         Name/Value
...
...
 0x00000024 (<unknown>: 24)              0x1cf4
 0x00000023 (<unknown>: 23)              0x2b8
 0x00000025 (<unknown>: 25)              0x4

What are these unknown attributes in the dynamic section? Based on some research this is an Android-specific ELF extension called relative relocations (RELR), see: https://groups.google.com/g/generic-abi/c/bX460iggiKg https://chromium.googlesource.com/chromium/src.git/+/master/third_party/android_crazy_linker/src/src/crazy_linker_relr_relocations.h#12 https://android.googlesource.com/platform/bionic/+/master/libc/include/elf.h#246

There is also a ".relr.dyn" section in the library. Binutils has no idea what it is, one of the tools actually complains about it (I think objcopy, when I tried to strip the .init and .fini sections).

Also, when runninng: LD_PRELOAD=libwidevinecdm.so /bin/ls

And analyzing the core dump:

(gdb) bt
#0  0x003fd6a4 in ?? ()
#1  0xb6f50a48 in call_init (l=<optimized out>, argc=argc@entry=1,
    argv=argv@entry=0xbef3fa14, env=env@entry=0xbef3fa1c) at dl-init.c:74
#2  0xb6f50b20 in call_init (env=<optimized out>, argv=<optimized out>,
    argc=<optimized out>, l=<optimized out>) at dl-init.c:37
#3  _dl_init (main_map=0xb6f739a0, argc=1, argv=0xbef3fa14, env=0xbef3fa1c)
    at dl-init.c:121
#4  0xb6f409a4 in _dl_start_user () from /lib/ld-linux-armhf.so.3

0x3fd6a4 is the virtual address for InitializeCdmModule_4, which is called by the ".init" section.

$ arm-linux-gnueabihf-nm -D libwidevinecdm-4.10.2252.0.so |grep InitializeCdmModule_4
arm-linux-gnueabihf-nm: libwidevinecdm-4.10.2252.0.so: unknown type [0x13] section `.relr.dyn'
003fd760 T InitializeCdmModule_4

Oh, and hey there is nm complaining about an unknown section ".relr.dyn". Interestingly, gdb sees this symbol at: (gdb) print InitializeCdmModule_4 $1 = {<text variable, no debug info>} 0xb61b1760

So to me that suggests maybe the relocation was not done, because glibc doesn't understand it? Not sure.

[matthuisman]

has anyone confirmed it actually works on a chromebook yet?

[wagnerch]

Yes, @dagwieers confirmed the image is working fine.

[matthuisman]

@wagnerch i can confirm chromium on RPI doesn't work. VIDEOJS: "DRM: FAILED_TO_CREATE_CDM" Chromium version is 88.0.4324.187

Now, that Widevine image says need at least Chrome 89 (requires Chromium M69 or later) So, is our version too old maybe? Maybe 89 adds support for this new blob? doesnt appear any newer version of chromium is available on pi

oooo, look here: https://blog.chromium.org/

"PartitionAlloc is Chromium’s memory allocator, designed for lower fragmentation, higher speed, and stronger security and has been used extensively within Blink (Chromium’s rendering engine).

In Chrome 89 the entire Chromium codebase transitioned to using PartitionAlloc everywhere (by intercepting and replacing malloc() and new) on Windows 64-bit and Android."

Maybe Chrome OS as well? Now, they can't do that for the blob on all platforms as then Firefox / other browsers won't work. So it appears as it's Chrome OS - they could do it as they know its running in Chromium / chrome browser. You can't install Firefox in Chrome OS I assume?

[wagnerch]

@matthuisman My guess is that is why we see a whole bunch of Chromium code strings in Widevine. But whether that would cause an issue or not is unclear. The newer library seems to definitely be using a relocation method that isn't supported by the dynamic linker on mainline glibc.

But I guess we won't know for sure until RPi rolls out 89 or 90. The Widevine side suggested that 2252 has a dependency on Chromium 90 and later.

Do we know someone that can take a look at the RELR relocation stuff? It is specific to a few platforms, including ARM, and there are some tell tale signs in the widevine library that they are using this now. The patches to glibc are at: https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/965112/

It was pushed to Chrome OS back in 2018, but I don't think we have seen a true Widevine update since then.

I am not an expert in Chrome OS, never have seen it at all -- not even in a store, but it looks like from the dependencies on chrome that it's a GBM-based browser: 0x00000001 (NEEDED) Shared library: [libgbm.so.1]

So I am guessing it's more on the Android-side.

[matthuisman]

wonder if this would work on the Pi: https://packages.debian.org/sid/armhf/chromium/download Ill try

[matthuisman]

actually, the normal debian apt has 89

so do sudo apt-get remove chromium-browser sudo apt-get install chromium

you now have version 89.0.4389.114 now I get "VIDEOJS: "DRM: REQUESTED_KEY_SYSTEM_CONFIG_UNAVAILABLE" so looks like maybe it isn't even finding the drm now.

[wagnerch]

If you run it from the terminal, you should see some output on stdout, it's basically the same sort of message that ISA produces. Since I am assuming portions of this code was ported from Chromium.

[wagnerch]

Got past the segmentation fault. The TLS issue is still there, workaround is to LD_PRELOAD=libwidevinecdm.so.

The segmentation fault is caused by the RELR relocations in ChromeOS. I applied this patch to LibreELEC: https://paste.ubuntu.com/p/MFkfhYvs2J/

Copied over the glibc including linker, and forced a binary to use the dynamic linker: arm-linux-gnueabihf-g++-8 -std=c++11 -Wall -fpermissive -o wvdl wvdl.cpp -Wl,--dynamic-linker=/storage/glibc-relr/ld-linux.so.3 -ldl

This binary now runs successfully:

 # LD_PRELOAD=/storage/.kodi/userdata/addon_data/script.module.inputstreamhelper/backup/13816.64.0/libwidevinecdm.so LD_LIBRARY_PATH=/storage/glibc-relr ~/wvdl
CDM Version: 4.10.2252.0

Code:

#include <iostream>
#include <dlfcn.h>

int main(int argc, char **argv)
{
    using std::cerr;

    void *dl = dlopen("libwidevinecdm.so", RTLD_LAZY);
    if (!dl) {
        cerr << "ERROR: " << dlerror() << "\n";
        return 1;
    }

    char* (*GetCdmVersion)();
    GetCdmVersion = dlsym(dl, "GetCdmVersion");
    std::string cdm_version = GetCdmVersion();
    cerr << "CDM Version: " << cdm_version << "\n";

    dlclose(dl);
}

[matthuisman]

great work. where / how did you find that patch? looking at that patch, could you try void *dl = dlopen("libwidevinecdm.so", RTLD_BOOTSTRAP); or various options of that without using the dyanmic linker?

[wagnerch]

Patch comes from this Chrome OS changeset to glibc: https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/965112/

I only picked off the glibc one and ported it forward to 2.32 (which LibreELEC is using). The others should be for compiling binaries with these types of relocations, so didn't bother with those.

RTLD_BOOTSTRAP isn't a known flag, you basically have RTLD_NOW which means it will bind all symbols, and RTLD_LAZY which will bind symbols as used. I don't think either will change the outcome of the TLS issue.

I think what we could do is in libssd_wv.so is force a link with libwidevinecdm.so via this option during linking through gcc: -Wl,--no-as-needed libwidevinecdm.so

I think your calling ld directly you should be able to just use --no-as-needed and include that libwidevinecdm.so. Really don't quite understand what this TLS problem is, but they did change the TLS model between 1679 and 2252 as well.

BTW, this might explain the TLS problem: https://stackoverflow.com/questions/45640573/gulp-node-error-while-loading-shared-libraries-cannot-allocate-memory-in-stati

[wagnerch]

Drumroll... It sort of works. Custom build of LibreELEC with the patch, starting Kodi with: LD_PRELOAD=/storage/.kodi/cdm/libwidevinecdm.so /usr/lib/kodi/kodi.bin --standalone -fs

Video is shown, but no audio. Not sure if that is because InitializeCdmModule_4 is double-called, or they changed other things specifically for Chrome. But there is no audio at all in Kodi, without the LD_PRELOAD it is fine.

Perhaps preloading also messes with audio initialization, no idea.

[matthuisman]

hopefully we can find a way without requiring changes to kodi or libreelec as that means Kodi 18 support is gone as they will have no more builds

there is no way we can strip stuff from the widevine blob?

[wagnerch]

Yeah, I don't see this solution as practical. There is no way we can get dynamic linker changes to all variations of builds. If they changed how audio is piped as well, then it's probably a lost cause. Not quite sure what is going on there, but the audio device is actually greyed out in Kodi system settings, and of course audiooutput.audiodevice is corrupted, so have to reset it after.

Hopefully we will see a 2209 arm build of Widevine. The only other thing I can think of is maybe it's possible to convert the library from RELR relocations to the usual relocations, far from my expertise to even know if it is possible.

[matthuisman]

yeh, if we can strip stuff from the blob to make it compatible - that would be better than nothing. then at least the RPI team may host that and IA helper could grab from there

https://manpages.ubuntu.com/manpages/eoan/man1/lld.1.html

wonder if you can re-pack?

[wagnerch]

Yeah, that's what I am wondering too. I am assuming branching in assembly like "bl 0x12345678" has a pointer to the arguments "0x012345678" stored in some relocation table, where maybe the dynamic linker at link-time depending on where it loaded the section will update the actual code to adjust the offset. Just guessing on how you might do "relocation".

The only assembly I have done is way way way back with the 6502. :) And it had relative instructions (+/- 256 locations from PC) which would not require relocation in theory, and absolute instructions (like JMP) which had a 16-bit address. So if those computers actually supported relocation (LOL) then I would think a dynamic linker would need to know all of the arguments to absolute instructions so it can compute the new address at load time. (Basically address at relocation pointer - base address of section + actual address where section was loaded)

Not sure how RELR is different then that approach, versus how relocations actually work.

[kszaq]

Great job, @wagnerch ! With your patched LE build, does the libwidevine load correctly if you don't preload it?

[wagnerch]

@kszaq The TLS issue still persists. If we didn't use dlopen() then we should be able to resolve that issue. But the bigger issue is a patched dynamic linker isn't a practical fix, and audio doesn't seem to work. Perhaps that is because ALSA is dynamically loaded via dlopen as well and fails to load as well as a result of preloading libwidevine.

Basically, this version of Widevine is using ChromeOS/Android-specific relocation methods which require a dynamic linker that haven't hit mainline glibc. It's a pretty big dealbreaker, if you ask me.

[wagnerch]

OK, audio is sorted. I guess what I was missing is some nuance with how LibreELEC starts Kodi. The proper way to do the preload is:

cat >> /storage/.config/kodi.conf <<-EOF
LD_PRELOAD=/storage/.kodi/cdm/libwidevinecdm.so
EOF

Apparently some of those other environment variables that LibreELEC set influence ALSA detection. So, sure just an invasive dynamic linker patch, LD_PRELOAD, and 2252 works on Linux platforms. Completely unreasonable. :)

[jakermx]

@wagnerch it is better to use config.txt on /flash mount point for LE

https://wiki.libreelec.tv/configuration/config_txt

Hope this helps Cheers

[wagnerch]

@wagnerch it is better to use config.txt on /flash mount point for LE

https://wiki.libreelec.tv/configuration/config_txt

config.txt is for bootloader/kernel configurations. This is an environment file for Kodi startup.

[dagwieers]

Basically, this version of Widevine is using ChromeOS/Android-specific relocation methods which require a dynamic linker that haven't hit mainline glibc. It's a pretty big dealbreaker, if you ask me.

We ought to give a heads up to the LibreELEC development team. It is not impossible to get this merged and tested. We have 19 days left to make this happen before this affects all ARM Kodi users. cc @chewitt

[jakermx]

My bad... then it could be set in autostart.sh

https://wiki.libreelec.tv/configuration/startup-shutdown

:D

[wagnerch]

I don't think autostart.sh will work either, because it runs in a separate process space from the kodi binary. We have to get the LD_PRELOAD before the kodi binary starts.

autostart.sh is fired by systemd in an independent process space.

# systemctl show kodi-autostart |egrep '^Type|^Exec'
Type=oneshot
ExecMainStartTimestamp=Tue 2021-05-11 20:24:34 EDT
ExecMainStartTimestampMonotonic=9766057
ExecMainExitTimestamp=Tue 2021-05-11 20:24:34 EDT
ExecMainExitTimestampMonotonic=9809918
ExecMainPID=1107
ExecMainCode=1
ExecMainStatus=0
ExecStart={ path=/bin/sh ; argv[]=/bin/sh -c . /etc/profile; exec /bin/sh /storage/.config/autostart.sh ; ignore_errors=yes ; start_time=[Tue 2021-05-11 20:24:34 EDT] ; stop_time=[Tue 2021-05-11 20:24:34 EDT] ; pid=1107 ; code=exited ; status=0 }
ExecStartEx={ path=/bin/sh ; argv[]=/bin/sh -c . /etc/profile; exec /bin/sh /storage/.config/autostart.sh ; flags=ignore-failure ; start_time=[Tue 2021-05-11 20:24:34 EDT] ; stop_time=[Tue 2021-05-11 20:24:34 EDT] ; pid=1107 ; code=exited ; status=0 }

The ideal solution is to force a dependency in libssd_wv.so via -Wl,--no-as-needed libwidevinecdm.so, or to bump the TLS reserves in glibc (which I am assuming is a compile-time thing, but we are already down that path).

I am looking into whether RELR relocations could be converted back into regular relocations, I believe it's possible, mostly because of Cary Coutant's original writeup where he indicates Google did the opposite: https://sourceware.org/legacy-ml/gnu-gabi/2017-q2/msg00003.html

This appears to be the utility referenced: https://android.googlesource.com/platform/bionic/+/87a0617ebe7561bf28d3a19fbe192372598969b8/tools/relocation_packer/src/main.cc https://github.com/chromium/chromium/tree/418115b3467e12e606e6e48ac3532f583e9f09ac/third_party/android_platform/bionic/tools/relocation_packer