Widevine playback

[matthuisman]

I assume widevine playback won't work in browsers due to no widevine build for arm64bit?

[XECDesign]

I haven't tried it, but I am seeing posts about extracting aarch64/arm64 builds of widevine. Obviously we can't redistribute.

https://gist.github.com/teacupx/9393507ad6250429707f0318b04f1a3b

[matthuisman]

Those are still all 32 bit versions for 32 bit userland. But from 64bit devices. :(

Hopefully raspberry pi having a popular 64bit os makes google take notice and release it

[XECDesign]

Those are still all 32 bit versions for 32 bit userland. But from 64bit devices. :(

Oh, strange. AFAIK, 64bit applications can't load 32bit libraries, so does that mean all aarch64 devices run 32bit versions of chrome?

Hopefully raspberry pi having a popular 64bit os makes google take notice and release it

I wouldn't hold my breath.

https://www.theregister.co.uk/2019/04/03/googles_widevine_drm/

[matthuisman]

Pretty sure all the arm64 Chromebooks are running 32bit OS and applications.

All I know is there is no arm64 widevine. Quite a few threads about it. Manjaro springs to mind. They thought Firefox was working on a 64bit arm version with widevine but nothing yet

[XECDesign]

Good to know, thanks.

[bernardodpc]

why is this closed? I still can't load widevine on chromium on raspberry pi 64 bits

[matthuisman]

Maybe because it's out of their hands. Its Up to google to provide an arm 64bit widevine

[bernardodpc]

Maybe because it's out of their hands. Its Up to google to provide an arm 64bit widevine

oh. I see. if someone gets news on this, it would be nice to post here as well alerting us

[lurch]

It's not something I've looked into myself, but #11 has links which look like they might be relevant?

EDIT: Ooops, copy'n'paste error! I meant #99

[matthuisman]

That's all 32bit still

[matthuisman]

I see new libwidevinecdm0 package. I assume this is for 32bit RPI os only?

[popcornmix]

Correct. There is no libwidevinecdm.so for arm64.

[matthuisman]

would you be able to advise if RPI had to get permission from Google to redistribute the binary? I assume Yes

[dagwieers]

Is this binary downloaded from an official location at Google, or it is being extracted from the official ChromeOS builds? (We would like to see whether we can do the same thing for Kodi too now)

[matthuisman]

@popcornmix

According to https://www.widevine.com/news, any Widevine version below 4.10.2209.0 on Linux will be removed 31/05/2021 (3 weeks time). I assume removed means = stop working.

Does Raspberry Pi team have a plan to update it before then?

[popcornmix]

@XECDesign what version are we using?

[matthuisman]

4.10.1679.0 http://archive.raspberrypi.org/debian/pool/main/w/widevine/

[XECDesign]

4.10.2252.0 isn't working for me

[matthuisman]

@XECDesign we are aware of that https://github.com/xbmc/inputstream.adaptive/issues/678

thats actually why I'm asking I was hoping maybe you had a different source for ARM Widevine (instead of pulling from Chrome OS)

[matthuisman]

@XECDesign possibly Chromium 89 is required: https://github.com/xbmc/inputstream.adaptive/issues/678#issuecomment-839239819

[XECDesign]

I can try getting a non-accelerated chromium 89 build going to check.

I had a bit of a play around in a chromeos chroot to see if I can figure what's happening but it quickly started to look like a bigger rabbit hole than I had time or knowledge to go down.

[XECDesign]

Chromium 90 doesn't magically fix the glibc issues, unfortunately. Patching glibc isn't enough either (and isn't something I'd want to do for the whole OS if it can be avoided).

It looks like we need the Bullseye version of glibc + the patch and maybe the LD_PRELOAD trick. If I can get to a point that's working, I'll try to bundle the appropriate files with chromium and use patchelf to force it to use them.

[XECDesign]

Upgrading glibc and patching it was enough to get v88 to work with 4.10.2252.0. I'll see if there's a non-intrusive way to get this to work in buster.

v90 doesn't work. It seems to load the .so and then denies it's available, haven't looked at what has changed there yet.

[matthuisman]

I tried chrome from the debian apt (89.0.4389.114) and couldn't get it to load the widevine lib either. I got "VIDEOJS: "DRM: REQUESTED_KEY_SYSTEM_CONFIG_UNAVAILABLE" Which suggests it did get loaded but then didn't want to do widevine or something... Sounds like the same issue you had with 90.

your using this glibc patch right? https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/965112/ It says the plan on pushing it upstream... is there any mention of it on upstream glibc?

[XECDesign]

So far I've only managed to make it work with 88 + bullseye + patched glibc. I am confident I can make 90 work once we have the HW acceleration patch for it.

The buster version of glibc can't handle the .so file with or without that patch. Trying force it to copy the working version of glibc in a different directory in a Buster install and forcing it to use that, didn't work. And we can't just upgrade glibc on Buster, as it breaks other packages.

I think 4.10.2252.0. will have to be a bullseye thing.

[wagnerch]

@XECDesign The big piece you need is the dynamic linker /lib/arm-linux-gnueabihf/ld-2.28.so to be updated (but you will probably have to use a static busybox to do it, i think). I think the patch would work fine on glibc-2.28, if you want to roll a test package and try it or swap that one binary. Also, you can see what I did in this comment: https://github.com/xbmc/inputstream.adaptive/issues/678#issuecomment-839287040

Where I installed a second glibc, and forced a linked executable to use the patched dynamic linker (this is the INTERP entry you see in program headers via readelf -l). Supposedly you can force a different interpreter (dynamic linker) by doing:

/path/to/new/ld-2.28.so chromium-browser

I assume that dynamic linker would be used for everything executed down, but don't know for sure. Never tried it. There is also patchelf which can manipulate the interpeter, but I think it's a lot easier to just move the shared library using a static busybox. Replacing the dynamic linker live could be a problem since tools like mv and such are dynamically linked.

[XECDesign]

Hi @wagnerch, thank you for all your work on this.

Although the patch applies to 2.28, ldd says it's it's not a dynamic executable and trying to load it spits out a permissions error. The behaviour is identical with and without the patch.

From a bullseye environment (glibc 2.31), ldd works and applying the patch makes widevine work in chromium 88 without any further changes.

Trying to use a second patched ld.so and leaving the original in place didn't quite work. Chromium errors out fairly on. But I think I was trying to use glibc 2.31 in buster and may have missed a dependency I should've dropped into the LD_LIBRARY_PATH directory and it tried to use an incompatible system one.

I'm not familiar with the usage of busybox you're describing. How does static busybox fit into the equation?

[wagnerch]

@XECDesign Yeah, it looks like you do need the full glibc, so you would probably have to do /path/to/new/ld.so --library-path to reference a location with all the glibc library. That is what I was doing in testing as well, but I tried to build glibc-2.28 and the tests actually failed (?).

This is what glibc runs to test itself before installling: /home/pi/glibc-2.28/build-tree/armhf-libc/elf/ld-linux-armhf.so.3 --library-path /home/pi/glibc-2.28/build-tree/armhf-libc:/home/pi/glibc-2.28/build-tree/armhf-libc/math:/home/pi/glibc-2.28/build-tree/armhf-libc/elf:/home/pi/glibc-2.28/build-tree/armhf-libc/dlfcn:/home/pi/glibc-2.28/build-tree/armhf-libc/nss:/home/pi/glibc-2.28/build-tree/armhf-libc/nis:/home/pi/glibc-2.28/build-tree/armhf-libc/rt:/home/pi/glibc-2.28/build-tree/armhf-libc/resolv:/home/pi/glibc-2.28/build-tree/armhf-libc/mathvec:/home/pi/glibc-2.28/build-tree/armhf-libc/support:/home/pi/glibc-2.28/build-tree/armhf-libc/crypt:/home/pi/glibc-2.28/build-tree/armhf-libc/nptl <executable>

As for busybox, I was trying to say if you are going to mv ld.so and cp in a replacement then you would need to use busybox, since mv & cp are dynamically linked and require ld.so.

I still have some weirdness where it says the library is not found, not sure what that is about.

[wagnerch]

@XECDesign FYI, Based on my testing, minimum version of glibc that works is 2.29. Not really sure why.

$ /home/pi/glibc-2.29/build-tree/armhf-libc/elf/ld-linux-armhf.so.3 --library-path /home/pi/glibc-2.29/build-tree/armhf-libc:/home/pi/glibc-2.29/build-tree/armhf-libc/math:/home/pi/glibc-2.29/build-tree/armhf-libc/elf:/home/pi/glibc-2.29/build-tree/armhf-libc/dlfcn:/home/pi/glibc-2.29/build-tree/armhf-libc/nss:/home/pi/glibc-2.29/build-tree/armhf-libc/nis:/home/pi/glibc-2.29/build-tree/armhf-libc/rt:/home/pi/glibc-2.29/build-tree/armhf-libc/resolv:/home/pi/glibc-2.29/build-tree/armhf-libc/mathvec:/home/pi/glibc-2.29/build-tree/armhf-libc/support:/home/pi/glibc-2.29/build-tree/armhf-libc/crypt:/home/pi/glibc-2.29/build-tree/armhf-libc/nptl:/home/pi ./wvdl
CDM Version: 4.10.2252.0

$ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

[wagnerch]

@XECDesign So it turns out the issue is not glibc-2.28, but actually Debian patches: arm/unsubmitted-ldconfig-cache-abi.diff arm/unsubmitted-ldso-abi-check.diff

If you revert those patches, and apply the SHT_RELR patch, then glibc-2.28 should be working.

https://www.mail-archive.com/search?l=debian-bugs-dist@lists.debian.org&q=subject:%22Bug%23943798%5C%3A+ld.so%5C%3A+old+Arm+ABI+detection+patch+causing+problems%2C+time+to+remove%5C%3F%22&o=newest&f=1

[XECDesign]

Thanks for the hint about those patches - removing them makes everything just work.

I'm still not having any luck trying to run things through ld-linux-armhf.so.3 even without trying to alter any paths. It only seems to work for some applications.

To keep things simple:

$ /lib/ld-linux-armhf.so.3 /usr/lib/chromium-browser/chromium-browser-v7 
[0521/162556.332466:ERROR:icu_util.cc(248)] Couldn't mmap icu data file
Trace/breakpoint trap

[XECDesign]

Ah, but using patchelf to set the interpeter instead gets around this. So I wonder if the widevine package should ship the necessary libraries and patch/unpatch chromium in postinst/rm.

[wagnerch]

@XECDesign Yeah, I didn't try Chromium, but I know it has a bunch of wrapper shell scripts, maybe related. As for publishing a glibc to Widevine, it's certainly an option to explore. But from what I could tell the glibc patch just appears to have logic to parse RELR (relative relocations) and apply the relocations. It's triggered by a section and several entries in the dynamic section.

Although, I guess the potential risk is this new section, and dynamic table entries are a bit unclear whether they are part of the official specification (System V Generic ABI). So who knows on that front, but LLVM picked up these patches into their linker, and it's kind of out there. Google switched over to using LLVM to produce this type of relocations in Android & ChromeOS from what I have understood, so I guess we will see if GNU binutils & glibc come around and apply it eventually just by pressure.

I think if you replaced glibc with removing the 2 patches and adding this glibc patch, installing it over a buster install then Chromium should work.

[XECDesign]

Added libwidevinecdm0 4.10.2252.0-1 to untested component.

http://archive.raspberrypi.org/debian/pool/untested/w/widevine/libwidevinecdm0_4.10.2252.0-1_armhf.deb

[kszaq]

@XECDesign Are ChormeOS system images source for the library? Does the license permit to use the Deb you provide in inputstream.helper for Kodi on all ARM targets?

[XECDesign]

ChromeOS images are the source. I believe you are in the same position whether you extract the file from ChromeOS images or these debs. I suspect ChromeOS images are a more reliable source, so you should at least have it as a fallback if the debs get pulled.

[dagwieers]

Team Kodi did not want to offer the Widevine CDM and only accepted an implementation that would let the end-user do the download and extraction. As we are not allowed to distribute.

The deb would offer a more efficient way of extracting the Widevine CDM, however we have had no confirmation that the redistribution inside of the deb package is permitted by Google.

There have been other locations where the Widevine CDM is stored and could have been used, but those were definitely not permitted redistributions.

On a positive note, we expect the ARM version will become a separate download once LaCrOS happens. Unfortunately it seems to be delayed, but we never expected changes that would break Linux compatibility.

[matthuisman]

@XECDesign Who takes care of the Debian ARM Kodi package that Raspberry Pi OS uses? that will also need updating to support the new blob

[dagwieers]

@matthuisman We have all stakeholders documented in the ISA Wiki, it could be a starting point to inform all parties to act up?

We are not certain (and I think it is unlikely) the old Widevine will be revoked on 2021-05-31. But we better be safe than sorry...

[matthuisman]

@dagwieers sorry, i wasn't talking about ISA - the Kodi debian package itself will need modifications https://www.deb-multimedia.org/dists/stable/main/binary-armhf/package/kodi https://www.deb-multimedia.org/dists/testing/main/binary-armhf/package/kodi is above what RPI OS uses? Who maintains that?

unless they decided to put the required modified files inside the kodi-inputstream-adaptive deb instead... which I guess would be fine as thats when it is needed

[XECDesign]

Who takes care of the Debian ARM Kodi package that Raspberry Pi OS uses? that will also need updating to support the new blob

That's provided by Rascas from the forum:

https://www.raspberrypi.org/forums/viewtopic.php?f=66&t=251645

[dagwieers]

@dagwieers sorry, i wasn't talking about ISA - the Kodi debian package itself will need modifications

I know, but it may help to get the right attention. It affects the inputstream-adaptive packager as well (In case he is not the same person involved in Kodi packaging).

But IMO a wider communication to all stakeholder is useful so they are forwarned.

[matthuisman]

if you want to do that - that would be great :)

[matthuisman]

Fyi for everyone on here. 64bit arm widevine cdm finally came out last month and has team rpi already have a new 64bit chromium build that is compatible: https://forums.raspberrypi.com/viewtopic.php?t=347736

[ashr123]

@matthuisman thanks! Is there a way to install it on Ubuntu 22.10?

[ashr123]

Anybody?

[ghollingworth]

You're asking in the wrong place, I'd suggest asking in the Ubuntu forums