search

ValveSoftware / steam-for-linux

Issue tracking for the Steam for Linux beta client
4.18k stars 173 forks source link

Libthai and new steamwebhelper #6155

Closed garpu closed 5 years ago

garpu commented 5 years ago

Your system information

Please describe your issue in as much detail as possible:

I updated the client to today's new build, and the store won't load with the following error in console: ./steamwebhelper: error while loading shared libraries: libthai.so.0: cannot open shared object file: No such file or directory

Library loads, and I can launch games. I can't open my friend list, store, communities, or profile. I don't use the Thai language. Is there a way to disable it? Or shouldn't this come with the Steam runtime libs?

Steps for reproducing this issue:

  1. Start Steam
  2. Load the store.
garpu commented 5 years ago

Installing libthai fixed the issue. I seem to recall there being an option for the Thai language before in Steam. Did a library get left out of the runtime?

ghost commented 5 years ago

Had the very same problem on Gentoo. Was able to get around it by making custom ebuilds for libdatrie and libthai, since non of them exists in the Gentoo repository or any overlays that I could find. ebuilds for those who may need them: libdatrie libthai

Your system information

sylware commented 5 years ago

custom light gnu/linux distro without libthai, exactly the same issue

0xC0ncord commented 5 years ago

On Gentoo GNU/Linux with the same issue. @fureloka's ebuilds fixed the problem instantly as soon as they were merged; didn't even need to restart Steam.

Your system information

chewi commented 5 years ago

I'll consider adding the ebuilds to the official Gentoo tree at the weekend but Pango automagically links against libthai so we need to fix that first.

kisak-valve commented 5 years ago

Hello, per "Fix regression causing black pages for web based components for some Linux users due to missing dependencies in new Chromium update" in the 2019-03-21 Steam client beta update, please retest this issue.

Note: Running steam without the steam runtime will still need ~this dependency and~ #6156 handled by the distro's package maintainer(s) for Steam.

sylware commented 5 years ago

Got the update of the beta client: libthai still missing even with the new steam runtime, and indeed, "anything webcomponent" is dark.

garpu commented 5 years ago

For the sake of testing, I uninstalled libthai, and I'm still getting the error that libthai is missing. This is with the build pushed today (2019-03-21).

I've got STEAM_RUNTIME_PREFER_HOST_LIBRARIES set to false, so I'm definitely using the steam runtime.

ETA: reinstalling libthai fixes it.

natethor commented 5 years ago

On Solus 4.0 I'm also having this same issue. I tested with the Steam Client and the Beta Update, reinstalled multiple times. I also tried running "STEAM_RUNTIME_PREFER_HOST_LIBRARIES=0 steam" and "STEAM_RUNTIME=1 steam", neither worked as a work-around.

OS: Solus 4.0 fortitude Kernel: x86_64 Linux 4.9.163-127.lts

sylware commented 5 years ago

wait for the update which will include libthai and its deps, and everything should work again.

garpu commented 5 years ago

I thought this update was supposed to have the updates?

sylware commented 5 years ago

give them time

TheDaftRick commented 5 years ago

Newest update and still have this issue on Solus

Built: Mar 21 2019, at 21:47:17 Steam API: v018 Steam package versions: 1553209175

Taking libdatrie.so.1 and libthai.so.0 from my Ubuntu machine and placing it in the /Steam/ubuntu12_64/ folder fixes the issue

sylware commented 5 years ago

once those deps are packaged with the steam client web component (I guess blink, the google fork of webkit for chrome), everything will work again on non-thai enabled distros. !!! Just thought about it: the licences of those deps: MIT or BSD or Lesser GPL? Coze if it's plain GPL (without the lesser exception), valve is not allowed to distribute them since they link with closed source programs. Wild guess without checking: it's lesser GPL and distribution is ok.

kisak-valve commented 5 years ago

Hello, per "Additional missing web component dependencies." in the 2019-03-22 Steam client beta update, please retest this issue.

garpu commented 5 years ago

I uninstalled libthai and libdatie, and it works again for me. No problems with steamwebhelper.

sylware commented 5 years ago

still broken, because libharfbuzz.so.0 is now missing: ./steamwebhelper: error while loading shared libraries: libharfbuzz.so.0: cannot open shared object file: No such file or directory

DerRidda commented 5 years ago

Also fixed for me in the latest update.

vmisupov commented 5 years ago

Have this issue on Gentoo too. Steam and Dota starts, but Dota cant start match and Steam Friend Chat does not work.


./ubuntu12_32/steam-runtime.old/i386/usr/lib/i386-linux-gnu/pango/1.6.0/modules/pango-thai-lang.so
    libthai.so.0 => not found
./ubuntu12_32/steam-runtime.old/amd64/usr/lib/x86_64-linux-gnu/pango/1.6.0/modules/pango-thai-lang.so
    libthai.so.0 => not found
./ubuntu12_32/steam-runtime/i386/usr/lib/i386-linux-gnu/pango/1.6.0/modules/pango-thai-lang.so'
    libthai.so.0 => not found

Have topic on forum: https://forums.gentoo.org/viewtopic-p-8320322.html

vmisupov commented 5 years ago

I'll consider adding the ebuild

Have you reported to bugs.gentoo? Need add ebuilds to portage and add deps for steam in wiki

sylware commented 5 years ago

This is not specific to gentoo.

Steam webcomponent and/or steam runtime is missing some dependencies: was libthai, now it's libharfbuzz.

chewi commented 5 years ago

No need to report to Gentoo, I'm already here. :stuck_out_tongue: @anyc dealt with libthai and libdatrie in steam-overlay but as mentioned, they've now been bundled anyway. Having just switched to the beta myself, I'm now missing libffi.so.6 and libselinux.so.1. I also see that libpcre.so.3 can be an issue.

To be honest, all this broken bundling is intensely frustrating. I'm already having to deal with the fact that Gentoo doesn't want to package ancient libgcrypt 1.5 any more, which is needed for old Source Engine games. I am the author of esteam, which helps to reduce some of these issues but it is currently only applied to games, not Steam itself. We've tried to cover the libraries Steam requires with regular dependencies but that's looking increasingly difficult. I'll see whether I can address this in the launcher script.

sylware commented 5 years ago

You mean that there is more than libharbuzz missing from the steam client/runtime?

chewi commented 5 years ago

Yes, although most people will have libharfbuzz anyway, it's a hard requirement of GTK3.

Options are limited. Changing or removing files in this directory results in Steam automatically repairing them before it even starts. I thought about LD_LIBRARY_PATH but this directory is always prepended to it. The only solution I've found is LD_PRELOAD. It's not great but it works. I'll try and put something together.

sylware commented 5 years ago

Yes, although most people will have libharfbuzz anyway, it's a hard requirement of GTK3.

I don't have GTK3, and "most people will have libharfbuzz.." is a very wrong reason, because if it was a good one, we would not have a gnu/linux port in the first place.

So, I can expect more missings deps once libharfbuzz is packaged in the steam client/runtime, pepe hands.

vmisupov commented 5 years ago

No need to report to Gentoo, I'm already here. stuck_out_tongue @anyc dealt with libthai and libdatrie in steam-overlay but as mentioned, they've now been bundled anyway. Having just switched to the beta myself, I'm now missing libffi.so.6 and libselinux.so.1. I also see that libpcre.so.3 can be an issue.

To be honest, all this broken bundling is intensely frustrating. I'm already having to deal with the fact that Gentoo doesn't want to package ancient libgcrypt 1.5 any more, which is needed for old Source Engine games. I am the author of esteam, which helps to reduce some of these issues but it is currently only applied to games, not Steam itself. We've tried to cover the libraries Steam requires with regular dependencies but that's looking increasingly difficult. I'll see whether I can address this in the launcher script.

Thank you very much for your work!)

TTimo commented 5 years ago

In the interest of reducing the deployment and testing churn on this, here are two more libraries that we will deploy in the next beta Steam Client update:

83f9537995bf3231d94bdc06adb56ffb  libharfbuzz.so.0
65477ec91c7650351891103fc2453ad5  libgraphite2.so.3

https://www.dropbox.com/s/bq6f7w3byz9aaf3/webhelper-runtime-20190325.tgz?dl=0

We hope that this is the last set of libraries that we include.

Libraries such as libselinux.so.1 and libpcre.so.3 that I see mentioned are already included in the steamrt:scout runtime. We believe that if you are seeing errors with those, it is because you are using a distribution-altered steam package, and we will not chase those further.

Several of the distribution maintainers are active on here, so please work this out with them, preferrably on the distribution's own bug trackers.

We anticipate to bring significant changes to the runtime setup over the next few months, I would encourage everyone to setup the Steam Client with as little distribution customization as possible.

chewi commented 5 years ago

@TTimo, knowing who you are, I hold a good deal of respect and I know that what I say carries little weight here but I feel the need to say this anyhow.

As a distro developer, I would much prefer that you bundled less, not more. This bundling largely seems to be to support libcef.so. Few, if any, distros seem to package it so it is reasonable that you would bundle it. Gentoo's excuse is that Chromium itself is hard work to maintain and takes forever to build so I don't really want to burden myself and the users with libcef as well. The libraries it directly needs are a very commonly available set though and none are any of the missing libraries mentioned above. These missing libraries are further optional subdependencies that would be avoided if you simply used the system copies of glib and Pango. Now I can imagine that you may have built Pango with libthai because you undoubtedly have Thai users to support. Gentoo, on the other hand, has seemingly got this far down the road without any Thai users complaining and even if we did support libthai, we wouldn't force it on everybody. So while I can accept that you may also need to bundle the likes of libthai, Steam's automatic repair feature also makes it very awkward for us to run it without. Please bear this in mind.

As for libpcre.so.3, you can hardly blame us for that as this soname is a Debian invention that is not recognised by upstream or any non-Debian distros. I had to make a compatibility package with a symlink.

chewi commented 5 years ago

I've just gone back to an older system to check what it looked like before and it was as I suggested above. libcef.so bundled with the system left to deal with glib and Pango, among others. I'm curious to know why this changed.

sylware commented 5 years ago

On Mon, Mar 25, 2019 at 11:20:14AM -0700, TTimo wrote:

We anticipate to bring significant changes to the runtime setup over the next few months, I would encourage everyone to setup the Steam Client with as little distribution customization as possible.

Users of minimal gaming gnu/linux distros can be very usefull benchmarks. (I'll gladly help fine tune the distribution packages)

I can already give some advices.

core gamer system dependencies on linux based OSes:

1 - elf dynamic loader/DNS network configuration: provided by the "libc".

 user level public customization of elf dynamic loader configuration is
 provided by LD_* environment variables. But there is a private non customizable
 part (don't count on /etc/ld.conf for instance, and expect a "system" content of ld_*
 variables).

 On steamos, a network configuration interface would be expected, which is
 the case with a networkmanager stack.

 big caveat: there is no binary compatibility between gnu libc and other
 libc, for instance musl libc. This is very sad, this is a failure of core
 ABI. Then gnu libc ABI (since elf ABI is not enough), would be the way to go.
 Don't expect the gnu libc "programs" being available to a gamer user level
 account.

 If valve deploys its own DNS servers, it could provide it's own network
 stack on top of the linux kernel network ABI.
 But I don't think it's a good idea (DNS servers DDOS).

2 - On a gcc toolchain: compile everything with --static-libgcc and --static-libstdc++. The gamer system libgcc_s will be loaded by the gamer system glibc for pthread_cancel to work. g++ and libgcc_s ABIs are HELL, stay as far as you can from them. I did solve tons of 3D drivers issues since my custom distro is compiled with those flags... but nearly all the gnu/linux distros out there do not, so it's more reasonable to put those compilation flags on valve and games side.

3 - do not expect to run any system administration level programs (could be hidden at the user level).

4 - sound, the alsa lib (pulseaudio is not to be expected since libasound could use dmix or pulseaudio for mixing). The gamer system alsa lib should be loaded, since its deployment and configuration could be specific and hardcoded into the libs. For instance, there is no way to tell a third party libasound to fetch it's data files elsewhere than the path, usually /usr/share/alsa, hardcoded at compilation time. There is still a possible conflict between applications using the gamer system alsa lib with those distributed by valve and using the steam runtime alsa lib, for instance if dmix is configured differently.

5 - input methods and windowing system. WSI: x11 or wayland (android is either wayland or native). In a perfect world, it should be dynamically discovered or forced by the gamer (xwayland...). The gamer system input methods should be used since it could be heavily customized (keyboard layout, virtual keyboard, CJK input methods...). Some configuration paths are hardcoded into x11 libs at build time, then if distributed, configuration file system location mistmatches could happen (for instance /usr/share/x11 is hardcoded in some x11 client libs).

6 - 3D APIs. GL or vulkan. In a perfect world, it should be dynamically discovered with a 100% software fallback based on the WSI type (some gamers may run the client on some systems unable to run their video games, for administrative purpose or social purpose).

7 - expect only a basic shell (for instance dash or busybox ash), and a mimimal set of user level posix commands. Be very conservative in their usage. Don't hesitate to code and distribute some programs to help your shell scripts (actually the right way to do it). Don't expect any high level script engines on the gamer system: python2/python3, perl5/perl6, lua, node.js (javascript), squirel, php, ruby, haskell, etc (PLZ!). But valve could distribute its prefered ones, ofc.

8 - do not expect a specific GFX toolkit to be there: GTK+, Qt, EFL, TCL_TK... only WSI native should be expected.

9 - do not expect a huge www browser like firefox or chromium (don't expect anything javascript). Basically, it should work with curl...

10 - do not count on the Linux Standard Base: only huge mainstream distros with an army of devs and integrators can achieve this. This is really not reasonable for small/medium distros.

11 - expect clean 64bits distros without 32bits support.

12 - everything else should be packaged and distributed by valve/games.

A good way to work on a binary distribution package, would be to do that in a chroot jail with system dependencies carefull accounted for, build and configured in "non-standard" file system location (file system fuzzing). You would start with:

TTimo commented 5 years ago

It is unfortunate that my comment about distribution alteration of the Steam packages being harmful is taking center stage here. Let's focus on the issue at hand instead.

Since no one has chimmed in on harfbuz and graphite we'll go ahead and assume that they are the last two libraries that we needed to include. This is confirmed by testing, the remaining libraries pulled from the host are core libraries and/or the graphics stack which wouldn't appropriate (or possible) to bundle.

Finally, a bit of background information regarding the runtime environment that the steamwebhelper process is expected to run in:

The LD_LIBRARY_PATH is setup so that ubuntu12_64/ is searched first, then the steamrt:scout runtime is searched (ubuntu12_32/steam-runtime/amd64/) and finally the host.

By comparison with the production client, the only difference is the addition of the first ubuntu12_64/ lookup. The reason is that the new CEF can no longer be easily compiled against the steamrt:scout runtime, but it hasn't changed so much that it requires a complete new runtime, so this is an adequate intermediate solution.

sylware commented 5 years ago

I have not get any steam beta client update. I'm still stuck with a missing libharfbuzz.

TTimo commented 5 years ago

I have not get any steam beta client update. I'm still stuck with a missing libharfbuzz.

That's because I posted the libraries ahead of time to this thread, so we can test it out without going through a full beta client update cycle, in case more iteration is needed. I think we will update the beta client today though.

sylware commented 5 years ago

Jez, could not test: dropox wants aka javascri pt. Do you have a direct link? ... I wait the update and pray that all the deps are distributed :( (libharfbuzz could depend on a finit-state automaton lib, rygel, if I recall properly, but it was a long time ago, maybe this dep is gone).

sylware commented 5 years ago

Maybe you could drop the file into a public github gist? (I always forget about gist) Did you check harfbuzz dep on rygel finite-automaton lib?

cpages commented 5 years ago

In the interest of reducing the deployment and testing churn on this, here are two more libraries that we will deploy in the next beta Steam Client update:

83f9537995bf3231d94bdc06adb56ffb  libharfbuzz.so.0
65477ec91c7650351891103fc2453ad5  libgraphite2.so.3

https://www.dropbox.com/s/bq6f7w3byz9aaf3/webhelper-runtime-20190325.tgz?dl=0

We hope that this is the last set of libraries that we include.

@TTimo I confirm that those were the last missing deps for me (NixOS). Thanks!

chewi commented 5 years ago

It is unfortunate that my comment about distribution alteration of the Steam packages being harmful is taking center stage here. Let's focus on the issue at hand instead.

Sorry, harfbuzz and graphite2 are libraries that Gentoo packages so we were already covered there.

Finally, a bit of background information regarding the runtime environment that the steamwebhelper process is expected to run in:

The LD_LIBRARY_PATH is setup so that ubuntu12_64/ is searched first, then the steamrt:scout runtime is searched (ubuntu12_32/steam-runtime/amd64/) and finally the host.

By comparison with the production client, the only difference is the addition of the first ubuntu12_64/ lookup. The reason is that the new CEF can no longer be easily compiled against the steamrt:scout runtime, but it hasn't changed so much that it requires a complete new runtime, so this is an adequate intermediate solution.

Thanks for the explanation, that makes sense. Steam used to have a variable called STEAM_RUNTIME_PREFER_HOST_LIBRARIES but I gather that's no longer used because host libraries are preferred by default. This only applies to games though. Could we have an equivalent for the platform libraries?

sylware commented 5 years ago

Got the update. Fixed here too.

TTimo commented 5 years ago

Steam used to have a variable called STEAM_RUNTIME_PREFER_HOST_LIBRARIES but I gather that's no longer used because host libraries are preferred by default.

It still works. That's a debug feature meant to be set to 0 to rely the runtime as much as possible and disable library pinning. On mesa-based graphics stacks that will fail because you lose working drivers, but it can be useful for developers on nvidia systems.

Thanks for testing!

chewi commented 5 years ago

Steam used to have a variable called STEAM_RUNTIME_PREFER_HOST_LIBRARIES but I gather that's no longer used because host libraries are preferred by default.

It still works. That's a debug feature meant to be set to 0 to rely the runtime as much as possible and disable library pinning.

Sorry, yes, I was tired and forgot the specifics. My point was that it would be useful to have a similar variable like STEAM_PLATFORM_PREFER_HOST_LIBRARIES that defaults to 0 but could be set to 1.