[Steam Beta client] Steamwebhelper doesn't start

batocera-linux / batocera.linux

batocera.linux

https://batocera.org

Other

1.9k stars 491 forks source link

[Steam Beta client] Steamwebhelper doesn't start #10948

Closed MastaG closed 5 months ago

MastaG commented 6 months ago

Batocera build version

v39 28-01-2024

Your architecture

Your Graphic Processor Unit (GPU)

Dedicated (e.g. GTX 1050; Radeon R9 380)

Graphical brand

Nvidia

Issue description

I've followed the wiki instructions to install the com.valvesoftware.Steam flatpak. It works but when I try to opt in for the beta client, it will restart and download the update but fail to start any longer. It will show me a window saying the Steamwebhelper process is not responding and I can:

Restart Steamwebhelper
Run Steamwebhelper without GPU acceleration
Run Steamwebhelper without Sandbox
Restart Steam
Exit

None of the options will get Steam running. I have to uninstall the flatpak and all the userdata from terminal and reinstall Steam in order to get it working again.

Expected result

Working Steam beta client

Reproduction steps

Install Steam from flatpak as per wiki instructions Login and opt in for beta client

Logs and data

No response

Tovarichtch commented 6 months ago

Issue seems to be upstream from a recent update. I don't think Batocera is the cause here.

MastaG commented 6 months ago

Sorry for this, it has already been reported: https://github.com/ValveSoftware/steam-for-linux/issues/10412

MastaG commented 6 months ago

Perhaps it is related to Batocera afterall, because the sniper runtime seems to be properly unpacked. Here's the steamwehelper.log:

steamwebhelper.sh[3866]: === Wed Jan 31 10:10:40 CET 2024 ===
steamwebhelper.sh[3866]: Starting steamwebhelper under bootstrap sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper
x86_64-linux-gnu-capsule-capture-libs: warning: Dependencies of libnvidia-pkcs11.so.545.29.06 not found, ignoring: Missing dependencies: Could not find "libcrypto.so.1.1" in LD_LIBRARY_PATH "", ld.so.cache, DT_RUNPATH or fallback /lib:/usr/lib
steam-runtime-launch-client[3888]: E: Can't find session bus: Error spawning command line `dbus-launch --autolaunch=6b3a9ea93eca9786eec2fc8a65b9ce24 --binary-syntax --close-stderr': Failed to execute child process "dbus-launch" (No such file or directory)
steamwebhelper.sh[3866]: Starting steamwebhelper with sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper
x86_64-linux-gnu-capsule-capture-libs: warning: Dependencies of libnvidia-pkcs11.so.545.29.06 not found, ignoring: Missing dependencies: Could not find "libcrypto.so.1.1" in LD_LIBRARY_PATH "/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32:/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/panorama:/app/lib/i386-linux-gnu/GL/default/lib:/app/lib/i386-linux-gnu/GL/nvidia-545-29-06/lib:/app/lib32:/app/lib/i386-linux-gnu:/lib64:/app/lib:/usr/lib/x86_64-linux-gnu/GL/default/lib:/usr/lib/x86_64-linux-gnu/GL/nvidia-545-29-06/lib:/usr/lib/x86_64-linux-gnu/openh264/extra:/usr/lib/x86_64-linux-gnu", ld.so.cache, DT_RUNPATH or fallback /lib:/usr/lib
steam-runtime-launch-client[3946]: E: Can't find session bus: Error spawning command line `dbus-launch --autolaunch=6b3a9ea93eca9786eec2fc8a65b9ce24 --binary-syntax --close-stderr': Failed to execute child process "dbus-launch" (No such file or directory)

MastaG commented 6 months ago

See: https://github.com/ValveSoftware/steam-for-linux/issues/10412#issuecomment-1919184637

@MastaG: You have a similar symptom, but for an entirely different reason. Please open a separate issue, and make its title as specific to your situation as possible, for example perhaps "steamwebhelper beta not starting under Flatpak: Failed to execute child process "dbus-launch".

I have confirmed that steamwebhelper does normally work correctly under Flatpak, so there must be something specific to your system that is making it fail in this way. It looks as though Steam is not receiving the DBUS_SESSION_BUS_ADDRESS environment variable.

Please try launching Steam as:

STEAM_LINUX_RUNTIME_VERBOSE=1 flatpak run com.valvesoftware.Steam

This will put a lot more information in steamwebhelper.log which will help to figure out what is happening on your system.

[edited to add]

Since the recent beta, running Steam under Flatpak requires a D-Bus session bus with the flatpak-portal service available. This is normally true on systems that can run Flatpak apps (and on desktop Linux in general), but I see from your linked issue report that you are using something called Batocera Linux, and I don't know whether that operating system is missing this functionality.

When running Steam un-sandboxed, a D-Bus session bus is strongly recommended but not strictly required: it's only when it has been installed as a Flatpak app that the D-Bus session bus is a functional requirement.****

smcv commented 6 months ago

Is Batocera intended to provide a working D-Bus session bus?

If yes, it should set the DBUS_SESSION_BUS_ADDRESS environment variable to its address, so that the variable can be inherited by Flatpak. Typically this is unix:path=$XDG_RUNTIME_DIR/bus if using systemd --user, or a randomly-generated address set by dbus-run-session or dbus-launch if not.

If no, then you will not be able to run the Steam beta client or future stable versions of the Steam client as a Flatpak app until a D-Bus session bus is added. You will also not be able to run Proton or Steam Linux Runtime games under a Flatpak version of the Steam client. This is a requirement for the Flatpak functionality that Steam uses to implement this feature.

Typical desktop distributions like Arch, Fedora or Ubuntu set up a working D-Bus session bus automatically, and probably a lot of software (including other parts of Steam) assumes that one is present.

MastaG commented 6 months ago

@smcv Thanks for sharing your insights. I guess the batocera developers should be able to provide information if dbus is properly supported (with the correct variables) when launching the Steam flatpak.

dmanlfc commented 6 months ago

dbus session is set as follows - export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket"

MastaG commented 6 months ago

I'm setting up a dev environment and I'll try to fix this.

smcv commented 6 months ago

dbus session is set as follows - export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket"

That isn't a correct thing to do. The session bus and the system bus are two different things, and are not interchangeable.

The system bus is used for communication with system services like NetworkManager and flatpak-system-helper. It's implemented by an instance of dbus-daemon --system (or maybe dbus-broker) running as a system uid, typically named something like _dbus or messagebus and run by pid 1 from /lib/systemd/system/dbus.service. Unprivileged session services like flatpak-portal are not allowed to own names on the system bus.

The session bus is used for communication within the scope of a single user's desktop session, for per-user services like flatpak-portal, xdg-desktop-portal and dconf. It's implemented by an instance of dbus-daemon --session (or maybe dbus-broker) running as the user ID of the user who owns the desktop session, typically run by systemd --user from /usr/lib/systemd/user/dbus.service.

If you're unsure how this is meant to fit together, a default installation or live-image of Debian with GNOME or KDE Plasma might be a useful reference. (Of course many other distributions and desktop environments are available, but I happen to be the maintainer of dbus in Debian, so I know that Debian does set this up correctly.)

smcv commented 6 months ago

A good way to test some of the required functionality for this would be:

in one terminal:
- /usr/libexec/flatpak-portal -vr (or wherever you install it in your distribution)
- after a bunch of other debug output, it should say F: Name acquired
in another terminal:
- flatpak run --command=bash com.valvesoftware.Steam
- grep instance-id /.flatpak-info
- flatpak-spawn -- grep instance-id /.flatpak-info
- you should see two different instance IDs
- in the other terminal, you should see something like F: Client Pid is 12345, F: Client Pid 12345 died
- if you see F: ioctl(0, TIOCSCTTY, 0) failed: Operation not permitted, that's harmless and you can ignore it
- any other error message is probably a bad thing

dmanlfc commented 6 months ago

thanks @smcv. we run from buildroot without systemd, so this will need amending accordingly. flatpak is not a priority of this project, but we should look to see what we can do.

smcv commented 6 months ago

we run from buildroot without systemd, so this will need amending accordingly

In that case you'll probably need to use dbus-run-session(1) or dbus-launch(1) to get a session bus for your graphical login sessions - that's how this was traditionally done before systemd, and how it's still done in sysvinit-based distributions.

dmanlfc commented 6 months ago

our GUI is minimal & we launch directly into EmulationStation. this is used at launch - dbus-launch --sh-syntax --exit-with-session so if i ssh in, i get DBUS_SESSION_BUS_ADDRESS='unix:path=/tmp/dbus-UETAp22P3j,guid=1b7e9310552477d681ffe6d365bce197'; export DBUS_SESSION_BUS_ADDRESS; DBUS_SESSION_BUS_PID=6829;

export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket looks to setup pipewire.

dmanlfc commented 6 months ago

@MastaG you didn't provide any ES logs... attach your es_launch_stdout.log when running Steam

MastaG commented 6 months ago

@MastaG you didn't provide any ES logs... attach your es_launch_stdout.log when running Steam

2024-02-02 15:10:26,030 DEBUG (emulatorlauncher:124):start_rom Running system: flatpak
2024-02-02 15:10:26,033 INFO (Emulator.py:85):game_settings_name game settings name: Steam.flatpak
2024-02-02 15:10:26,033 DEBUG (unixSettings.py:21):__init__ Creating parser for /userdata/system/batocera.conf
2024-02-02 15:10:26,034 DEBUG (unixSettings.py:69):loadAll Looking for global.* in /userdata/system/batocera.conf
2024-02-02 15:10:26,034 DEBUG (unixSettings.py:69):loadAll Looking for controllers.* in /userdata/system/batocera.conf
2024-02-02 15:10:26,034 DEBUG (unixSettings.py:69):loadAll Looking for flatpak.* in /userdata/system/batocera.conf
2024-02-02 15:10:26,034 DEBUG (unixSettings.py:69):loadAll Looking for flatpak.folder["/userdata/roms/flatpak"].* in /userdata/system/batocera.conf
2024-02-02 15:10:26,034 DEBUG (unixSettings.py:69):loadAll Looking for flatpak["Steam.flatpak"].* in /userdata/system/batocera.conf
2024-02-02 15:10:26,034 DEBUG (unixSettings.py:69):loadAll Looking for display.* in /userdata/system/batocera.conf
2024-02-02 15:10:26,035 DEBUG (Emulator.py:46):__init__ uimode: Full
2024-02-02 15:10:26,035 DEBUG (unixSettings.py:69):loadAll Looking for flatpak-renderer.* in /userdata/system/batocera.conf
2024-02-02 15:10:26,035 DEBUG (unixSettings.py:69):loadAll Looking for flatpak["Steam.flatpak"]-renderer.* in /userdata/system/batocera.conf
2024-02-02 15:10:26,035 DEBUG (emulatorlauncher:136):start_rom Settings: {'emulator': 'flatpak', 'core': 'flatpak', 'hud_support': False, 'videomode': 'default', 'ratio': 'auto', 'video_threaded': False, 'smooth': True, 'shaderset': 'sharp-bilinear-simple', 'rewind': False, 'bezel': 'consoles', 'forceNoBezel': False, 'gamemode_enable': False, 'video_frame_delay_auto': True, 'controllers.bluetooth.enabled': '1', 'controllers.ps3.enabled': '1', 'controllers.ps3.driver': 'bluez', 'controllers.xarcade.enabled': '1', 'controllers.gpio.enabled': '0', 'controllers.gpio.args': 'map=1,2', 'controllers.db9.enabled': '0', 'controllers.db9.args': 'map=1', 'controllers.gamecon.enabled': '0', 'controllers.gamecon.args': 'map=1', 'controllers.guns.recoil': 'gun-quiet', 'showFPS': 'false', 'uimode': 'Full', 'emulator-forced': False, 'core-forced': False}
2024-02-02 15:10:26,035 DEBUG (emulatorlauncher:138):start_rom emulator: flatpak, core: flatpak
2024-02-02 15:10:26,037 INFO (controllersConfig.py:376):getGamesMetaData looking for game metadata (flatpak, steam)
2024-02-02 15:10:26,038 INFO (emulatorlauncher:157):start_rom guns disabled.
2024-02-02 15:10:26,038 INFO (emulatorlauncher:170):start_rom wheels disabled.
2024-02-02 15:10:26,070 DEBUG (emulatorlauncher:187):start_rom minTomaxResolution
2024-02-02 15:10:26,070 DEBUG (emulatorlauncher:188):start_rom video mode before minmax: 1920x1200.59.95
2024-02-02 15:10:26,201 DEBUG (emulatorlauncher:194):start_rom current video mode: 1920x1200.59.95
2024-02-02 15:10:26,201 DEBUG (emulatorlauncher:195):start_rom wanted video mode: default
2024-02-02 15:10:26,233 DEBUG (emulatorlauncher:207):start_rom resolution: 1920x1200
2024-02-02 15:10:26,233 DEBUG (videoMode.py:74):changeMouse changeMouseMode(True)
2024-02-02 15:10:26,236 DEBUG (emulatorlauncher:482):callExternalScripts calling external script: ['/usr/share/batocera/configgen/scripts/nvidia-workaround.sh', 'gameStart', 'flatpak', 'flatpak', 'flatpak', '/userdata/roms/flatpak/Steam.flatpak']
2024-02-02 15:10:26,237 DEBUG (emulatorlauncher:482):callExternalScripts calling external script: ['/usr/share/batocera/configgen/scripts/powermode_launch_hooks.sh', 'gameStart', 'flatpak', 'flatpak', 'flatpak', '/userdata/roms/flatpak/Steam.flatpak']
2024-02-02 15:10:26,254 DEBUG (emulatorlauncher:482):callExternalScripts calling external script: ['/usr/share/batocera/configgen/scripts/rotation_fix.sh', 'gameStart', 'flatpak', 'flatpak', 'flatpak', '/userdata/roms/flatpak/Steam.flatpak']
2024-02-02 15:10:26,255 DEBUG (emulatorlauncher:482):callExternalScripts calling external script: ['/usr/share/batocera/configgen/scripts/tdp_hooks.sh', 'gameStart', 'flatpak', 'flatpak', 'flatpak', '/userdata/roms/flat

MastaG commented 6 months ago

This is the output of flatpak with dbus-launch.

su - batocera -c "DISPLAY=:0.0 dbus-launch flatpak run com.valvesoftware.Steam -bigpicture"

Note that the directories 

'/userdata/saves/flatpak/binaries/exports/share'
'/userdata/saves/flatpak/data/.local/share/flatpak/exports/share'

are not in the search path set by the XDG_DATA_DIRS environment variable, so
applications installed by Flatpak may not appear on your desktop until the
session is restarted.

F: /userdata/saves/flatpak/data/.cache/.flatpak/com.valvesoftware.Steam/xdg-run/pulse is not a symlink to "../../flatpak/pulse" as expected: readlinkat: Invalid argument
INFO:root:https://github.com/flathub/com.valvesoftware.Steam/wiki
INFO:root:Will set XDG dirs prefix to /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam
DEBUG:root:Checking input devices permissions
INFO:root:Overriding TZ to Europe/Amsterdam
steam.sh[2]: Running Steam on org.freedesktop.platform 23.08 64-bit
steam.sh[2]: STEAM_RUNTIME is enabled automatically
setup.sh[74]: Steam runtime environment up-to-date!
steam.sh[2]: Steam client's requirements are satisfied
tid(108) burning pthread_key_t == 0 so we never use it
[2024-02-02 15:22:47] Startup - updater built Jan 30 2024 18:28:00
[2024-02-02 15:22:47] Startup - Steam Client launched with: '/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/steam' '-no-cef-sandbox' '-bigpicture'
[2024-02-02 15:22:47] Opted in to client beta 'publicbeta' via beta file
You are in the 'publicbeta' client beta.
02/02 15:22:47 Init: Installing breakpad exception handler for appid(steam)/version(1706731428)/tid(108)

(process:4798): flatpak-WARNING **: 15:22:48.010: Failed to connect to bus: Could not connect: No such file or directory
[2024-02-02 15:22:48] Loading cached metrics from disk (/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/package/steam_client_metrics.bin)
[2024-02-02 15:22:48] Using the following download hosts for Public, Realm steamglobal
[2024-02-02 15:22:48] 1. https://client-update.akamai.steamstatic.com, /, Realm 'steamglobal', weight was 1000, source = 'update_hosts_cached.vdf'
[2024-02-02 15:22:48] 2. https://cdn.cloudflare.steamstatic.com, /client/, Realm 'steamglobal', weight was 1, source = 'update_hosts_cached.vdf'
[2024-02-02 15:22:48] 3. https://cdn.steamstatic.com, /client/, Realm 'steamglobal', weight was 1, source = 'baked in'
[2024-02-02 15:22:48] Verifying installation...
[2024-02-02 15:22:48] Verification complete

Steam logging initialized: directory: /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/logs

(process:4798): flatpak-WARNING **: 15:22:48.250: Failed to connect to bus: Could not connect: No such file or directory
XRRGetOutputInfo Workaround: initialized with override: 0 real: 0xf045b8f0
XRRGetCrtcInfo Workaround: initialized with override: 0 real: 0xf045a1c0
steamwebhelper.sh[114]: === Fri Feb  2 15:22:49 CET 2024 ===
steamwebhelper.sh[114]: Starting steamwebhelper under bootstrap sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper

(process:4798): flatpak-WARNING **: 15:22:49.292: Failed to connect to bus: Could not connect: No such file or directory

(process:4798): flatpak-WARNING **: 15:22:49.303: Failed to connect to bus: Could not connect: No such file or directory

(process:4798): flatpak-WARNING **: 15:22:49.308: Failed to connect to bus: Could not connect: No such file or directory

(process:4798): flatpak-WARNING **: 15:22:49.308: Failed to connect to bus: Could not connect: No such file or directory

(process:4798): flatpak-WARNING **: 15:22:49.322: Failed to connect to bus: Could not connect: No such file or directory
CAppInfoCacheReadFromDiskThread took 78 milliseconds to initialize
Steam Runtime Launch Service: starting steam-runtime-launcher-service
Steam Runtime Launch Service: steam-runtime-launcher-service is running pid 212
bus_name=com.steampowered.PressureVessel.LaunchAlongsideSteam

(process:4798): flatpak-WARNING **: 15:22:50.362: Failed to connect to bus: Could not connect: No such file or directory

(process:4798): flatpak-WARNING **: 15:22:50.389: Failed to connect to bus: Could not connect: No such file or directory

(process:4798): flatpak-WARNING **: 15:22:50.967: Failed to connect to bus: Could not connect: No such file or directory

(process:4798): flatpak-WARNING **: 15:22:51.561: Failed to connect to bus: Could not connect: No such file or directory
steamwebhelper.sh[232]: === Fri Feb  2 15:22:59 CET 2024 ===
steamwebhelper.sh[232]: Starting steamwebhelper under bootstrap sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper
steamwebhelper.sh[309]: === Fri Feb  2 15:23:09 CET 2024 ===
steamwebhelper.sh[309]: Starting steamwebhelper under bootstrap sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper
src/steamUI/steamuisharedjscontroller.cpp (545) : Failed creating offscreen shared JS context
src/steamUI/steamuisharedjscontroller.cpp (545) : Failed creating offscreen shared JS context
02/02 15:23:10 Init: Installing breakpad exception handler for appid(steam)/version(1706731428)/tid(108)
assert_20240202152310_28.dmp[386]: Uploading dump (out-of-process)
/tmp/dumps/assert_20240202152310_28.dmp
BRefreshApplicationsInLibrary 1: 0ms
assert_20240202152310_28.dmp[386]: Finished uploading minidump (out-of-process): success = yes
assert_20240202152310_28.dmp[386]: response: CrashID=bp-a1eebfec-e8f4-4bfa-ab1d-061b62240202
assert_20240202152310_28.dmp[386]: file ''/tmp/dumps/assert_20240202152310_28.dmp'', upload yes: ''CrashID=bp-a1eebfec-e8f4-4bfa-ab1d-061b62240202''
steamwebhelper.sh[398]: === Fri Feb  2 15:23:19 CET 2024 ===
steamwebhelper.sh[398]: Starting steamwebhelper under bootstrap sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper
steamwebhelper.sh[473]: === Fri Feb  2 15:23:29 CET 2024 ===
steamwebhelper.sh[473]: Starting steamwebhelper under bootstrap sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper
src/steamUI/steamuisharedjscontroller.cpp (545) : Failed creating offscreen shared JS context
src/steamUI/steamuisharedjscontroller.cpp (545) : Failed creating offscreen shared JS context
reaping pid: 309 -- unknown
src/steamexe/main.cpp (264) : Assertion Failed: ReapProcess: waitid failed: 'No child processes'. Possibly leaking a zombie.

src/steamexe/main.cpp (264) : Assertion Failed: ReapProcess: waitid failed: 'No child processes'. Possibly leaking a zombie.

02/02 15:23:32 Init: Installing breakpad exception handler for appid(steam)/version(1706731428)/tid(108)
assert_20240202152332_34.dmp[551]: Uploading dump (out-of-process)
/tmp/dumps/assert_20240202152332_34.dmp
assert_20240202152332_34.dmp[551]: Finished uploading minidump (out-of-process): success = yes
assert_20240202152332_34.dmp[551]: response: CrashID=bp-d3b5c59e-88fe-4a40-9d40-3251e2240202
assert_20240202152332_34.dmp[551]: file ''/tmp/dumps/assert_20240202152332_34.dmp'', upload yes: ''CrashID=bp-d3b5c59e-88fe-4a40-9d40-3251e2240202''
steamwebhelper.sh[552]: === Fri Feb  2 15:23:40 CET 2024 ===
steamwebhelper.sh[552]: Starting steamwebhelper under bootstrap sniper steam runtime at /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-sniper

smcv commented 6 months ago

our GUI is minimal

If you want it to support running non-trivial Flatpak apps (Steam or otherwise), then there is a limit to how minimal it can be: it will need xdg-desktop-portal, an xdg-desktop-portal backend (probably xdg-desktop-portal-gtk if you aren't running a mainstream desktop environment), and a working D-Bus session bus that is configured to auto-start those components and Flatpak's own services (flatpak-session-helper and flatpak-portal).

Or, if it's too minimal to provide those, then please make it clear to users that running Flatpak apps or Steam is not going to be supportable.

this is used at launch - dbus-launch --sh-syntax --exit-with-session

If you want that session bus instance to be used for applications, then some shell script needs to eval its output, if that isn't already happening (and components that start later will need to avoid overwriting DBUS_SESSION_BUS_ADDRESS).

export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket looks to setup pipewire.

Pipewire normally uses the session bus, not a system bus pretending to be a session bus. The more points of divergence there are between an ordinary desktop Linux system and this minimal GUI, the fewer apps can be expected to work correctly.

dmanlfc commented 6 months ago

Yes, we need to discuss whether we want to have flatpak support. It was added by a dev who is absent and thus not maintained. Therefore the versions & dependencies are not updated. It's not our core focus. Alas unless someone steps up we may need to drop it.

n2qz commented 6 months ago

If you want that session bus instance to be used for applications, then some shell script needs to eval its output, if that isn't already happening (and components that start later will need to avoid overwriting DBUS_SESSION_BUS_ADDRESS).

We already handle this from the script /usr/bin/emulationstation-standalone which starts our launcher, and it is inherited by other apps launched from ES, such as xterm, emulators, and native game ports.

When launching steam apps we use the script /usr/bin/batocera-steam which executes some variant of DISPLAY=:0.0 flatpak run com.valvesoftware.Steam ... to launch the Steam process.

export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket looks to setup pipewire.

This is injected into the environment via /etc/profile.d/dbus.sh.

I don't know enough about dbus (or about our process for launching steam apps) to know if it would make sense to preserve the original value from the root user, or generate a new DBUS_SESSION_BUS_ADDRESS here instead.

MastaG commented 6 months ago

Yes, we need to discuss whether we want to have flatpak support. It was added by a dev who is absent and thus not maintained. Therefore the versions & dependencies are not updated. It's not our core focus. Alas unless someone steps up we may need to drop it.

My knowledge of dbus / xdg-desktop-portal and how to make it compatible with flatpak from a buildroot perspective is also very limited. However soon the Steam beta with its sniper runtime will be merged into production and then it will fail to start for everyone. So unless we can have someone figuring it all out before the v39 release, we might as well drop or temporary disable flatpak for this release.

In the meantime users can resort to the unofficial betacera.pro scripts if they want to use Steam.

dmanlfc commented 6 months ago

@smcv

/usr/libexec/flatpak-portal -vr
F: load portals from /usr/share/xdg-desktop-portal/portals
F: loading /usr/share/xdg-desktop-portal/portals/gtk.portal
F: portal implementation for gnome
F: portal implementation supports org.freedesktop.impl.portal.FileChooser
F: portal implementation supports org.freedesktop.impl.portal.AppChooser
F: portal implementation supports org.freedesktop.impl.portal.Print
F: portal implementation supports org.freedesktop.impl.portal.Notification
F: portal implementation supports org.freedesktop.impl.portal.Inhibit
F: portal implementation supports org.freedesktop.impl.portal.Access
F: portal implementation supports org.freedesktop.impl.portal.Account
F: portal implementation supports org.freedesktop.impl.portal.Email
F: portal implementation supports org.freedesktop.impl.portal.DynamicLauncher
F: portal implementation supports org.freedesktop.impl.portal.Lockdown
F: Bus acquired, creating skeleton
F: Name lost

(/usr/libexec/flatpak-portal:2775): GLib-GIO-WARNING **: 00:42:38.211: Unexpected reply 2 when releasing name org.freedesktop.portal.Flatpak
F: unreffing portal main ref
F: skeleton finalized, exiting

smcv commented 6 months ago

export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket looks to setup pipewire.

This is injected into the environment via /etc/profile.d/dbus.sh.

I happen to be a dbus maintainer, and this setup seems very strange to me. As I said, the session bus and the system bus are not the same thing, so using one as the other won't work. I would recommend comparing with how D-Bus works in a more typical desktop distribution, and keeping the "operating system" and "user session" contexts separate - even if, in your specific system, they happen to be both running as uid 0?

Unfortunately, the price you pay for building your own minimal/focused distro is that you're responsible for doing all the system integration that would already have been done for you in a less minimal environment. I'm not familiar with buildroot, but if it isn't designed to be used to run desktop software, then you're going to have difficulty integrating desktop software into it without a relatively in-depth understanding of how ordinary desktop distributions work.

I'm sure there are some targeted simplifications that can be made for a single-purpose distribution, but if you're using third-party components in ways that they're not designed for, making that work often requires a deeper understanding of those components than just using them as intended.

flatpak support [...] was added by a dev who is absent and thus not maintained

Unfortunately it sounds as though this feature is not necessarily supportable by your current active team, so removing it might set more realistic expectations for your users.

if it would make sense to preserve the original value from the root user, or generate a new DBUS_SESSION_BUS_ADDRESS here

If a user account is going to run something resembling a desktop session, then that user needs to be running their own session bus, which is not the same as the system bus. Software that is designed for desktop sessions, like Flatpak and Steam, will expect and assume this to be true.

F: Bus acquired, creating skeleton
F: Name lost

(/usr/libexec/flatpak-portal:2775): GLib-GIO-WARNING **: 00:42:38.211: Unexpected reply 2 when releasing name org.freedesktop.portal.Flatpak

Yes, this is never going to work, which means any Flatpak app that relies on the ability to do "sub-sandboxing" (Chromium, Steam, and anything with a Chromium-based web browser engine) is not going to work properly either. Seeing "Name lost" without first seeing "Name acquired" indicates that the service failed to acquire its well-known D-Bus name, which is something that services typically need to do during startup.

This is happening because it's expecting to talk to the same user account's session bus (where owning any name is allowed), but your distribution is telling it to connect to the system bus (where processes aren't allowed to own names unless there is specific configuration to allow it, to avoid an attacker being able to impersonate a system service) and pretend that it was the session bus.

The warning Unexpected reply 2 is consistent with this: flatpak-portal is trying to say "I no longer need this name" during exit, but the dbus-daemon has replied DBUS_RELEASE_NAME_REPLY_NON_EXISTENT (numeric value 2), which means "you already don't have that name, and neither does anyone else".

dmanlfc commented 6 months ago

@smcv yes it seems to be the case. We don't have users to therefore create sessions as the desktop (openbox) is as minimal as possible. Do you know if we can leverage the session bus with a service account or dare I say it, root account?

smcv commented 6 months ago

Do you know if we can leverage the session bus with a service account or dare I say it, root account?

What user ID do Flatpak, Openbox and games run as? Is it root, or is it some other account like user or games?

Whatever uid that is, the way to get a correct session bus is one of these:

run dbus-run-session -- your-session-startup-script where your-session-startup-script is the top level executable of your user session running under your correct X11 DISPLAY, so that your-session-startup-script and all of its child and descendant processes will inherit DBUS_SESSION_BUS_ADDRESS
or eval "$(dbus-launch --exit-with-x11 --sh-syntax)" in your session startup script (assuming it's a shell script)
or run dbus-launch --exit-with-x11 -- your-session-startup-script

Even if you are running the entire GUI and all apps and games as root (which I would not recommend), there is still a distinction made between "the operating system" and "the root user's per-user session" in the design of various components.

Even on a minimal system with no real concept of privilege separation between users, I would recommend having a user account - the name user is conventional - and running the GUI as that user account, rather than as root. You can allow that user to elevate privileges back to root with something like sudo, doas or pkexec if you want to, but running the GUI as the non-root user will remove an unnecessary difference between ordinary Linux desktops and this special-purpose distribution, and make it clearer where the line is drawn between "the OS" and "the session".

smcv commented 6 months ago

In a previous comment, I recommended looking at how "normal" desktop Linux distributions do this.

In Debian, the entry point into X11 sessions when not using systemd involves sourcing all the script fragments in /etc/X11/Xsession.d, which includes these:

https://salsa.debian.org/utopia-team/dbus/-/blob/debian/bookworm/debian/75dbus_dbus-launch?ref_type=heads to start the session bus;
https://salsa.debian.org/utopia-team/dbus/-/blob/debian/bookworm/debian/95dbus_update-activation-env?ref_type=heads to upload any environment variables set by the rest of Xsession.d into the list of environment variables that the session bus will use to start user services like flatpak-portal

For a special-purpose operating system, you would probably want to inline a heavily simplified version of those into your "start the session" entry point.

MastaG commented 6 months ago

Correct me if I'm wrong.

But batocera uses initscripts (so no systemd). First there is a /etc/init.d/S01dbus which starts the dbus service:

# Create needed directories.
[ -d /run/dbus ] || mkdir -p /run/dbus
[ -d /var/lock/subsys ] || mkdir -p /var/lock/subsys
[ -d /tmp/dbus ] || mkdir -p /tmp/dbus

RETVAL=0

start() {
    printf "Starting system message bus: "

    dbus-uuidgen --ensure
    dbus-daemon --system
    RETVAL=$?
    echo "done"
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/dbus-daemon
}
...

Then there's a /etc/init.d/S31emulationstation script, which basically loads two profiles and starts X:

#!/bin/sh
BOOTCONF="/boot/batocera-boot.conf"

. /etc/profile.d/xdg.sh
. /etc/profile.d/dbus.sh

case "$1" in
    start)
        enabled="$(/usr/bin/batocera-settings-get system.es.atstartup)"
        if [ "$enabled" != "0" ]; then
        startx &
...

Contents of /etc/profile.d/xdg.sh:

#!/bin/sh

export XDG_RUNTIME_DIR=/var/run

Contents of /etc/profile.d/dbus.sh:

#!/bin/sh

export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket"

Then there's the /etc/X11/xinit/xinitrc which basically starts openbox and tells it to run emulationstation:

#!/bin/bash

mkdir -p "/var/log"
mkdir -p "/userdata/system/logs"

display_log="/userdata/system/logs/display.log"
nvidia_log="/var/log/nvidia.log"
amd_log="/userdata/system/logs/amd.log"

# hide mouse cursor
unclutter --noevents -b

# disable dpms to prevent screen from blanking
xset -dpms
xset s off

# uncomment to allow coredumps for ES
# ulimit -c unlimited

## [ Prime settings ]
# nvidia
if [ -f "/var/tmp/nvidia.prime" ]; then
    echo "Xrandr Providers:" >> "$nvidia_log"
    xrandr --listproviders >> "$nvidia_log"
    provider_count=$(xrandr --listproviders | grep 'Providers:' | awk '{print $4}')
    # ensure we have 2 or more providers
    if [ "$provider_count" -ge 2 ]; then
        echo "Setting Nvidia Prime environment variables" >> "$nvidia_log"
        # find the Nvidia provider number
        provider_number=$(xrandr --listproviders | awk '/NVIDIA/ {print $2}' | tr -d ':')
        if [ -z "$provider_number" ]; then
            echo "Multiple xrandr providers however none appear to be a NVIDIA device!" >> "$nvidia_log"
        else
            export __NV_PRIME_RENDER_OFFLOAD=$provider_number
            export __VK_LAYER_NV_optimus=NVIDIA_only
            export __GLX_VENDOR_LIBRARY_NAME=nvidia
            echo "Nvidia Prime provider = $provider_number" >> "$nvidia_log"
        fi
    else
        echo "You do not have multiple xrandr providers to support Nvidia Prime!" >> "$nvidia_log"      
    fi
fi

# amd
if [ -f "/var/tmp/amd.prime" ]; then
    echo "Setting AMD Prime environment variable" >> "$amd_log"
    amd_prime_value=$(<"/var/tmp/amd.prime")
    if [ -z "$amd_prime_value" ]; then
        export DRI_PRIME=1
    else
        export DRI_PRIME="$amd_prime_value"
        echo "AMD Prime environment variable set to $amd_prime_value" >> "$amd_log"
    fi
fi

openbox --config-file /etc/openbox/rc.xml --startup "emulationstation-standalone"

I think up until this point everything is running as root.

smcv commented 6 months ago

First there is a /etc/init.d/S01dbus which starts the dbus service:

That's the system bus, not the session bus. They're different.

For reference, in Debian with sysvinit (or in Devuan, which is a Debian fork removing all systemd things), the 1:1 equivalent of this is /etc/init.d/dbus.

export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket"

As previously mentioned, this is wrong, because /run/dbus/system_bus_socket is the system bus, but DBUS_SESSION_BUS_ADDRESS is for the session bus.

Then there's the /etc/X11/xinit/xinitrc which basically starts openbox and tells it to run emulationstation

This script would be a good place to start a dbus-daemon --session for whatever user is running the openbox session. For example you could add, somewhere near the top of that script:

eval "$(dbus-launch --exit-with-x11 --sh-syntax)"

and that would be the equivalent of how this works in e.g. Debian or Devuan.

(For comparison, in Debian, /etc/X11/xinit/xinitrc sources /etc/X11/Xsession which sources files in /etc/X11/Xsession.d, which include the ones I linked in https://github.com/batocera-linux/batocera.linux/issues/10948#issuecomment-1929969914. In a more-special-purpose, less-configurable, more-minimal operating system like this one, it makes sense to remove a few layers of indirection.)

I think up until this point everything is running as root.

If that's how this operating system has been designed, then, fine. It isn't how I would personally have done it, but I'm sure it can be made to work.

If the X11 session in this OS runs as root, then the thing to compare with in a more conventional desktop Linux like Debian (or Devuan) would be what happens if you log in to something like nodm or xdm as root (assuming the system is configured to allow it).

export XDG_RUNTIME_DIR=/var/run

This is weird too. XDG_RUNTIME_DIR is another thing that is meant to be scoped like part of a user's session, not part of the operating system. For the XDG_RUNTIME_DIR belonging to root (uid 0), /run/user/0 or /var/run/user/0 would be conventional.

MastaG commented 6 months ago

Thanks a lot @smcv you're a real expert on this. I can confirm that eval "$(dbus-launch --exit-with-x11 --sh-syntax)" at least starts it properly. I'll integrate these changes a bit further on and report back my findings.

dmanlfc commented 6 months ago

It's already done in emulationstation-standalone

MastaG commented 6 months ago

It's already done in emulationstation-standalone

Yes but it misses the dbus-update-activation-environment --verbose --all I'm now building an image with.

/etc/profile.d/xdg.sh removed
/etc/profile.d/dbus.sh removed

Changed:

diff --git a/package/batocera/core/batocera-system/batocera-system.mk b/package/batocera/core/batocera-system/batocera-system.mk
index fd5c2a8024..b8c4babdb0 100644
--- a/package/batocera/core/batocera-system/batocera-system.mk
+++ b/package/batocera/core/batocera-system/batocera-system.mk
@@ -102,8 +102,6 @@ define BATOCERA_SYSTEM_INSTALL_TARGET_CMDS

        # variables
        mkdir -p $(TARGET_DIR)/etc/profile.d
-       cp $(BR2_EXTERNAL_BATOCERA_PATH)/package/batocera/core/batocera-system/xdg.sh $(TARGET_DIR)/etc/profile.d/xdg.sh
-       cp $(BR2_EXTERNAL_BATOCERA_PATH)/package/batocera/core/batocera-system/dbus.sh $(TARGET_DIR)/etc/profile.d/dbus.sh

        # list of modules that doesnt like suspend
        mkdir -p $(TARGET_DIR)/etc/pm/config.d
diff --git a/package/batocera/emulationstation/batocera-emulationstation/batocera-emulationstation.mk b/package/batocera/emulationstation/batocera-emulationstation/batocera-emulationstation.mk
index 81d4b489bb..db391ed8ff 100644
--- a/package/batocera/emulationstation/batocera-emulationstation/batocera-emulationstation.mk
+++ b/package/batocera/emulationstation/batocera-emulationstation/batocera-emulationstation.mk
@@ -153,6 +153,8 @@ endif

 define BATOCERA_EMULATIONSTATION_XORG
        $(INSTALL) -D -m 0755 $(BR2_EXTERNAL_BATOCERA_PATH)/package/batocera/emulationstation/batocera-emulationstation/xorg/xinitrc $(TARGET_DIR)/etc/X11/xinit/xinitrc
+       $(INSTALL) -D -m 0755 $(BR2_EXTERNAL_BATOCERA_PATH)/package/batocera/emulationstation/batocera-emulationstation/xorg/00dbus_dbus-launch $(TARGET_DIR)/etc/X11/Xsession.d/00dbus_dbus-launch
+       $(INSTALL) -D -m 0755 $(BR2_EXTERNAL_BATOCERA_PATH)/package/batocera/emulationstation/batocera-emulationstation/xorg/01dbus_update-activation-env $(TARGET_DIR)/etc/X11/Xsession.d/01dbus_update-activation-env
 endef

 define BATOCERA_EMULATIONSTATION_WAYLAND
diff --git a/package/batocera/emulationstation/batocera-emulationstation/emulationstation-standalone b/package/batocera/emulationstation/batocera-emulationstation/emulationstation-standalone
index 121282b47d..bf75712c7c 100755
--- a/package/batocera/emulationstation/batocera-emulationstation/emulationstation-standalone
+++ b/package/batocera/emulationstation/batocera-emulationstation/emulationstation-standalone
@@ -158,9 +158,6 @@ do
     # save the current output in which es is started (used to switch screen, when unplugged)
     batocera-resolution currentOutput > "/var/run/switch_screen_current"

-    # dbus launch is required for the gio/gvfs/trash:///...
-    eval "$(dbus-launch --sh-syntax --exit-with-session)"
-
     cd /userdata # es need a PWD
     %BATOCERA_EMULATIONSTATION_PREFIX% emulationstation ${GAMELAUNCHOPT} --exit-on-reboot-required %BATOCERA_EMULATIONSTATION_ARGS% ${CUSTOMESOPTIONS}

/etc/X11/Xsession.d/00dbus_dbus-launch

# $Id:$

STARTDBUS=
DBUSLAUNCH=/usr/bin/dbus-launch

if [ -z "$DBUS_SESSION_BUS_ADDRESS" ] && [ -x "$DBUSLAUNCH" ]; then
  STARTDBUS=yes
fi

if [ -n "$STARTDBUS" ]; then
  # Note that anything that is D-Bus-activated between here and
  # 95dbus_update-activation-env will not have the complete environment
  # set up by Xsession.d, unless the Xsession.d snippet that sets the
  # environment variable also calls dbus-update-activation-environment.
  # See <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815503>
  eval "$($DBUSLAUNCH --exit-with-session --sh-syntax)"
fi

/etc/X11/Xsession.d/01dbus_update-activation-env

# vim:set ft=sh sw=2 sts=2 et:

if [ -n "$DBUS_SESSION_BUS_ADDRESS" ] && \
    [ -x "/usr/bin/dbus-update-activation-environment" ]; then
  # subshell so we can unset environment variables
  (
    # unset login-session-specifics
    unset XDG_SEAT
    unset XDG_SESSION_ID
    unset XDG_VTNR

    # tell dbus-daemon --session
    # to put the Xsession's environment in activated services'
    # environments
    dbus-update-activation-environment --verbose --all
  )
fi

Let's see what happens.

EDIT: Now that I'm thinking, we launch Steam by doing: su - batocera -c "DISPLAY=:0.0 flatpak run com.valvesoftware.Steam -bigpicture"

I don't think su will start an Xession for user "batocera".

smcv commented 6 months ago

If you are running Flatpak as user ID batocera, then you need to be running a dbus-daemon --session as user ID batocera - possibly in addition to running one as root (each uid needs its own session bus). For example you could do something like this:

 su - batocera -c 'eval "$(dbus-launch --exit-with-x11 --sh-syntax)"; DISPLAY=:0.0 flatpak run com.valvesoftware.Steam -bigpicture'

But sharing one X11 display between root processes and non-root processes is unusual, and will break some applications. The normal Linux distribution thing to do is to run system services and maybe the Xorg server as root, but then run the whole GUI session (openbox, D-Bus session bus, applications, games, Flatpak, Steam, anything else that uses X11) as a non-root user. In Debian, all of /etc/X11/Xsession and /etc/X11/Xsession.d would normally run as the non-root user.

Like I said before, the more you diverge from how normal Linux distributions work, the fewer things you can expect to work correctly.

MastaG commented 6 months ago

Yeah I think it should be something like:

su - batocera -c 'eval "$(dbus-launch --exit-with-x11 --sh-syntax)"; dbus-update-activation-environment --verbose --all; DISPLAY=:0.0 flatpak run com.valvesoftware.Steam -bigpicture'

MastaG commented 6 months ago

@smcv

Feb  7 13:07:09 batocera dbus-daemon[278]: [system] Rejected send message, 3 matched rules; type="method_call", sender=":1.23" (uid=0 pid=4887 comm="dbus-update-activation-environment --verbose --all") interface="org.freedesktop.DBus" member="UpdateActivationEnvironment" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus)

The system bus seems to reject the dbus-update-activation-environment command issued by Xsession.d/01dbus_update-activation-env.

MastaG commented 6 months ago

Alright, I've modified the xinitrc to read:

--- a/package/batocera/emulationstation/batocera-emulationstation/xorg/xinitrc
+++ b/package/batocera/emulationstation/batocera-emulationstation/xorg/xinitrc
@@ -7,6 +7,33 @@ display_log="/userdata/system/logs/display.log"
 nvidia_log="/var/log/nvidia.log"
 amd_log="/userdata/system/logs/amd.log"

+STARTDBUS=
+DBUSLAUNCH=/usr/bin/dbus-launch
+
+if [ -z "$DBUS_SESSION_BUS_ADDRESS" ] && [ -x "$DBUSLAUNCH" ]; then
+  STARTDBUS=yes
+fi
+
+if [ -n "$STARTDBUS" ]; then
+  eval "$($DBUSLAUNCH --exit-with-session --sh-syntax)"
+fi
+
+if [ -n "$DBUS_SESSION_BUS_ADDRESS" ] && \
+    [ -x "/usr/bin/dbus-update-activation-environment" ]; then
+  # subshell so we can unset environment variables
+  (
+    # unset login-session-specifics
+    unset XDG_SEAT
+    unset XDG_SESSION_ID
+    unset XDG_VTNR
+
+    # tell dbus-daemon --session
+    # to put the Xsession's environment in activated services'
+    # environments
+    dbus-update-activation-environment --verbose --all
+  )
+fi
+
 # hide mouse cursor
 unclutter --noevents -b

And then I'm trying to start Steam from ssh shell:

su - batocera -c 'DISPLAY=:0.0 eval "$(dbus-launch --exit-with-x11 --sh-syntax)"; DISPLAY=:0.0 dbus-update-activation-environment --verbose --all; DISPLAY=:0.0 flatpak run com.valvesoftware.Steam -bigpicture'

The output of reads:

Session lifetime based on X11 requested, but X11 initialization failed.
dbus-update-activation-environment: setting DISPLAY=:0.0
dbus-update-activation-environment: setting SHELL=/bin/bash
dbus-update-activation-environment: setting PWD=/userdata/system
dbus-update-activation-environment: setting LOGNAME=batocera
dbus-update-activation-environment: setting HOME=/userdata/system
dbus-update-activation-environment: setting TERM=xterm-256color
dbus-update-activation-environment: setting USER=batocera
dbus-update-activation-environment: setting SHLVL=1
dbus-update-activation-environment: setting PATH=/usr/local/bin:/bin:/usr/bin
dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment
INFO:root:https://github.com/flathub/com.valvesoftware.Steam/wiki
INFO:root:Will set XDG dirs prefix to /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam
DEBUG:root:Checking input devices permissions
INFO:root:Overriding TZ to Europe/Amsterdam
steam.sh[2]: Running Steam on org.freedesktop.platform 23.08 64-bit
steam.sh[2]: STEAM_RUNTIME is enabled automatically
setup.sh[74]: Steam runtime environment up-to-date!
steam.sh[2]: Steam client's requirements are satisfied
tid(108) burning pthread_key_t == 0 so we never use it
[2024-02-07 16:17:46] Startup - updater built Jan 13 2024 00:51:43
[2024-02-07 16:17:46] Startup - Steam Client launched with: '/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/steam' '-no-cef-sandbox' '-bigpicture'
02/07 16:17:46 Init: Installing breakpad exception handler for appid(steam)/version(1705108172)/tid(108)

(process:11853): flatpak-WARNING **: 16:17:46.700: Failed to connect to bus: Could not connect: No such file or directory
Looks like steam didn't shutdown cleanly, scheduling immediate update check
[2024-02-07 16:17:46] Loading cached metrics from disk (/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/package/steam_client_metrics.bin)
[2024-02-07 16:17:46] Failed to load cached hosts file (File 'update_hosts_cached.vdf' not found), using defaults
[2024-02-07 16:17:46] Using the following download hosts for Public, Realm steamglobal
[2024-02-07 16:17:46] 1. https://cdn.steamstatic.com, /client/, Realm 'steamglobal', weight was 1, source = 'baked in'
[2024-02-07 16:17:46] Checking for update on startup
[2024-02-07 16:17:46] Checking for available updates...
[2024-02-07 16:17:46] Downloading manifest: https://cdn.steamstatic.com/client/steam_client_ubuntu12
[2024-02-07 16:17:46] Manifest download: send request
[2024-02-07 16:17:46] Manifest download: waiting for download to finish
[2024-02-07 16:17:47] Manifest download: finished
[2024-02-07 16:17:47] Download skipped: /client/steam_client_ubuntu12 version 1705108172, installed version 1705108172, existing pending version 0
[2024-02-07 16:17:47] Nothing to do
[2024-02-07 16:17:47] Verifying installation...
[2024-02-07 16:17:47] Performing checksum verification of executable files
[2024-02-07 16:17:47] Verification complete

Steam logging initialized: directory: /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/logs

XRRGetOutputInfo Workaround: initialized with override: 0 real: 0xf06698f0
XRRGetCrtcInfo Workaround: initialized with override: 0 real: 0xf06681c0
steamwebhelper.sh[116]: Runtime for steamwebhelper: defaulting to /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-heavy
steamwebhelper.sh[116]: Running under Flatpak, disabling sandbox
steamwebhelper.sh[116]: CEF sandbox already disabled
CAppInfoCacheReadFromDiskThread took 0 milliseconds to initialize

(process:11853): flatpak-WARNING **: 16:17:48.300: Failed to connect to bus: Could not connect: No such file or directory
...

So it seems the first command DISPLAY=:0.0 eval "$(dbus-launch --exit-with-x11 --sh-syntax)" returns:

Session lifetime based on X11 requested, but X11 initialization failed.

smcv commented 6 months ago

dbus-launch needs X11 to already be up, and I don't think combining environment variables with shell builtins is working the way you expect it to. You could try

su - batocera -c 'eval "$(DISPLAY=:0.0 dbus-launch --exit-with-x11 --sh-syntax)"; DISPLAY=:0.0 dbus-update-activation-environment --verbose --all; DISPLAY=:0.0 flatpak run com.valvesoftware.Steam -bigpicture'

su - batocera -c 'export DISPLAY=:0.0 && eval "$(dbus-launch --exit-with-x11 --sh-syntax)" && dbus-update-activation-environment --verbose --all && flatpak run com.valvesoftware.Steam -bigpicture'

Sorry, I cannot spend my working time on debugging your custom operating system or teaching you more about shell script.

smcv commented 6 months ago

The system bus seems to reject the dbus-update-activation-environment command

That's expected. Changing the activation environment on the system bus would be an easy root privilege escalation vulnerability, so it's specifically blocked, as a mitigation against third-party packages that configure the system bus insecurely (like in CVE-2014-8148).

dbus-update-activation-environment always connects connects to the session bus, so it will normally work as intended, except on systems that have been (mis?)configured to pretend that the system bus is also a session bus.

MastaG commented 6 months ago

su - batocera -c 'eval "$(DISPLAY=:0.0 dbus-launch --exit-with-x11 --sh-syntax)"; DISPLAY=:0.0 dbus-update-activation-environment --verbose --all; DISPLAY=:0.0 flatpak run com.valvesoftware.Steam -bigpicture'

Sorry about that, I was too quick but setting DISPLAY=:0.0 before the actual dbus-launch command works :)

dbus-update-activation-environment: setting DISPLAY=:0.0
dbus-update-activation-environment: setting SHELL=/bin/bash
dbus-update-activation-environment: setting PWD=/userdata/system
dbus-update-activation-environment: setting LOGNAME=batocera
dbus-update-activation-environment: setting HOME=/userdata/system
dbus-update-activation-environment: setting TERM=xterm-256color
dbus-update-activation-environment: setting USER=batocera
dbus-update-activation-environment: setting SHLVL=1
dbus-update-activation-environment: setting PATH=/usr/local/bin:/bin:/usr/bin
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/tmp/dbus-5KKAe47oeC,guid=69895a20a010dd8ab7b5cafa65c3a5ff
dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment

Note that the directories 

'/userdata/saves/flatpak/binaries/exports/share'
'/userdata/saves/flatpak/data/.local/share/flatpak/exports/share'

are not in the search path set by the XDG_DATA_DIRS environment variable, so
applications installed by Flatpak may not appear on your desktop until the
session is restarted.

INFO:root:https://github.com/flathub/com.valvesoftware.Steam/wiki
INFO:root:Will set XDG dirs prefix to /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam
DEBUG:root:Checking input devices permissions
INFO:root:Overriding TZ to Europe/Amsterdam
steam.sh[2]: Running Steam on org.freedesktop.platform 23.08 64-bit
steam.sh[2]: STEAM_RUNTIME is enabled automatically
setup.sh[74]: Steam runtime environment up-to-date!
steam.sh[2]: Steam client's requirements are satisfied
tid(108) burning pthread_key_t == 0 so we never use it
[2024-02-07 16:47:11] Startup - updater built Jan 13 2024 00:51:43
[2024-02-07 16:47:11] Startup - Steam Client launched with: '/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/steam' '-no-cef-sandbox' '-bigpicture'
02/07 16:47:11 Init: Installing breakpad exception handler for appid(steam)/version(1705108172)/tid(108)

(process:14988): flatpak-WARNING **: 16:47:11.301: Failed to connect to bus: Could not connect: No such file or directory
[2024-02-07 16:47:11] Loading cached metrics from disk (/userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/package/steam_client_metrics.bin)
[2024-02-07 16:47:11] Failed to load cached hosts file (File 'update_hosts_cached.vdf' not found), using defaults
[2024-02-07 16:47:11] Using the following download hosts for Public, Realm steamglobal
[2024-02-07 16:47:11] 1. https://cdn.steamstatic.com, /client/, Realm 'steamglobal', weight was 1, source = 'baked in'
[2024-02-07 16:47:11] Verifying installation...
[2024-02-07 16:47:11] Verification complete

Steam logging initialized: directory: /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/logs

(process:14988): flatpak-WARNING **: 16:47:11.426: Failed to connect to bus: Could not connect: No such file or directory
XRRGetOutputInfo Workaround: initialized with override: 0 real: 0xf0f098f0
XRRGetCrtcInfo Workaround: initialized with override: 0 real: 0xf0f081c0
steamwebhelper.sh[114]: Runtime for steamwebhelper: defaulting to /userdata/saves/flatpak/data/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/steam-runtime-heavy
steamwebhelper.sh[114]: Running under Flatpak, disabling sandbox
steamwebhelper.sh[114]: CEF sandbox already disabled
CAppInfoCacheReadFromDiskThread took 0 milliseconds to initialize

(process:14988): flatpak-WARNING **: 16:47:12.245: Failed to connect to bus: Could not connect: No such file or directory

(process:14988): flatpak-WARNING **: 16:47:12.252: Failed to connect to bus: Could not connect: No such file or directory

(process:14988): flatpak-WARNING **: 16:47:12.254: Failed to connect to bus: Could not connect: No such file or directory

(process:14988): flatpak-WARNING **: 16:47:12.274: Failed to connect to bus: Could not connect: No such file or directory
mmap() failed: Cannot allocate memory
mmap() failed: Cannot allocate memory

(process:14988): flatpak-WARNING **: 16:47:12.320: Failed to connect to bus: Could not connect: No such file or directory

(process:14988): flatpak-WARNING **: 16:47:12.321: Failed to connect to bus: Could not connect: No such file or directory
Steam Runtime Launch Service: starting steam-runtime-launcher-service
Steam Runtime Launch Service: steam-runtime-launcher-service is running pid 243

(process:14988): flatpak-WARNING **: 16:47:12.831: Failed to connect to bus: Could not connect: No such file or directory
bus_name=com.steampowered.PressureVessel.LaunchAlongsideSteam

(process:14988): flatpak-WARNING **: 16:47:12.850: Failed to connect to bus: Could not connect: No such file or directory

(process:14988): flatpak-WARNING **: 16:47:13.437: Failed to connect to bus: Could not connect: No such file or directory

(process:14988): flatpak-WARNING **: 16:47:13.836: Failed to connect to bus: Could not connect: No such file or directory
sh: line 1: lsb_release: command not found
sh: line 1: lsb_release: command not found

(process:14988): flatpak-WARNING **: 16:47:14.040: Failed to connect to bus: Could not connect: No such file or directory
[2024-02-07 16:49:12] Background update loop checking for update. . .
[2024-02-07 16:49:12] Checking for available updates...
[2024-02-07 16:49:12] Downloading manifest: https://cdn.steamstatic.com/client/steam_client_ubuntu12?t=3509651005
[2024-02-07 16:49:12] Manifest download: send request
[2024-02-07 16:49:12] Manifest download: waiting for download to finish
[2024-02-07 16:49:13] Manifest download: finished
[2024-02-07 16:49:13] Download skipped: /client/steam_client_ubuntu12?t=3509651005 version 1705108172, installed version 1705108172, existing pending version 0
[2024-02-07 16:49:13] Nothing to do

Now dbus-update-activation-environment is setting DBUS_SESSION_BUS_ADDRESS which is nice. I'll continue to try out the Steam beta update tomorrow :)

MastaG commented 6 months ago

Opened up a PR with the WIP changes: https://github.com/batocera-linux/batocera.linux/pull/11007 Still needs testing.

Gugus-1 commented 6 months ago

Thanks for all the work you're doing... It also doesn't work for me...

MastaG commented 6 months ago

Well the whole dbus sysstem and user session seems to be setup correctly. However flatpak doesn't seem to cooperate with me...

I have a /etc/profile.d/dbus.sh with:

#!/bin/sh

STARTDBUS=
DBUSLAUNCH=/usr/bin/dbus-launch

if [ -z "$DBUS_SESSION_BUS_ADDRESS" ] && [ -x "$DBUSLAUNCH" ]; then
  STARTDBUS=yes
fi

if [ -n "$STARTDBUS" ]; then
  eval "$($DBUSLAUNCH --exit-with-session --sh-syntax)"
fi

if [ -n "$DBUS_SESSION_BUS_ADDRESS" ] && \
    [ -x "/usr/bin/dbus-update-activation-environment" ]; then
  # subshell so we can unset environment variables
  (
    # unset login-session-specifics
    unset XDG_SEAT
    unset XDG_SESSION_ID
    unset XDG_VTNR

    # tell dbus-daemon --session
    # to put the Xsession's environment in activated services'
    # environments
    dbus-update-activation-environment --verbose --all
  )
fi

if [ -z "$XDG_RUNTIME_DIR" ]; then
  export XDG_RUNTIME_DIR=/var/run
fi

Because it's in /etc/profile.d/ it will automatically run when I login using ssh:

root@192.168.0.17's password: 

      ____    __   ____  _____  ___  ____  ____    __   
     (  _ \  /__\ (_  _)(  _  )/ __)( ___)(  _ \  /__\  
      ) _ < /(__)\  )(   )(_)(( (__  )__)  )   / /(__)\ 
     (____/(__)(__)(__) (_____)\___)(____)(_)\_)(__)(__)
                 R E A D Y   T O   R E T R O

-- type 'batocera-check-updates' to check for stable branch --
-- add 'butterfly' switch to check for latest arch developments  --

Model: ROG_STRIX_B650E_I_GAMING_WIFI
System: Linux 6.7.2
Architecture: x86_64
CPU Model: AMD Ryzen 7 7800X3D 8-Core Processor
CPU Cores: 
CPU Max Frequency: 5050 MHz
CPU Features: avx2 sse4_1
Temperature: 39°C
Available Memory: 29688/31306 MB
Display Resolution: 1920x1200
Display Refresh Rate: 59.95Hz
Data Partition Format: ext4
Data Partition Available Space: 3,4T
Network IP Address: 192.168.0.17
OS version: 39cp 2024/02/07 22:49

dbus-update-activation-environment: setting SHELL=/bin/bash
dbus-update-activation-environment: setting SSH_AUTH_SOCK=/tmp/dropbear-a203fc00/auth-46f59db5-6
dbus-update-activation-environment: setting EDITOR=/bin/vi
dbus-update-activation-environment: setting PWD=/userdata/system
dbus-update-activation-environment: setting LOGNAME=root
dbus-update-activation-environment: setting HOME=/userdata/system
dbus-update-activation-environment: setting LANG=nl_NL.UTF-8
dbus-update-activation-environment: setting SSH_CONNECTION=192.168.0.19 35306 192.168.0.17 22
dbus-update-activation-environment: setting TERM=xterm-256color
dbus-update-activation-environment: setting USER=root
dbus-update-activation-environment: setting SHLVL=0
dbus-update-activation-environment: setting PS1=[\u@\h $PWD]\$ 
dbus-update-activation-environment: setting SSH_CLIENT=192.168.0.19 35306 22
dbus-update-activation-environment: setting PATH=/bin:/sbin:/usr/bin:/usr/sbin
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/tmp/dbus-melUEx6I6G,guid=c82072fcdc4b3eb43a11d81a65c40f7d
dbus-update-activation-environment: setting SSH_TTY=/dev/pts/1
dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment

So far so good, now lets run /usr/libexec/flatpak-portal -vr:

[root@BATOCERA /userdata/system]# /usr/libexec/flatpak-portal -vr
F: load portals from /usr/share/xdg-desktop-portal/portals
F: loading /usr/share/xdg-desktop-portal/portals/gnome-keyring.portal
F: portal implementation for gnome
F: portal implementation supports org.freedesktop.impl.portal.Secret
F: loading /usr/share/xdg-desktop-portal/portals/gtk.portal
F: portal implementation for gnome
F: portal implementation supports org.freedesktop.impl.portal.FileChooser
F: portal implementation supports org.freedesktop.impl.portal.AppChooser
F: portal implementation supports org.freedesktop.impl.portal.Print
F: portal implementation supports org.freedesktop.impl.portal.Screenshot
F: portal implementation supports org.freedesktop.impl.portal.Notification
F: portal implementation supports org.freedesktop.impl.portal.Inhibit
F: portal implementation supports org.freedesktop.impl.portal.Access
F: portal implementation supports org.freedesktop.impl.portal.Account
F: portal implementation supports org.freedesktop.impl.portal.Email
F: portal implementation supports org.freedesktop.impl.portal.ScreenCast
F: portal implementation supports org.freedesktop.impl.portal.RemoteDesktop
F: portal implementation supports org.freedesktop.impl.portal.Lockdown
F: portal implementation supports org.freedesktop.impl.portal.Background
F: portal implementation supports org.freedesktop.impl.portal.Settings
F: portal implementation supports org.freedesktop.impl.portal.Wallpaper
F: loading /usr/share/xdg-desktop-portal/portals/gnome-shell.portal
F: portal implementation for gnome
F: portal implementation supports org.freedesktop.impl.portal.Access
F: Bus acquired, creating skeleton
F: Name acquired

Now open a second terminal spawn bash in the steam container for some diagnostics:

[root@BATOCERA /userdata/system]# flatpak run --command=bash com.valvesoftware.Steam

Note that the directory 

'/userdata/saves/flatpak/data/.local/share/flatpak/exports/share'

is not in the search path set by the XDG_DATA_DIRS environment variable, so
applications installed by Flatpak may not appear on your desktop until the
session is restarted.

[root@BATOCERA /userdata/saves/flatpak/data]# grep instance-id /.flatpak-info
instance-id=206647625

[root@BATOCERA /userdata/saves/flatpak/data]# flatpak-spawn -- grep instance-id /.flatpak-info
(process:10238): flatpak-WARNING **: 00:19:38.618: Failed to connect to bus: Could not connect: No such file or directory
Can't find bus: Fout bij ontvangen van gegevens: Verbinding is weggevallen

[root@BATOCERA /userdata/saves/flatpak/data]# env | grep -i dbus
DBUS_FATAL_WARNINGS=0
DBUS_SYSTEM_BUS_ADDRESS=unix:path=/run/dbus/system_bus_socket
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/flatpak/bus

Flatpak seems to set a system and session bus address but when running the flatpak-spawn command I'm seeing the same errors as when Steam is running.

taleteller commented 5 months ago

Well 3 weeks have passed and now its on productive! steam no longer starts via batocera.

MastaG commented 5 months ago

Yes because the new runtime was merged into stable.

smcv commented 5 months ago

Try running Steam with flatpak run -vv to find out what D-Bus session bus the flatpak process thinks it's meant to be using. You will find that it's running an xdg-dbus-proxy or flatpak-dbus-proxy process that hopefully connects to your real session bus outside the Flatpak sandbox, and offers a socket for Steam (and flatpak-spawn) to connect to inside the sandbox.

smcv commented 5 months ago

dbus-update-activation-environment: setting (some stuff)

I'm surprised not to see a DISPLAY here. If the flatpak-portal can't inherit a valid DISPLAY from the activation environment, then this is not going to work.

smcv commented 5 months ago

Because it's in /etc/profile.d/ it will automatically run when I login using ssh

Are you starting one D-Bus session bus for the GUI session, and another one when you log in using ssh? If yes, then anything you do in the ssh session is not going to help the GUI session: they're living in different worlds.

I would, again, suggest looking at how this fits together in a more typical Linux distribution.

lucadellasantina commented 5 months ago

I am having the same issue with the latest v39. Steam used to start and after the steamclient was autoupdated it cannot start anymore due to steamwebhelper component.

taleteller commented 5 months ago

I wonder, since Steam is broken in all versions, is this issue having devs attention? The proposed pull request seem not to solve it and I see no activity.

pycage commented 5 months ago

Based on the information from this discussion and a bit of trial and error I was able to build a workaround using /userdata/system/custom.sh alone to make Steam work again. Maybe it could be of help for implementing a proper fix.

This workaround was made for Batocera 36.

Edit /userdata/system/custom.sh and reboot:

#!/bin/bash

mkdir -p /run/user/1000
chown batocera /run/user/1000

cat >/usr/bin/with-dbus-session <<EOF
#! /bin/bash
unset DBUS_SESSION_BUS_ADDRESS
export XDG_RUNTIME_DIR=/run/user/1000
eval \`dbus-launch --sh-syntax\`
dbus-update-activation-environment --all
\$*
kill \$DBUS_SESSION_BUS_PID
EOF
chmod 755 /usr/bin/with-dbus-session

mkdir -p /userdata/system/batocerahome
chown batocera /userdata/system/batocerahome
sed -i /etc/passwd -e "s/1000::\/userdata\/system:/1000::\/userdata\/system\/batocerahome:/"

sed -i /usr/lib/python3.10/site-packages/configgen/generators/flatpak/flatpakGenerator.py -e "s/flatpak run/with-dbus-session flatpak run/"

Basically, I made user batocera use a dedicated home directory and wrapped the flatpak startup in a script to create a temporary D-Bus session bus. This makes Steam start and launch games again. For some reason, the Steam system menu is not rendered in bigpicture mode anymore on my system, but this may be totally unrelated. You can still navigate it blindly, though.

Remember that these are temporary runtime modifications to the root system and batocera-save-overlay would make them permanent. So be careful with batocera-save-overlay.

schlick7 commented 5 months ago

The above edit to custom.sh does not work for me on Batocera 39. Also tried uninstalling, rebooting, and then reinstalling and it froze on "Unpacking Steam Linux Runtime container"

pycage commented 5 months ago

Batocera 39 uses Python 3.11 instead of Python 3.10. So the path to flatpakGenerator.py on the last line needs to be adjusted. Everything else remains the same.

Here's custom.sh for Batocera 39 (and other versions using Python 3.11):

#!/bin/bash

mkdir -p /run/user/1000
chown batocera /run/user/1000

cat >/usr/bin/with-dbus-session <<EOF
#! /bin/bash
unset DBUS_SESSION_BUS_ADDRESS
export XDG_RUNTIME_DIR=/run/user/1000
eval \`dbus-launch --sh-syntax\`
dbus-update-activation-environment --all
\$*
kill \$DBUS_SESSION_BUS_PID
EOF
chmod 755 /usr/bin/with-dbus-session

mkdir -p /userdata/system/batocerahome
chown batocera /userdata/system/batocerahome
sed -i /etc/passwd -e "s/1000::\/userdata\/system:/1000::\/userdata\/system\/batocerahome:/"

sed -i /usr/lib/python3.11/site-packages/configgen/generators/flatpak/flatpakGenerator.py -e "s/flatpak run/with-dbus-session flatpak run/"

Please note: This is for starting the Steam client from the "Ports" system group. If you want to directly launch games from the "Steam" system group, you may have to patch /usr/bin/batocera-steam the same way I did for flatpakGenerator.py.