search

ValveSoftware / steam-for-linux

Issue tracking for the Steam for Linux beta client
4.26k stars 175 forks source link

Steam Client startup stalls when large number games installed. #7117

Open RockafellaJaz opened 4 years ago

RockafellaJaz commented 4 years ago

Your system information

There is a lot of disk usage and then the client just exits with no error messages in the terminal.

Looking inside the client steam directory I can see that the client seems to be rapidly updating the registry.vdf file by creating two others that contain the current date and time in the name.

Manually removing all the 'appmanifest_.acf' files from the library folder allows the client to start. Adding back some of the files say 50 at a time and then starting the client, increases the startup time and disk access but the client does eventually load, until I reach about 200 games then there is a long wait with a lot of disk access and the steam-client just exits with no error.

Only way around this seems to be to remove the appmanifest_.acf files, load the client and then tell it to re-discover all my games. This as you can imagine takes a long time but the client does not crash and successfully adds all the games to registry.vdf. If I then exit the client and try to load it again the same thing happens, lots of disk accessing then the client exits with no error messages.

I have tried the beta client. I have tried deleting the steam directory and allow it to rebuild it. I have also tried using a separate HOME environment All have the same issue.

Output from terminal:

Running Steam on linuxmint 19.3 64-bit STEAM_RUNTIME is enabled automatically Pins up-to-date! Steam client's requirements are satisfied /media/linux-games/steam-library/steam-home/.steam/ubuntu12_32/steam [2020-05-16 22:31:08] Startup - updater built May 15 2020 03:04:07 [2020-05-16 22:31:09] Verifying installation... [2020-05-16 22:31:09] Verification complete STEAM_RUNTIME_HEAVY: ./steam-runtime-heavy

Steps for reproducing this issue:

  1. Launch Steam
  2. Add library containing more than 200 installed games.
longshanks197 commented 4 years ago

Thanks @RockafellaJaz for the quality find regarding the library size. I can confirm that this appears to by my issue as well (Issue #7115). I also exceed 200 games in my library and after I moved my appmanifest files the client started. I restored a number of appmanifest files and I was ok up to about 480 games but 500 was too many. Please mark #7115 as related to this issue. Thanks!

bugsbony commented 4 years ago

Good news, I found a work-around.

I have a lot of games installed, and mainly use btrfs as filesystem on slow hard disks, I started encountering this issue with a beta a few weeks ago, I switched to the non-beta, and of course the issue came back with this new version a few days ago.

I spent some time investigating the problem, in particular using inotifywait -m, and noticed that steam was doing a LOT of writing to ~/.steam/registry.vdf and related temporary files like registry.vdf20200402203737.tmp.swap and registry.vdf20200402203737.

I moved the .steam folder to a ramdisk (only that folder, not the sub-folders, thanks to symlinks), updated the symlinks, and steam not only launched, but it was VERY fast (~10s to the logging window). I tested putting .steam on a ext4 filesystem and it worked too, but it was not as fast (~150s) so i fear with even more installed games it might reach the limit which seemed to be a bit more than 3 minutes (?). I also tested putting it on a ssd with btrfs, it took 60s.

So it seems the problem is the far too many accesses to ~/.steam/registry.vdf and what appears to be a time limit before steam aborts.

This is really unnecessary, that just 1 file should be written so many times. It is only this file, all the other files haven't moved, the only other files on the ramdisk are steam.pid, steam.token, steam.pipe, and the symlinked folders.

RockafellaJaz commented 4 years ago

I can confirm that it does indeed seem to be btrfs that is causing the problem, I had completely forgot I had moved my steam home onto a drive with a btrfs file system. I copied the registry.vdf file to one of my partitions with an ext4 file system and put a symlink to it in the .steam folder on the btrfs drive. Steam started up no problem at all except with a long delay as it does its unnecessary writes. Thanks for the idea, I will put the file into a ramdisk to speed up loading.

Glad I don't have it on an SSD with all those file writes wasting my drive.

RockafellaJaz commented 4 years ago

For anyone interested. This is how I moved registry.vdf into ram to speed up steam loading until this issue is fixed. Assuming steam directory is in the home folder.

bugsbony commented 4 years ago

Looking back at the beta updates, it is pretty clear that the issue comes from "Fixed registry.vdf sometimes becoming corrupt after a forced system reboot" from the May 8th beta. My guess is that they added a sync after each write, but the result is bad. I've checked and there is 3 CLOSE_WRITE of registry.vdf and 9 MOVED_TO registry.vdf for every single installed game, all this for adding these 6 lines per installed game to registry.vdf:

"gameID"
{
        "Installed"             "1"
        "Updating"              "0"
        "Running"               "0"
}

And of course, even those that are able to launch steam are still getting up to 3 minutes of additional startup time since this update.

Seegras commented 4 years ago

I can confirm; this is the issue, I can make steam run by moving my registry.vdf into ram, as per @RockafellaJaz workaround. In my case it's not three minutes. It would probably be around an hour.

nonno-cicala commented 4 years ago

I have the same problem with 265 games installed, even on EXT4. I worked around the problem putting registry.vdf on a ramdisk, on both beta client and the regular one. Thanks @RockafellaJaz for identifying the problem and a good workaround :+1:

Newbytee commented 4 years ago

I seem to have something similar to this. Steam doesn't crash, but there's extremely severe input lag (i.e. takes multiple seconds to react) in the entirety of Steam, both web components and non-web components (such as the top bar and the About Steam dialogue). This seems to mostly go away if I wait for a while.

RockafellaJaz commented 4 years ago

@Newbytee That sounds like a separate issue as I am not experiencing anything like that.

Does a program like htop show any high CPU usage when the input lag happens? Does this only happen after you have installed a lot of games? Does setting the Library Settings to Low Bandwidth / Low Performance improve it any? Have you tried using a separate fresh steam install? Create a folder, open a terminal at its location, then run:

You can test it out in there then just delete the folder once you are finished.

If it still persists I would create a new issue.

jcnils commented 3 years ago

Your system information: 

Steam client version (build number or date): 1603992987

Distribution (e.g. Ubuntu): Manjaro 20.3  (Gnome)

Opted into Steam client beta?: Yes and No (I tried with both)

Have you checked for system updates?: Yes

CPU: AMD Ryzen 2700x

Kernel Version: 5.9.8-2 (I also tried with 5.8 and 5.7)

Video Card: 2080TI

Video Card: Driver Version: 455.xx

RAM: 32 GB

The weirdest thing is when I click to open steam top shows 90~95% idle. And it takes about 11 minutes to open Steam. The Steam Library is in another nvme ssd, ext4, mounted in my /home, as in /home/name/Games/SteamLibrary It has 68 games consuming 800GB of 2TB storage

I tried a clean install by:

Running sudo pacman -Rns steam-manjaro (I am using steam-manjaro in this example, but native has the same problem)

Deleted the folders and files

/home/name/.local/share/Steam/
/home/name/.steam/
/home/name/.steampath
/home/name/.steampid

I kept the steam library folders, but I deleted a bunch of lost files and folders related to steam.

Then I installed steam again using sudo pacman -Syu steam-manjaro

Steam opens the first time in 15 seconds. I log in and close it. Then I open and close again a few times. Always 7~15 seconds.

I added the Steam Library, it froze steam for a while, but eventually it gets back. Then I closed Steam. It took 11 minutes to open steam.

The time to open Steam is between 5~12 minutes with the Steam Library on.

dar72 commented 3 years ago

Is there any plan to actually fix this? I don't want to be messing around with RAM disks. I use ext4 on a spinning disk, never been a problem before and isn't a problem for anything else. It's mounted with noatime if that makes a difference.

I think I have this issue though my disk usage isn't really very high. If I keep trying to start Steam repeatedly it eventually works. Doesn't seem to take a long time to start when it actually succeeds though I'm probably not the best judge of how fast things are - my opinion on that is dated.

Can't we just increase the timeout value? I don't mind waiting, I'd just rather not have to run "steam" in a terminal 10 times in a row to get it to finally open.

Also, when it does fail, is there a way to disable it creating crash dumps? Seems like it probably slows down the failed attempts?

bugsbony commented 3 years ago

I thought I'd check if anything changed, but sadly no.

I may have far too many games installed, but it takes more than 10 minutes to launch steam. With a script, that moves the .steam folder to a ramdisk and rsync it after it changes, it only takes around 20 seconds. The script only moves the .steam folder, not its subfolders as they are symlinks, the only significant file in that folder is registry.vdf. So all the other steam files are still on a slow hard drive.

It seems around half a second is wasted for every game installed, so even if you have only 20 games installed, you will wait 10 seconds for nothing each time you launch steam. (If steam is installed on a slow hard drive, doesn't matter where the games are installed).

All this because steam does a sync after each change to ~/.steam/registry.vdf, which happens at least once for every single installed game, probably because of a wrapper that simulates the windows registry.

I love steam, but it's really sad that they still haven't fixed this issue.

dar72 commented 3 years ago

Replying to https://github.com/ValveSoftware/steam-for-linux/issues/7117#issuecomment-917143236

For me this seems to be fixed since the re-design of the downloads and library screens

RockafellaJaz commented 3 years ago

I had actually forgotten about this issue as I have been using my script to move the file into RAM for so long.

I executed the following code inside the folder in RAM containing registry.vdf and launched steam. inotifywait -m -q -e create -e delete -e modify .

It recorded a total of 11937 CREATE/MODIFY/DELETE actions during launch. The Storage Manager / Steam Library Folder Screen says I have 1322 games currently installed across 3 drives.

It looks like the problem is still not fixed. This seems like a really unnecessary amount of disk writes that could probably be executed in RAM then written once to disk.

I am currently using Steam BETA -- Package Version 1631310966.

bugsbony commented 3 years ago

It's definitely not fixed, I've retested it on the latest beta 2 days ago, I think there's been a new beta since, but I doubt it was suddenly fixed.

Though it used to crash after something like 3 minutes, it doesn't now, but this might have changed quite some time ago as they changed the title from "crash" to "stalls" last year.

The number of writes wouldn't be much of a problem if it didn't sync every time, which prevents the write cache from doing its job. Syncs are made to make sure the changes is not lost in case of a sudden crash/power loss, but in this case it's totally worthless as these changes (installed/running/updating states of games) are overwritten on startup anyway.

happysmash27 commented 2 years ago

Wow!

So, I had been having a similar issue that linked to this one, where the error was a vague: 

src/common/pipes.cpp (883) : fatal stalled cross-thread pipe.
src/common/pipes.cpp (883) : fatal stalled cross-thread pipe.
src/common/pipes.cpp (883) : Fatal assert; application exiting
src/common/pipes.cpp (883) : Fatal assert; application exiting

Registry.vdf was only 12k, though, so it wouldn't make sense that Steam would have problems with it.

The mention of disk usage did pique my interest, however, since I noticed that Steam stopped working while my disk was pegged at 100%, due to Monero very slowly syncing the blockchain and causing a lot of disk IO on /mnt/MEGA, which is also the disk where most of my Steam games are stored.

So to test if that may be the issue, I closed the Monero daemon, and now Steam launches fine. So, high disk IO on libraries other than the main disk can also cause Steam to fail to start.

kisak-valve commented 2 years ago

Hello, per "More improvements to reduce login time for users with large libraries" in the 2022-11-02 Steam client beta update, please opt into Steam's client beta and retest this issue.

nonno-cicala commented 2 years ago

It doesn't crash any more here, but it takes 1 minute and 10 seconds to start whereas it took 6 seconds with the registry.vdf on a ramdisk, so I'll keep the ramdisk

bugsbony commented 2 years ago

So, my setup has changed a lot since then, upgraded my 10 years old cpu/motherboard/memory, new video card, fiber internet, and my .steam folder is now on a ssd. Though I still have far too many games installed, mostly on a slow disk. I still use my script that puts the ~/.steam/ folder on a ramdisk, but not its subfolders thanks to symlinks, to essentially put all ~/.steam/registry.vdf* creations/modifications on the ramdisk.

I've made a full comparison (times from launch to visible library): ~/.steam/ location stable beta
ramdisk 14s 12s
ssd (sata) 33s 29s
10Tb slow hard disk 3m15 2m53

(in all cases all ~/.steam/ subfolders are on a ssd, I didn't bother changing that)

So there is definitely an improvement, but the most important part is still there: very slow, inefficient, probably slightly harmful to the drive, (and useless) modification of ~/.steam/registry.vdf for every single installed game.

RockafellaJaz commented 2 years ago

I tested on my ext4 HDD and steam did not crash but took 2 minutes 41 seconds to load.

I monitored both my RAM drive and .steam on my HDD using the command: inotifywait -m -q -e create -e delete -e modify . | tee >(ts "%d-%m-%y %H_%M_%S" > ~/STEAM_TEST.txt)

The output files are attached.

The problem is still the registry.vdf file, I currently have over 1000 games installed and during steam startup the file was created and modified over 8000 times. Can this not be accomplished in RAM?

inotify_registry_vdf_hdd.txt inotify_registry_vdf_ram.txt

Dominux commented 10 months ago

Hi, I encounter the issue of slowly or even infinitely loading steam client on both Manjaro and PopOS!. For now I got my newly installed PopOS!. Even tho on Windows at the same machine and drive I got my steam-client loaded just in a second or two

Here's my logs:

steam.sh[375371]: Running Steam on pop 22.04 64-bit
steam.sh[375371]: STEAM_RUNTIME is enabled automatically
setup.sh[375507]: Steam runtime environment up-to-date!
steam.sh[375371]: Steam client's requirements are satisfied
tid(375569) burning pthread_key_t == 0 so we never use it
[2024-01-18 15:51:05] Startup - updater built Jan 13 2024 00:51:43
[2024-01-18 15:51:05] Startup - Steam Client launched with: '/home/dominux/.local/share/Steam/ubuntu12_32/steam'
01/18 15:51:05 Init: Installing breakpad exception handler for appid(steam)/version(1705108172)/tid(375569)
[2024-01-18 15:51:05] Loading cached metrics from disk (/home/dominux/.local/share/Steam/package/steam_client_metrics.bin)
[2024-01-18 15:51:05] Using the following download hosts for Public, Realm steamglobal
[2024-01-18 15:51:05] 1. https://client-update.akamai.steamstatic.com, /, Realm 'steamglobal', weight was 1000, source = 'update_hosts_cached.vdf'
[2024-01-18 15:51:05] 2. https://cdn.cloudflare.steamstatic.com, /client/, Realm 'steamglobal', weight was 1, source = 'update_hosts_cached.vdf'
[2024-01-18 15:51:05] 3. https://cdn.steamstatic.com, /client/, Realm 'steamglobal', weight was 1, source = 'baked in'
[2024-01-18 15:51:05] Verifying installation...
[2024-01-18 15:51:05] Verification complete

Steam logging initialized: directory: /home/dominux/.local/share/Steam/logs

XRRGetOutputInfo Workaround: initialized with override: 0 real: 0xec3059c0
XRRGetCrtcInfo Workaround: initialized with override: 0 real: 0xec3041f0
/usr/share/themes/Pop/gtk-2.0/main.rc:775: error: unexpected identifier 'direction', expected character '}'
/usr/share/themes/Pop/gtk-2.0/hacks.rc:28: error: invalid string constant "normal_entry", expected valid string constant
steamwebhelper.sh[375578]: Runtime for steamwebhelper: defaulting to /home/dominux/.local/share/Steam/ubuntu12_64/steam-runtime-heavy
steamwebhelper.sh[375578]: glibc >= 2.34, partially disabling sandbox until CEF supports clone3()
CAppInfoCacheReadFromDiskThread took 0 milliseconds to initialize
Steam Runtime Launch Service: starting steam-runtime-launcher-service
Steam Runtime Launch Service: steam-runtime-launcher-service is running pid 375705
bus_name=com.steampowered.PressureVessel.LaunchAlongsideSteam
[2024-01-18 15:53:05] Background update loop checking for update. . .
[2024-01-18 15:53:05] Checking for available updates...
[2024-01-18 15:53:05] Downloading manifest: https://client-update.akamai.steamstatic.com/steam_client_ubuntu12?t=3231357305
[2024-01-18 15:53:06] Manifest download: send request
[2024-01-18 15:53:06] Manifest download: waiting for download to finish
[2024-01-18 15:53:06] Manifest download: finished
[2024-01-18 15:53:06] Download skipped: /steam_client_ubuntu12?t=3231357305 version 1705108172, installed version 1705108172, existing pending version 0
[2024-01-18 15:53:06] Nothing to do