search

ValveSoftware / steam-for-linux

Issue tracking for the Steam for Linux beta client
4.26k stars 174 forks source link

Shader Pre-caching are not detected in client beta (2021 Aug 4) #7972

Closed spiffeeroo closed 3 years ago

spiffeeroo commented 3 years ago

Your system information

Please describe your issue in as much detail as possible:

Issue The existing shader pre-caching folders located on additional Library Folders are not detected by Steam. The Steam Shader Pre-caching is listed as 0MB in the client beta instead of the previous value (GB+) in stable client. Every time Steam and game are relaunched, shader pre-caching re-downloads the shaders for the exact same game. New Shader Pre-Caching disk size gets updated, but the value gets reset to 0MB when Steam client beta is restarted.

Expected Steam client beta detects Shader Pre-caching folder on additional Library Folders.

Steps for reproducing this issue:

  1. Use systemd-networkd for networking. (Recently added by me 2021 Sep 1)
  2. Add additional Library Folder and install game.
  3. Enable Shader-Precaching and let it download shader cache for game.
  4. Observe Shader Pre-caching disk size increase in Settings -> Shader Pre-caching.
  5. Switch to Steam client beta (July 31 2021 or newer).
  6. Observe Shader Pre-caching disk usage gets reset to 0MB.
Orognad commented 3 years ago

Same here I have DotA2 in a different partition on a ssd and the game download shaders every day. Shader size changes too, yesteday I had 2.9GB of shaders and today I have 1.9GB. The game downloaded 600MB of precache shaders today.

Fedora 34 steam version latest available for Fedora: Jul 20 2021 Main steam library in a HDD, secondady library only DotA2 in a SSD. Warframe installed in main library but it only downloaded 1.5gb precache content and only DotA2 download prechache content every day. Same problem in Manjaro 21.0.

danginsburg commented 3 years ago

I am having difficulty reproducing this in the Aug 9 2021 Beta build of Steam. I followed the reproducer steps, but when in the Steam beta branch I am still seeing the shader pre-caching data being picked up from the secondary Steam library folder and it is not being re-downloaded. Is it still happening for you in the latest Beta? If so, could you please attach a copy of ~/.steam/steam/logs/shader_log.txt to the bug.

spiffeeroo commented 3 years ago

I am still reproducing this bug on the latest Steam client beta (Aug 11 2021). Here is my ~/.steam/steam/logs/shader_log.txt: https://gist.github.com/spiffeeroo/5eafa088922cdd83c1307850d8360121

What I find strange is the bug seems to manifest on my computer once a large quantity of games/tools ( 50+) are installed to Steam. With less games installed, the bug does not seem to immediately occur. After installing one game too many (i.e. 51 to 52 games), exiting and re-opening Steam causes the 0MB cache problem. Afterward when I uninstall a game, exit and relaunch Steam, Steam downloads the shader cache of all installed game once again. The shader cache size reported by Steam also seems to intermittently flip-flop between 0MB and 2978 MB (in my case at the moment) in a series of launching Steam.

One difference I found between stable and beta version is in ~/.steam/steam/config/libraryfolders.vdf. The beta removes "mounted" "1" from that file. When rolling back from beta to stable, the added library folder and games displayed in Steam is missing. However, the game folders/files still exist on the local filesystem.

MatteAce commented 3 years ago

I incurred in the same bug. The Steam Beta Client is trying to redownload the cache for 103 games (all in external NTFS libraries) each time I reboot Steam. I'm on Manjaro/KDE, all updates done.

kisak-valve commented 3 years ago

Hello, in case there's a pattern, what video card / driver version is everybody using?

MatteAce commented 3 years ago

Hello, in case there's a pattern, what video card / driver version is everybody using?

I'm on Nvidia GTX 1060, drivers are the latest Nvidia drivers installed by Manjaro.

Vladimir-A commented 3 years ago

Hello, in case there's a pattern, what video card / driver version is everybody using?

5700xt / mesa 21.2.1 I confirm that the error appears after installing a large number of games / tools (50+) on Steam.

Willdrick commented 3 years ago

I'm still being hit with this problem, and even reinstalled games on the default library ~/.steam/steam/steamapps/common/ and not only is redownloading the shader cache each reboot, but also it's proccessing it as if it were the first time running the game (on games like Warframe with over 3.5gb worth of shaders it takes a really long time)

Are there any news on this front? Even reverting back to the regular client I'm getting these unwanted downloads.

OS: Fedora 34 (gnome) Linux 5.13.12-201.fsync.fc34.x86_64 Mesa 21.3.0-devel

Steam: Steam Build Aug 20 2021, 20:39:31 Steam Api: V020 Steam Package versions 1629497411

brain-anti-freeze commented 3 years ago

I also have this issue (for a while now). After a game exits it downloads the shaders again in their entirety and re-processes them. It is limited to once per game per Steam client launch however.

I do have multiple libraries (since the recent update that made working with them easier) and this issue occurs for every game regardless of which library it is in.

In this case I've run Days Gone, its downloaded two buckets (8259971 and 8890371). I've relaunched Steam, ran Days Gone again and its downloaded the exact same files again (same size and contents).

[steam@steambox swarm]$ md5sum transcoded_video.foz d08a556cfba82f42a9b22c06901fabe8 transcoded_video.foz [steam@steambox swarm]$ md5sum steam_shader_cache/bucket_8259971.swarc da7f8a051de37d250a022b9784d682ce steam_shader_cache/bucket_8259971.swarc [steam@steambox swarm]$ pkill steam [steam@steambox swarm]$ ls -lh downloads/steam_shader_cache/ total 18M -rwxr-xr-x 1 steam steam 36M Aug 29 15:03 bucket_8259971.swarc [steam@steambox swarm]$ ls -lh downloads/steam_shader_cache/ total 100M -rwxr-xr-x 1 steam steam 161M Aug 29 15:04 bucket_8259971.swarc [steam@steambox swarm]$ ls -lh downloads/steam_shader_cache/ total 422M -rwxr-xr-x 1 steam steam 449M Aug 29 15:10 bucket_8259971.swarc [steam@steambox swarm]$ ls -lh steam_shader_cache/ total 449M -rwxr-xr-x 1 steam steam 449M Aug 29 15:10 bucket_8259971.swarc [steam@steambox swarm]$ md5sum steam_shader_cache/bucket_8259971.swarc da7f8a051de37d250a022b9784d682ce steam_shader_cache/bucket_8259971.swarc [steam@steambox swarm]$ md5sum transcoded_video.foz d08a556cfba82f42a9b22c06901fabe8 transcoded_video.foz

Is this a fault in the content delivery system rather than the shader processing ? I'd expect it to only download deltas for these files (so in this cause the download size would be 0).

content_log.txt has an erroneous message (Warning! Mounted library folder "/mnt/steamdata/SteamLibrary" is read-only), the path is definitely not read-only. And some messages about the downloads which seem to indicate its not using deltas and is downloading the full file again download 0/394044400, store 0/0, reuse 0/0, delta 0/0, stage 0/470773018

It only takes 50 seconds to process the shaders and I think it might be necessary (to detect any new ones captured during that session) but the download seems wrong to me (and it takes much longer, ties up bandwidth). The only error message I see in shader_log.txt is related to the bucket that had the transcoded media file in it, although that file is correctly updated.

[2021-08-29 15:11:25] Processing downloaded cache for AppID 1259420, BucketID 8890371. [2021-08-29 15:11:25] Empty/corrupt bucket depot? AppID 1259420.

content_log.txt

brain-anti-freeze commented 3 years ago

It only takes 50 seconds to process the shaders and I think it might be necessary (to detect any new ones captured during that session)

Actually that's not true, there isn't any point in replaying the downloaded file except after a new version of the file has been downloaded.

spiffeeroo commented 3 years ago

The culprit was due to using systemd-networkd in my case after finding the time to troubleshoot some more. Disabling systemd-networkd and switching to NetworkManager resolved my issues in the Steam beta. I have opened and exited Steam ten times to confirm.

Even after deleting Steam folders located under the /home folder, the problem continued to persist on my desktop computer. I configured Arch manually on this desktop computer including enabling systemd-networkd. Then, I decided to just re-install Arch Linux on my laptop this weekend using the official archinstall script, which somewhat defaults to systemd-networkd. For the rest of the networking, I have systemd-resolved enabled with the stub-resolver symlinked to /etc/resolv.conf and systemd-resolvconf package installed. I had the same problem on this laptop as well. (Additional library folders were not a factor, and it was an incorrect diagnosis on my part.)

On a whim, I disabled systemd-networkd and enabled NetworkManager. Now I no longer have the random 0MB shader cache problem upon switching from stable client to beta. After quitting a game, it no longer immediately re-validates and downloads the shader cache like with systemd-networkd.

The Steam client beta seems to have some networking issues with systemd-networkd. That would explain the peculiar behavior on my computers.

brain-anti-freeze commented 3 years ago

Interesting, I'm on Arch too. I just disabled systemd-networkd and enabled NetworkManager but I'm still seeing the same behaviour, the full cache redownloads after the game exits.

Vthyarilops commented 3 years ago

I'm having the same issue in the last 2 beta clients and use NetworkManager on Ubuntu 18.04, shader cache went from 11gb to 0mb, then went to 880mb and now is at 0mb again, the actual folder is still 880mb, now it doesn't even try to download any cache or process them and is sitting at 0mb, RX580 on Mesa 21.2.1. I switched to stable client and firstly it removed my library so I had to re add that, but once I restarted the client it recognized the 880mb cache and started downloading a couple of caches and is sitting at about 1gb

kisak-valve commented 3 years ago

I might have seen this issue on one of my systems. Looking at shader_log.txt, these look like a couple potentially relevant lines:

[2021-09-05 14:22:33] Crawling complete, found 5 different buckets:
[2021-09-05 14:22:33] We have a hit cache of generation 2, back-end bumped theirs to 4, dropping caches.

Contents of <Library folder>/steamapps/shadercache/ totaled about 14GB while the pre-cache settings read as 0MB. Maybe an old cache is getting retained and re-detected as out of date?

Zorrototo commented 3 years ago

For the sake of it I removed my shadercache folder and started Steam to see if it helps.

Here is my system information, I can confirm that using NetworkManager doesn't help at all as this is what is default on Manjaro KDE. https://gist.github.com/Zorrototo/6921a977a4edd73e1ded44eb7938b25a

Kisak, is there something new in the way shaders are handled now? It downloaded only small chunks for me everytime, nothing bigger than 20MB and sometime only a few hundred KB. Is delta patching for shaders now implemented?

//EDIT: it doesn't change anything, issue still occurs.

kisak-valve commented 3 years ago

Steam shader cache reported a 0 bytes while it is 100+ gigabytes on disk

Issue transferred from https://github.com/ValveSoftware/steam-for-linux/issues/8115. @kakra posted on 2021-09-30T10:52:44:

Your system information

Please describe your issue in as much detail as possible:

I've moved my Steam shader cache to a different partition (from btrfs to xfs), and replaced the original shaders directory with a symlink. When fossilize runs, the symlink is properly resolved to the new path and can be seen in process monitors. Steam actually updates files there as timestamps change, and the symlink hasn't been replaced back to directory by Steam. So everything works as expected.

The xfs partition itself is mounted via automounter (using systemd automounts), so it will be mounted on first access by Steam. This also works properly.

The shader cache itself contains over 100 GB of data according to du -sh.

The reasoning behind this was to decouple normal system IO from shader cache IO by redirecting that to a dedicated disk with a simpler file system than btrfs. It actually makes a difference for system responsiveness, so this experiment proved useful.

I'm not sure if this was already before due to some Steam-internal changes, or if this is related to the symlink:

I've noticed that Steam no longer shows the size of the shader cache in the settings dialog, it just reports 0 bytes. If shaders are processed after a game quits, a few megabytes of storage are accounted for the shader cache according to settings. Starting other games lets the value increase further. But if I shut Steam down and start it again, the shader cache is back to 0 bytes according to the settings dialog. On disk, it remains at 100+ GB.

I observed this behavior like 1-2 weeks now, so it was present before the latest beta update which makes me think it could be related to the symlink - but that actually doesn't make much sense given how Steam still accounts new shader data to the cache in the settings dialog and loses this value after a restart.

Plagman commented 3 years ago

Can people affected by this issue try today's Client Beta?

kakra commented 3 years ago

Things I noticed with the latest Beta:

Looks like Steam recreates a shader data now - probably due to format changes? Old shader cache data is either not automatically discarded, or still not counted.

Some questions:

brain-anti-freeze commented 3 years ago

Resolved for me with today's beta update

Zorrototo commented 3 years ago

I guess a steam --reset would work to clear the shader cache completely, I tried to click the Clear Download Cache button in Steam settings, and after Steam restart my shader cache went from 3+GB before, and it downloaded a few hundred MB, to 2.5GB, so it removed something but didn't clean the whole cache.

//EDIT: I just tried steam --reset and from Steam settings my shader cache appears to be a few MB, but the actual folder is 5.4GB still.

thecosmicslug commented 3 years ago

Resolved for me with today's beta update

was just about to state that rejoining the beta this morning fixed it for me also!

spiffeeroo commented 3 years ago

This problem is resolved for me as well (while using systemd-networkd or NetworkManager). My desktop and laptop (wireless and ethernet) are no longer affected by the 0MB shader cache bug. Multiple relaunches of Steam still correctly shows the previous shader cache size (currently 8100 MB on my desktop).

My laptop's root file system were btrfs, ext4, and xfs at various points (Arch and Debian 11), so file system type did not seem to be a factor at least for this case.

kakra commented 3 years ago

Still with the beta client, I didn't opt-out meanwhile...

Well, it's not quite fixed for me. While the shader cache now remembers its size seemingly correctly across restarts of the client, it still doesn't account for all the data in the shadercache directory:

It looks like Steam keeps stale or temporary data around it doesn't clean up - or it's not stale but also not accounted for.

Steam also re-downloads the same shader updates each time again on each start. Sometimes just a few, sometimes 30+ games. I'm pretty sure that is shader related because games do not have daily updates - but only a few downloads have the shader download icon in the history. Usually, those are tiny (a few hundert kilobytes up to a few megabytes), only some are huge (multiple gigabytes, those huge downloads usually have the shader download icon).

@kisak-valve This seems all messed up, and I'm no longer sure if the report #8115 I issued is really a duplicate of this.

lucifertdark commented 3 years ago

For the last few months Steam has reported a wildly different size for the pre-cached files to what my file manager reports is in the folder, I'm wondering if we're all seeing a bug that doesn't actually exist? right now Steam says I have 1.2Gb pre-cached & the folder contains 2.8Gb of files, what if that's actually 100% correct & we're mistaken thinking it's a bug? Perhaps a dev can weigh in on this & solve the puzzle?

Zorrototo commented 3 years ago

My shader cache is 3.9GB on disk, 1439MB in Steam, and I cleared the files manually eventually, and reset Steam a dozen days ago https://github.com/ValveSoftware/steam-for-linux/issues/7972#issuecomment-932722574 to try to clear any problem from past updates. Seems like something is weirdly not working as intended.

kakra commented 3 years ago

It makes perfect sense that the actual shader cache may be much smaller, and the storage not accounted for is really temporary or intermediate files. But then Steam should have a proper cleanup function: Most of this has modification times in the last 3 months although I only played a few titles in that period. So either stale data needs to be cleaned up automatically, or Valve adds a button to manually clean up unused data so I don't have to purge both stale and legit data by deleting the whole folder.

My latest observation seems to show that Steam stores around twice the amount of data than it actually accounts for in the shader pre-caching settings.

Zorrototo commented 3 years ago

Maybe it needs some kind of cache to recreate the shader cache, after significant changes in the system (video drivers update, things like that).

kakra commented 3 years ago

If it's needed, it needs to be accounted for.

But to reply to your theory: Yes, it needs that.

Edited: Moving over to #7533

kisak-valve commented 3 years ago

Hello @kakra, the disk usage discrepancy is being tracked at #7533.

kakra commented 3 years ago

@kisak-valve Thanks for the info. So I'd consider this partly closed for me: Shader caches are detected on start, but Steam re-downloads the same data over and over again with each start which somewhat feels like: "yeah, we detect the amount of data now but we still don't see which data we already have".

kisak-valve commented 3 years ago

Closing as fixed.