search

ValveSoftware / steam-for-linux

Issue tracking for the Steam for Linux beta client
4.22k stars 174 forks source link

Music library update thread using more cpu resources than expected #7372

Closed StatusCode404 closed 3 years ago

StatusCode404 commented 4 years ago

Your system information

Please describe your issue in as much detail as possible:

Describe what you expected should happen and what did happen. Please link any large code pastes as a Github Gist

After starting Steam, the process goes to 100% and stays there constantly and never terminates. It's been doing that since the last update. Restarting or killing the process and restarting does not stop the issue.

Steps for reproducing this issue:

  1. Start Steam or Restart Steam
StatusCode404 commented 4 years ago

Is there any update on this? Anything that I should provide to help resolve this? I killed the process over the weekend but restarted it today. It hovered around 10%-20% for about half-hour. Now it is a flat 100% after running for half a day. What is it doing? image

TTimo commented 4 years ago

To confirm: the Steam client otherwise works (it did not hang during startup for instance). And your issue is that one (or more) of the threads in the steam process are stuck using 100% CPU.

I think there are some open issues in this database that related to this.

StatusCode404 commented 4 years ago

To confirm: the Steam client otherwise works (it did not hang during startup for instance). And your issue is that one (or more) of the threads in the steam process are stuck using 100% CPU.

I think there are some open issues in this database that related to this.

* Is this specific to steam beta, or does the main release client do this as well.

I'll try this next. I suppose all I have to do is opt out ot Beta and the stable version will download?

* Can you identify which thread(s) is stuck. A good way to do this is to use htop: "To enable thread views in htop, launch htop, and press F2 to enter htop setup menu. Choose "Display option" under "Setup" column, and toggle on "Tree view" and "Show custom thread names" options. Presss F10 to exit the setup."

I have pasted the image at the bottom of this message

* Check your `~/.steam/steam/logs` folder for any information that may be relevant. Such as some warnings / messages spamming and constantly repeating?

I opened every single log file in that folder using glogg and enabled "follow" for each. There were no repeating red flags you tend to see with infinite error exceptions in a loop. Half the files were not even update today. Out of those that were updated today, all stopped logging anything after initial bootstrap startup of the client.
The only 3 files that updated after an hour or so basically showed nothing important (no: failures, warnings, errors). The 3 files that updated were appinfo, configstore, and connection logs. All of them just logged normal sort of info stuff. None of them repeating and filling up disk space. All 3 log intervals are large at 15mins (appinfo and configstore) to an hour (connection).

Screenshot from 2020-09-22 10-09-59

StatusCode404 commented 4 years ago 
* Is this specific to steam beta, or does the main release client do this as well.

I have tested with the main release client for a day and it works fine. The high core usage did not happen. It is only happening with the beta.

TTimo commented 4 years ago

Is this due to the friends list? See https://github.com/ValveSoftware/steam-for-linux/issues/7241 - you are not reporting high cpu in steamwebhelper though so I think this is something else.

StatusCode404 commented 4 years ago

Is this due to the friends list? See #7241 - you are not reporting high cpu in steamwebhelper though so I think this is something else.

I don't think so because I'm always offline.

luni3359 commented 4 years ago

I'm not sure if this applies here, but I was experiencing 100% CPU usage as well, and it turned out to be that the Allow background processing of Vulkan shaders option that was making it do so. The problem went away after Steam finished processing the shaders.

StatusCode404 commented 4 years ago

I have that disabled and still experienced the 100%

robobenklein commented 4 years ago

Also seeing this issue on the stable branch: image

Background shader processing is on, but I don't think it's that because it's been stuck at 100% for many hours.

image (Game running 05:00-17:00, I was not at my computer 17:00-03:00 and nothing else was consuming CPU)

Steam built Sep 3 2020, 21:18:20 (package 1599174997)

TTimo commented 4 years ago

@robobenklein can you identify which of your steam threads (by name) is using a full core (it was 2074097 in your screenshot)? Notice that in @StatusCode404's report all the steam threads are idle except the main one.

^ probably incorrect, htop may have more steam threads listed that are not captured in the snapshot. I thought it would group threads before showing child processes, but apparently not.

robobenklein commented 4 years ago

I will try to capture as much info when I encounter the bug again, I restarted steam before making a full stack trace to have it's stdout in a terminal.

leon-matthews commented 4 years ago

I'm also seeing 100% CPU usage on Ubuntu 20.04, stable steam, on the thread CLocalLibraryCr: steam-100-percent Update: The exact same thread is up to 2hrs of CPU time now, I'm going to close Steam down. Note that I'm 'offline' in friends, am not installing or upgrading any games - I've just been working on another project all morning.

TTimo commented 4 years ago

That's the music library crawl. It gets bogged down and very slow, especially when you have titles with high file count installed. Unturned for instance, with 37k files in Maps/Germany/Foliage ..

mlassnig commented 4 years ago

Can confirm. Disabling the music library crawl solved the 100% CPU issue for me.

StatusCode404 commented 4 years ago

Excellent @TTimo !

The latest stable Steam release started doing the same thing of consuming 100% of a core. I switched to Beta again, and same thing!

I removed all paths in the Music settings and disabled "Scan at startup", "Scan Steam folders for soundtracks", and I reset the music database so there were no items.

Voila!!!! no more 100% cpu usage!

Now someone has to bisect the code to figure out why this has suddenly become a problem for stable and beta when it never was before. For now we have a workaround, but it is distastes-full to have disabled functionality because of something wrong with it especially on a stable release!

Further notes, my entire music library is in folder in a mounted btrfs RAID5 volume. The music folder is 104.2 GB with 19,058 items. There are no access permission issues with the folder being owned by me. Also since this has never been a problem before with previous steam versions.

leon-matthews commented 4 years ago

I can confirm that the work-around of disabling scanning music library worked for me. Thanks!

WGH- commented 3 years ago

I noticed that it ventured into Proton directory, which contained a symlink to the root, and basically ended up with infinite recursion.

openat(AT_FDCWD, "/mnt/games/SteamLibrary/steamapps/common/Proton 5.13/dist/share/default_pfx/dosdevices/z:/mnt/games/SteamLibrary/steamapps/common/Proton 5.13/dist/share/default_pfx/dosdevices/z:/mnt/games/SteamLibrary/steamapps/common/Proton 5.13/dist/share/default_pfx/dosdevices/z:/mnt/games/SteamLibrary/steamapps/common/Proton 5.0/dist/share/default_pfx/dosdevices/z:/mnt/games/SteamLibrary/steamapps/common/Proton 5.0/dist/share/default_pfx/dosdevices/z:/mnt/games/SteamLibrary/steamapps/common/Proton 5.0/dist/share/default_pfx/dosdevices/z:/mnt/games/SteamLibrary/steamapps/common/Proton 5.0/dist/share/default_pfx/dosdevices/z:/mnt/games/SteamLibrary/steamapps/common", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC|O_DIRECTORY) = 123

WGH- commented 3 years ago

The scanner keeps reenabling itself for me, by the way.

liskin commented 3 years ago

Same here, the scanner gets infinitely deep:

[pid 3738769] access("/data/tomi/.steam/steamapps/common/Proton 4.11/dist/share/default_pfx/dosdevices/z:/proc/1/root/proc/1/root/proc/1/root/proc/1/root/data/hudba/Children of Bodom/2000 - Hate Me (single)", F_OK) = 0
robobenklein commented 3 years ago

So a temporary workaround could be either disabling following of symlinks in the scan or add a symlink depth cutoff? (I think following a maximum of 2 symlinks would be acceptable.)

Although I normally would not think that anything in the auto-generated wine prefix needs to be searched for music, right?

Plagman commented 3 years ago

Please re-test today's Beta client, thanks!

"Fixed a regression where the client would follow recursively follow symlinks in Proton install folders while discovering music files" in the 2021-02-24 Steam client beta update.

mlassnig commented 3 years ago

@Plagman Thanks!

I added a Proton bundle (just to make sure one exists, judging from the ChangeLog), reset the music database, and restarted. Took a minute or so and my albums were found again, with no excessive CPU usage anymore.

kisak-valve commented 3 years ago

Closing as fixed in the 2021-03-22 Steam client update.

kkartaltepe commented 2 years ago

It seems like this is still an issue with an Oct 13 2021 build, Steam package version: 1634158817. One core was burning 100% cpu until disabling the music library scanning in settings. Perf reported CLocalLibraryCr as the steam stack as someone had reported earlier (though the thread seemed be doing a whole lot of nothing but locking mutexes).

f0o commented 2 years ago

I can confirm that 1634158817 consumes 5 cores while absolutely idle. Music scanning en/disable has no effect.

//edit seems to be the Chat Window that drains 500% CPU. closing the chat window (even with just one chat) removes all CPU load

exa18 commented 2 years ago

confirm, opened only Deck and rised CPU usage to 1/3 (all cores). This issue also prevents display to blank (power manager). Also list steam in htop show over 100 pids open where ps reports only 12-15. ref and also despite of output 14 from ps axjf | grep steam | wc -l I get this: Screenshot_2022-05-05_21-28-02

edenist commented 1 year ago

Just popping in so say this is still an issue on the latest release version of the steam client.

Removing all music library locations and disabling auto-scans resolved the issue for me, but the scan still appears to be getting bogged down somewhere.