Steam constantly pauses after reaching 100% verifying

[hjpaul7]

Your system information

• Steam client version (build number or date): September 25th

• Distribution (e.g. Ubuntu): Arch

• Opted into Steam client beta?: Yes

• Have you checked for system updates?: Yes

• Steam Logs: steam-logs.tar.gz

• GPU: Nvidia

Please describe your issue in as much detail as possible:

Download game (or DLC in this instance). After download complete, steam starts verifying. After reaching 100% the button goes back to "Resume".

Tried:

• Restarting Steam
• Verifying files
• Switch download region
• Clear download cache
• Complete uninstall and then reinstall (80GB)
• Reboot computer
• Try stable

Steps for reproducing this issue:

1. Download
2. Fails to progress after verifying

https://github.com/ValveSoftware/steam-for-linux/assets/63761039/2df22a31-8d77-4cdd-af98-98a6e4f815d8

Edit: I've tried multiple games, they all end in the same result. Eventually one might complete, but I see nothing in the logs to indicate why. Did not have this issue before the new Steam UI build.

Which if I can ask, can Linux users please get a "beta" branch of the build before the new Steam UI. I'm not complaining but I think this is a valid discussion, there are too many broken functions in the current stable and beta build. Please let us use the last well functioning build till the bugs get sorted.

Please.

[kisak-valve]

Hello @hjpaul7, looking at your content.log, it looks like Steam is detecting 3 files that are in an inconsistent state compared to the content servers:

[2023-10-01 22:12:42] AppID 1091500 update changed : Running Update,
[2023-10-01 22:12:42] AppID 1091500 update changed : Running Update,Verifying,
[2023-10-01 22:12:47] Validation: corrupt chunk 7a011d34d3b3aaddb478e4764f916625926b3060 has 99e90e00957c3254abda366a84e04ba143458eb9 at offset 1460509127 (1048576 bytes)
[2023-10-01 22:12:47] Validation: corrupt chunk 02826beeb014ac353e0ca23821afb38397fdeff3 has 860d3cb4b19d4e1a48f773e5c0f9a2f37ba3f456 at offset 1482529223 (1048576 bytes)
[2023-10-01 22:12:53] Validation: 2 chunks corrupt of 6044 total in file "archive\pc\ep1\ep1_1_nightcity.archive"
[2023-10-01 22:12:53] Current download rate: 0.000 Mbps
[2023-10-01 22:13:12] Validation: corrupt chunk b51df834b02457617ab5aab4d24b175757ff082f has a25d182a1ded6c43f9bc14e5b41ccf39c359cc8e at offset 12224806095 (1048576 bytes)
[2023-10-01 22:13:13] Validation: 1 chunks corrupt of 12564 total in file "archive\pc\ep1\ep1_2_gamedata.archive"
[2023-10-01 22:13:13] Validation: corrupt chunk 5bc8afd25e7037eff8f0e77ae52e6873afdd6043 has c75a8b93999a6fe1d000097d434352187946145b at offset 9437184 (1048576 bytes)
[2023-10-01 22:13:13] Validation: corrupt chunk fc43c4bfb69a8fa24f0fd39d3704867ea0085191 has 4d08f29ed7ab12ed0ed089818e32bcb1a37ac1ab at offset 361758720 (1048576 bytes)
[2023-10-01 22:13:15] Validation: 2 chunks corrupt of 1570 total in file "archive\pc\ep1\lang_en_voice.archive"
[2023-10-01 22:13:15] Validation: full scan in "/home/hustin/Games/SteamLibrary/steamapps/downloading/1091500" found 3/26 mismatching files (5.24 MB/22.34 GB) in 29 seconds).
[2023-10-01 22:13:15] AppID 1091500 update canceled : Staged file validation failed (0 missing, 3 corrupt, 0 unreadable) (Corrupt update files)
[2023-10-01 22:13:15] AppID 1091500 update changed : Running Update,Verifying,Stopping,

What filesystem are you using with /home/hustin/Games/SteamLibrary? It wouldn't hurt to do a filesystem check on that partition.

[hjpaul7]

What filesystem are you using with /home/hustin/Games/SteamLibrary? It wouldn't hurt to do a filesystem check on that partition.

@kisak-valve Thank you, totally didn't see that. It's ext4, I'll run a filesystem check on the partition and report back.

[rurigk]

I'm also experiencing similar issue with Apex legends and some other games not sure if its the same problem

I'm also on Arch linux I'm also in a ext4 partition without encryption I tested it in multiple disks with ext4 partitions (nvme, sata3) I already ran fsck in all my disks My logs also have Validation: corrupt chunk instances I also reinstalled the game

It happens for me every time i update the game since al least 3 months When the game updates it doesn't throw any error and marks the game update as completed but when i launch the game the game crashes saying that some file is corrupted, then i verify the game and then y says update file corrupted

One important thing to say is that one of many tries is successfull and updates the game (without corruption) It may be on the second try or on the twentieth try, its not consistent

This are my logs steam-logs.tar.gz

[hjpaul7]

I have ran a filesystem check on the partition and came across no issues. Also ran a check on my other drives, also no issues.

[chill63]

I am on Arch as well and seem to be having the exact same symptoms, mostly with larger games such as Starfield. Others have the issue, but are usually fixed by a re-validation. Checking the content logs, it seems like sometimes it is reporting corrupt files, other times it says "DISK READ ERROR". I've run multiple different filesystem and SSD checking utilities, none of which have had any errors.

[30p87]

I'm on Arch too (linux 6.5.7.arch1-1), with the exact same error. EXT4 with a GPT table on an NVMe and HDD, unencrypted. I ran multiple checks on the NVMe before and now, in the UEFI settings and in the OS. Everything seems fine, the smart data is clean.

At some point CS2 crashed two times in a row, every time it redownloaded content. I reinstalled it, which failed with "Corrupt update files" in the UI. Looking into the logs, it does throw some Detected write gap xx MB in file "y" errors and later, while checking, Validation: x chunks corrupt of z total in file "y" a lot of times, where file y fluctuates randomly and has no correlation to the files failing while installing. So it seems the checking process randomly fails those files for no reason, as the sha1 hash does not change either. The files that are supposedly corrupt are sometimes the same for multiple retries, that could be pure chance tho. All files failing are .vpk files, often csgo/pak01_xxx.vpk or csgo_lv/pak01_xxx.vpk, but sometimes also csgo/maps/xxx.vpk. Installing, checking and playing any other game works perfectly fine.

I reinstalled steam, used another disk as location (also EXT4 on GPT), changed the download location (servers), cleared caches, repaired the library, used another beta version (eg. demo_viewer), tried to use the Windows version via Proton, but everything having to do with CS2 just ... fails.

As the checking process is obviously faulty, in the stable and Beta version of Steam, with any server providing hashes and any game, the only option left seems to be that some update in arch broke functions the validator uses.

To avoid and quickfix such mistakes, an option to just skip the validation and register the game without checks would be nice, as one could manually move the files in steamapps/downloading to steamapps/common, but steam does not recognize a new game this way.

Edit: It seems like other Source games, eg. TF2, are also affected now (apart from it being broken anyway according to protondb). Portal, Portal 2, HL 2, HL 1, all fail. So even GoldSrc games are affected. In contrast, other Games like The Forest, The Stanley Parable and Furry Feet install perfectly fine.

Log containing extensive testing on CS 2: steam-logs.tar.gz Logs with other Source and non-Source games: steam-logs-others.tar.gz

[rurigk]

I found something I think this issue is not from Steam itself but from whatever they use to verify the files

I also confirmed that sha256sum and sha512sum are working incorrectly

This is the result of running sha256sum root_lgnd_skins.opt.starpak where the file root_lgnd_skins.opt.starpak is in the paks/Win64/ directory relative to the game directory

sha256sum root_lgnd_skins.opt.starpak on Archlinux run 4 times

bc0ce2a6e69c03286f3e4054807cb2546b350992300e39ffbb168350de219b0c  root_lgnd_skins.opt.starpak
e135726550713ac1d0e1c699fb03accf1bb3bf844413e393e7a1e7234c4d1d29  root_lgnd_skins.opt.starpak
596f7904cdfb6d43df1898d515ffb9130926575161de17ddf8efc2561e038ac7  root_lgnd_skins.opt.starpak
5c57b7fc9955278748e7db241b7ad0a0c9d5e883feeab469e31bb90ebe62a7f3  root_lgnd_skins.opt.starpak

sha256sum root_lgnd_skins.opt.starpak on Nobara run 4 times

a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737  root_lgnd_skins.opt.starpak
a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737  root_lgnd_skins.opt.starpak
a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737  root_lgnd_skins.opt.starpak
a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737  root_lgnd_skins.opt.starpak

certutil -hashfile root_lgnd_skins.opt.starpak SHA256 on Windows run 4 times

a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737
a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737
a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737
a49f938e6a67f280ee393ffe8a3affe49902b83bfa8933fc21a7224b0441e737

I also tested with a MKV video file bigger than the root_lgnd_skins.opt.starpak file and it also failed some times

@kisak-valve do you have any idea what can be happening?

[30p87]

After looking into @rurigk's findings, I can confirm that after hashing a lot of files (eg. hashing the whole game dir by piping find's output into sha1sum) , many hash programs (sha1sum, sha256sum, sha512sum and md5sum) will show different hashes for some files between runs. Eg. sha1sum of csgo_lv/pak01_003.vpk is 31bfadf56198f647a8bcb7c0c95658cb6c71c67d after running for i in $(find . -type f); do sha1sum $i > /dev/null; done, it's 415af01742efbcbc8a2b9fbc0a13fac7821c87ba, which was a result I got earlier ... for some reason. After running the for command again the result is now e19e215657a8fd1a556fb44b1c1d183c185f7da6, then e34e0e18eb3ae4c2f7b7e60f8a37ace9f627dc8a and after that fb22a989a5e933dde8c5f3d3b394dbfa4fcf5693 ... you get the idea. After copying the file to /tmp for testing purposes, the copy now has a sha1 hash of 0ab79b730e374035821f4a74628a10f7fc2be3b8 - the original is still fb22a989a5e933dde8c5f3d3b394dbfa4fcf5693. And actually, checking with hexdiff there are a few bytes off: On position 63798226 an ff changed to an f7 and 65896338 changed from ff to f7 again. Two very small parts of a huge file, two very minor changes that should not happen, yet they do. And rightfully, any hashing tool shows a difference, causing the steam validator to fail. Why other games don't have this problem? Idk, that's not a problem 1 am me can even think about. But it will be tomorrow.

What is even weirder is that, while a lot of files mentioned in the steam validator log also show up as diffs in my test, not all of them do, and vice versa.

So it seems like it's actually a problem in some part of Arch itself, despite no issue like this being known to my knowledge and no other effects being shown except this one. But again, a problem for future me.

Edit: Now the hashes stay exactly the same among reinstalls, but steam still complains about corrupt chunks.

[rurigk]

I wrote a test in rust to check if openssl had something wrong but i think its not openssl

I tested with a pure rust implementation and with openssl

This is a small file and works as expected

But when i test a bigger file it fails in both algorithms

I'm thinking that it may be the file system but im not sure, i want to try using tmpfs to test but i only have 32 GB of ram

@30p87 I also tried csgo_lv/pak01_003.vpk but no corruption found, can you try with big files of other games or video?

[rurigk]

@30p87 i finally found why it was failing

It was one ram module, even after i tested multiple times with memtest86 everything was ok (idk why) but in the recent days the module started to fail consistently enough

So if your disks are in good state put your ram modules to the maximum speed allowed by your processor specs and test with memtest86 multiple times

[hjpaul7]

it looks like Steam is detecting 3 files that are in an inconsistent state compared to the content servers:

Wouldn't verifying the integrity of the files force the redownload of the corrupted files? After verifying, it goes right back to pausing it.

[christianlx]

Just like to thanks @rurigk for the solution. That was absolutly not my first idea as I didn't have really issues with my computer (sometimes firefox bugs but that's all) I searched for hours disk issues or steam problems and just changing the memory solved this installations failed

[hjpaul7]

Just like to thanks @rurigk for the solution. That was absolutly not my first idea as I didn't have really issues with my computer (sometimes firefox bugs but that's all) I searched for hours disk issues or steam problems and just changing the memory solved this installations failed

@christianlx You replaced the ram or changed the speeds? Mine is already set to their max

[christianlx]

Changing speed did not work I replaced the ram

[30p87]

The issue first occurred shortly after adding two additional sticks of ram. That should've been no problem, and ram tests were all good.

Since a few days, CS2 is again able to be downloaded. After this is it enters a semi-permanent state of trying to update, to no avail. The logs show no sign of corruption, but updates being triggered multiple times for no reason with no progress. Toying around with pausing the (not working) updating and killing steam eventually got it to reliably start, just to immediately stuck at the red VALVe logo. Killing/Stopping the 'game' results in a 5 second freeze of the whole graphical session. (Fuck you NVidia.) Using -vulkan and other launch options suggested on protondb don't work either.

I believe the constant update state, not working and the original issue are still connected in some way. Valve wouldn't manage to create three different issues with one game that worked before, would they?

[hjpaul7]

I'm hesitant to replace the ram, I'm leaning towards something is up with Steam. I have tried to install Like a Dragon Gaiden on three different nvme's, I have ran checks on all three drives and it's the same thing. Once it gets to 100% verifying it goes back to "Resume" at 65.01GB/65.01GB.

[rurigk]

@hjpaul7 Just do a Memtest to check

[hjpaul7]

I'll be damned...I replaced the ram and it worked.