Verify Integrity of Game Cache ignores filename capitalisation

[DarkStarSword]

Verify Integrity of Game Cache will claim that all files successfully verify even if some of the filenames have the incorrect case. This may happen under certain circumstances when copying the files off a filesystem that is not case sensitive, such as ntfs or vfat, or when those filesystems have been mounted with certain options (e.g. lowntfs).

For testing, this script can be used to recursively rename all files in a directory to lower case: https://raw.github.com/DarkStarSword/junk/master/steam/breakcase.py

I also have written a script that can rename all the files to the correct case listed in the .manifest files, announcement here: http://steamcommunity.com/app/221410/discussions/0/846941710185740762/

For example:

After running the above breakcase.py script on the Osmos directory, I get the following error when trying to launch it:

Game update: AppID 29180 "Osmos", ProcID 21884, IP 0.0.0.0:0
ERROR: ld.so: object 'gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
/bin/sh: 1: /home/ian/.local/share/Steam/SteamApps/common/Osmos/Osmos: not found
Game removed: AppID 29180 "Osmos", ProcID 21884 
saving roaming config store to 'sharedconfig.vdf'
roaming config store 2 saved successfully

Yet running verify integrity of game cache still reports "All files successfully validated"

If I then rename osmos to Osmos and osmos.bin32 to Osmos.bin32 and launch the game it does launch, but all the graphics are messed up - lots of quadrilateral shapes are drawn on the screen, but no text is drawn and it is impossible to work out what is going on - essentially Osmos can't find it's textures. Running my abovementioned check_acf.py script with --rename then allows Osmos to work properly again.

I would propose that the verify integrity of game cache feature is extended to rename files to correct any capitalisation issues in a similar manner to my check_acf.py script.

There may still be some edge cases if the filesystem that SteamApps is mounted on is case insensitive (such as using mount -t lowntfs instead of mount -t ntfs3g).

Processor Information:
    Vendor:  GenuineIntel
    Speed: 1600 Mhz
    8 logical processors
    4 physical processors
    HyperThreading:  Supported
    FCMOV:  Supported
    SSE2:  Supported
    SSE3:  Supported
    SSSE3:  Supported
    SSE4a:  Unsupported
    SSE41:  Supported
    SSE42:  Supported

Network Information:
    Network Speed:  

Operating System Version:
    Debian GNU/Linux 7.0 (wheezy) (64 bit)
    Kernel Name:  Linux
    Kernel Version:  3.7.0-rc6-ian+
    X Server vendor:  The X.Org Foundation
    X Server release:  11204000

Video Card:
    Driver:  NVIDIA Corporation Quadro FX 880M/PCIe/SSE2

    Driver Version:  3.3.0 NVIDIA 310.19
    Desktop Color Depth: 24 bits per pixel
    Monitor Refresh Rate: 60 Hz
    VendorID:  0x10de
    DeviceID:  0xa3c
    Number of Monitors:  2
    Number of Logical Video Cards:  1
    Primary Display Resolution:  1600 x 1200
    Desktop Resolution: 3200 x 1200
    Primary Display Size: 15.75" x 11.81"  (19.69" diag)
                                            40.0cm x 30.0cm  (50.0cm diag)
    Primary Bus: PCI Express 16x
    Primary VRAM: 1024 MB
    Supported MSAA Modes:  2x 4x 8x 16x 

Sound card:
    Audio device: Conexant CX20585

Memory:
    RAM:  3897 Mb

Miscellaneous:
    UI Language:  English
    LANG:  en_AU.UTF-8
    Microphone:  Not set
    Total Hard Disk Space Available:  413250 Mb
    Largest Free Hard Disk Block:  112581 Mb

Installed software:

Recent Failure Reports:
    Fri Jan  4 03:00:14 2013 GMT: file ''/tmp/dumps/assert_20130104140012_1.dmp'', upload no: ''HTTP response code said error''
    Mon Jan  7 03:30:40 2013 GMT: file ''/tmp/dumps/assert_20130107143038_1.dmp'', upload no: ''HTTP response code said error''

[johndrinkwater]

Had two users in group chat have this problem so far after copying Team Fortress 2 from their Windows library to their Linux one, its a case issue so verification didn’t highlight the problem. Though, like Issue #123 it might just be labelled unsupported.

13:44 - Krywk: solved
13:44 - Krywk: my team fortress 2 folder was called 'team fortress 2'
13:44 - Krywk: Steam was looking for 'Team Fortress 2'
13:44 - Krywk: renamed the folder
13:44 - Krywk: there you go

[DarkStarSword]

I've noticed some oddities when downloading Half-Life (Beta) and Psychonauts in the Linux client where the case of some directories that Steam creates does not match that in the manifest files.

I'm not sure if this is a mismatch within the manifest files or something else, but it certainly makes things... interesting. Let me know if any of this belongs in a new bug report.

Left side is the filename on disk, right side is the capitalisation from 3840_3963299974783782956.manifest:

Psychonauts/documents - Documents Psychonauts/workresource - WorkResources Psychonauts/workresource/fonts - WorkResource/Fonts Psychonauts/workresource/localization - WorkResource/Localization Psychonauts/workresource/pclevelpackfiles - WorkResource/PCLevelPackFiles Psychonauts/workresource/pixelshaders - WorkResource/PixelShaders Psychonauts/workresource/savedgames - WorkResource/SAVEDGAMES Psychonauts/workresource/scripts - WorkResource/Scripts Psychonauts/workresource/sounds - WorkResource/Sounds Psychonauts/workresource/textures - WorkResource/Textures Psychonauts/workresource/textures/icons - WorkResource/Icons Psychonauts/workresource/vertexshaders - WorkResource/VertexShaders Psychonauts/workresource/vertexshaders/modules - WorkResource/Modules

Half Life is a bit more interesting since it has multiple manifest files with contradicting capitalisation:

Half-Life/platform/Resource 3_252432840369628186.manifest lists resource with a lower case r 8_3125558230937844363.manifest lists Resource with a capital R Half-Life/platform/Steam 3_252432840369628186.manifest lists steam with a lower case s 8_3125558230937844363.manifest lists Steam with a capital S

My script isn't explicitly looking for these kind of contradictions, so there may be others that it didn't pick up.

[Half-Shot]

Bumping this as steam cloud also seems to remove capitalization which obviously on other operating systems doesn't matter. Cloud saving on Dust : An Elysian Tail is broken as a result.

[Cheeseness]

This seems to also seems to be an issue when receiving updates to games. Within the past 6 months or so, I've had two games (Crypt of the NecroDancer and Grim Fandango Remastered) have filename case changes which weren't propagated to my Linux game cache. In both of these cases, it resulted in game breaking behaviour.

I see that this issue was marked as assigned over a year ago. Has there been any progress with getting it fixed in the client?

[CoolSilver]

Filesystem space issues due to this bug. Dual booting and sharing game library folder creates duplicated files with different case in Linux even though they were updated in Windows. Windows sees only the one file but the file system knows the other case mismatched file exists. Storage space is being mishandled and used when it shouldn't be. Dual boot SteamOS likely will have similar issues.

[DarkStarSword]

@CoolSilver sharing a library between Windows and Linux is not recommended - for some games it works OK, but others had issues if the same files exist in both Windows and Linux depots with differing content causing them to require updates whenever switching OS.

It is reasonable to copy the Windows install to a separate Linux library (or vice versa) to reduce redownloading identical files though, with the caveat that the filenames from Windows may not have the correct case (hence my check_acf.py script to fix them).

[Cheeseness]

It's a bit late to mention it, but this bug wreaked havoc with Firewatch's launch. It allowed a filename case change to the binary to slip under the radar, and resulted in users being unable to play for several hours while we tried to work out a nice solution that wouldn't trigger an entire re-download of the game for Linux users ( we couldn't find one ).

[Cheeseness]

The Living Dungeon also hit this issue during Linux testing. The developer also claims that at some point, the case of files in the depot unexpectedly changed between build uploads.

[Cheeseness]

Is this addressed by yesterday's client update?

I don't have anything that I can specifically test to confirm with.

[Plagman]

That change only had to do with launch configurations, not SteamPipe depots, sorry.

[Cheeseness]

No probs! Hope this can still get addressed at some point in the future.

[virtualdxs]

Just hit this when upgrading from the Proton version of Lethal League Blaze to the native port. steamapps/common/LLBlaze/LLBLAZE_Data/ was incorrectly named steamapps/common/LLBlaze/LLBlaze_Data/ causing the application to crash immediately failing to load libmono.so. Uninstalling and reinstalling was the necessary fix.

[TerryCavanagh]

Hey, developer of Dicey Dungeons chiming in here just to say that this affected us on our last update too. (I searched the issues list and this looks like the right place to post this, apologies if not)

The exact thing that happened was:

• We added a new asset to the game in our v1.8 update, ui/gamepad/A_button.png.
• The exact asset the game was actually looking for was ui/gamepad/a_button.png, lowercase a, doh.
• On Windows and Mac, this worked fine because those OS don't care about case sensitivity. But on linux - yep, hard crash.
• So, we quickly updated the game to change the asset name to a_button.png, lowercase.
• However, the update on linux did not update the file name. Integrity check didn't fix it either. To fix the bug, users had to completely uninstall and reinstall the game.

Hope this can get fixed!

[dsalt]

I just hit this one with Starward Rogue. I noticed when the game complained File Images/Tilesets/GoldShip/SubTilesets/ElegantTechnology/FloorDecorations/Lights2_Dict does not exist.

I had a look at the installed files and found RuntimeData/Images/Tilesets/GoldShip/SubTilesets/ElegantTechnology/FloorDecorations/Lights2_dict.png. One letter different, and it's a case error.

SteamDB's page for its game data depot lists that file as Lights2_Dict.png, so I did a quick verification of the installed files; no errors found despite the filename error.

I ended up renaming the file.

I then put the full file list for that depot into the X11 primary selection then did this:

$ xclip -o|cut -d' ' -f1|xargs -d\\n ls -d >/dev/null
ls: cannot access 'RuntimeData/Images/Achievements/PerfectClearBoss_Battleswarm.png': No such file or directory
ls: cannot access 'RuntimeData/Images/Achievements/PerfectClearBoss_Battleswarm_Disabled.png': No such file or directory
ls: cannot access 'RuntimeData/Images/Achievements/PerfectClearBoss_WallMaster.png': No such file or directory
ls: cannot access 'RuntimeData/Images/Achievements/PerfectClearBoss_WallMaster_Disabled.png': No such file or directory
$ ls RuntimeData/Images/Achievements/PerfectClearBoss_{Battle,Wall}*
RuntimeData/Images/Achievements/PerfectClearBoss_BattleSwarm.png
RuntimeData/Images/Achievements/PerfectClearBoss_BattleSwarm_Disabled.png
RuntimeData/Images/Achievements/PerfectClearBoss_Wallmaster.png
RuntimeData/Images/Achievements/PerfectClearBoss_Wallmaster_Disabled.png

I expect that those four names as listed on SteamDB are correct, but I've not (yet) seen anything here which indicates either way.

These case errors should be reported when verifying.

Ideally, Steam would list the replaced/new files, and also list any unexpected files so that the user can clean them up if needed. But if it can detect names which differ only in case from what's expected (and, evidently, it can), it should be fixing that automatically.