Library Case Sensitivity Issue

[AlurianNighthawk]

Your system information

• Steam client version (build number or date): build 1/4/19
• Distribution (e.g. Ubuntu): Ubuntu 18.04
• Opted into Steam client beta?: No
• Have you checked for system updates?: Yes
• Filesystem: ZFS

Please describe your issue in as much detail as possible:

Recently played an old game (Jedi Knight Academy) using Proton. It created an all lowercase set of folders similar to my library name: windowssteam as opposed to using my library WindowsSteam.

On my next load of steam, my library was removed and i was unable to re-add it. Steam complained of the filesystem not having executable permissions (wrong). Deleting the incorrect windowssteam folder that was added caused steam to magical recognize my library again.

This leads me to believe that libraries are not case-sensitive as they are in the underlying filesystem.

Repeatable Test Case

zfs create -o casesensitivity=insensitive zfs-data/Steam/TestCase zfs set quota=2000G zfs-data/Steam/TestCase (since we apparently can't have more than 2TB of data) Add zfs-data/Steam/TestCase as a Library install game (I chose Defy Gravity) to the TestCase Library mkdir zfs-data/Steam/testcase restart steam library Test Case fails to load rmdir zfs-data/Steam/testcase restart steam library TestCase loads correctly

Proposed solution: it would be nice if Steam was case-sensitive in regards to library folders.

[jarrard]

I think it is case sensitive atm, it should be the other way around, NOT case sensitive so that it can find correct path and files without getting confused with duplicates which should not exist for a proton/windows game if you at all have a clue.

Example, KCD and DRG are windows games which need manually copying of DLL files to exe path, this is likely happening because the game is obeying case sensitivity when looking for the Shared dll, but windows doesn't really care, folder and Folder are the same thing, so often a game might just look for folder and not Folder...

There are countless examples, problem is I don't know if proton/steam can affect a games case sensitivity for finding files/folders.. possibly it can't.

[AlurianNighthawk]

I think if it was case sensitive it would have opened WindowsSteam as specified rather than the first directory that had a similar name.

Note that all lowercase would be the first option (and the one it incorrectly picked): ~/tmpdir touch hello ~/tmpdir touch HelLo ~/tmpdir ls -l total 1.0K -rw-rw-r-- 1 alurian alurian 0 Jan 28 19:19 hello -rw-rw-r-- 1 alurian alurian 0 Jan 28 19:19 HelLo

[jarrard]

so your talking about library folders on your ext4 linux partition, while I'm talking about the issue within the game folder itself. We are talking about different things.

[AlurianNighthawk]

Haha, yeah I just realized that and was about to edit.

Yes, in reference to proton, perhaps it should do a non-case-sensitive match before trying to create directories.

My frustration was more in Steam having an epileptic fit when it couldn't find its library anymore.

[craig-sanders]

You're running zfs, so if your steam files are in their own dataset, you can use 'zfs set' to make that dataset case-insensitive. If they're not already in their own dataset, you can make one and move them.

e.g. my steam folder is the zfs dataset "hex/var/games/steam" (hex is the name of the zpool), mounted as /var/games/steam and owned by my uid/gid. I also created a symlink ~/.steam pointing to /var/games/steam.

To set it to case insensitive, I would run:

zfs set casesensitivity=insensitive hex/var/games/steam

I haven't done this because I haven't run into any case-sensitivity problems yet ....so I don't know whether it would solve your problem or not. It's worth a try. You can always set it back to "sensitive" if it doesn't help.

WARNING: DO NOT SET THIS ON OTHER DATASETS. THIS IS ONLY SAFE TO DO ON A DATASET DEDICATED TO WINDOWS GAMES. (or other software that expects a case-insensitive filesystem).

I expect I will have to do this myself, because I used to occasionally have to rename files years ago when I ran steam under WINE. I'm only just starting to migrate all my gaming back to my linux desktop (threadripper 1950x with 64GB), and I need to get a pair of new drives for a zfs mirror so I can transfer my games from the win7 steam box i built out of spare parts a few years back.

Personally, I'd create a case-insensitive dataset just for windows games and add it as a steam library - install windows games there (this should also be shareable via samba with a Windows VM or an actual windows machine). Linux native games should run fine on a standard case-sensitive filesystem (and some may even break on case-insensitive filesystems).

BTW, earlier you said:

I think if it was case sensitive it would have opened WindowsSteam as specified rather than the first directory that had a similar name.

Have you looked in any of the .acf files, or in the .../steamapps/common directory? It's a mess. Some are all caps, some all-lower, some mixed case. And in my experience, some games do stuff like open an all lower-case filename when it's in mixed case on disk....and, worse, different parts of the code may try to open the same file with different upper/lower case combinations . It works on case-insensitive FAT and NTFS, not on most *nix filesystems.

Windows programmers expect a case-insensitive filesystem so are very sloppy about making sure that the case matches. It's never an issue on windows, so it never occurs to them that not consistently using the same case for filenames is important. Steams devs are mostly windows programmers, so they're likely to make the same portability mistakes that other windows programmers do.

The only way to fix this on ext4 and most other linux filesystems without patching the game code is to rename the file and/or create symlinks whenever you run into the problem. ZFS, however, allows you to set particular datasets to be case-insensitive.

[craig-sanders]

Sorry, my mistake, I just checked the zfs man page again: You can't set a dataset to be case-insensitive AFTER it has been created. You have to create it with something like:

zfs create -o casesensitivity=insensitive,mountpoint=/var/games/steam poolname/dataset

[AlurianNighthawk]

I agree that that is likely the correct workaround (haven't had the problem since as I was only playing a single older game for a day).

That being said, I am talking about the way that the steam application itself finds a library folder on disk or doesn't as the case may be. Frankly, in order to screw this up in the first place, they had to introduce case insensitivity somewhere in their code as the underlying filesystem is case sensitive.

The configuration for the library is also case sensitive (config.vdf): "BaseInstallFolder_2" "/media/hdd/Steam/WindowsSteam"

If I had to take a guess without seeing their code, I expect it happens in the function that reads the configuration for the steam library.

That's why I posted an issue. It's a bug. It's Valve's bug.

[craig-sanders]

Sure, the steam client on linux can and should be fixed, but providing a case-insensitive file-system for old windows games isn't just a workaround, it's a fix for programs that work as expected in their native environment but don't work correctly on a case-sensitive file-system. WINE itself does directory scanning to work around this, but it's slow (see https://wiki.winehq.org/Case_Insensitive_Filenames).

One of my favourite old games, F.E.A.R. and its sequels Extraction Point & Perseus Mandate, has its installdir in the .acf files as:

$ grep FEAR *.acf
appmanifest_21090.acf:  "installdir"        "FEAR Ultimate Shooter Edition"
appmanifest_21110.acf:  "installdir"        "FEAR Ultimate Shooter Edition"
appmanifest_21120.acf:  "installdir"        "FEAR Ultimate Shooter Edition"

but the actual directory on disk is all lower-case: .../steamapps/common/fear ultimate shooter edition/. This was not a problem on my win7 steam box and it works fine on Proton too (as it used to on WINE years ago), but I renamed the directory to have the correct case when I copied it to my linux box. There are probably other filename case issues under that sub-directory which WINE works around with directory scanning but at least it doesn't have to scan for the correct installdir every time a file is opened by the game.

[AlurianNighthawk]

Library scanning only appears to be done on load of the Steam application and upon request (e.g. adding a new library). I can intentionally add a folder similar to the library, windowssteam, and continue launching games for the original WindowsSteam until i restart Steam at which point the library will disappear.

Example: zfs create -o casesensitivity=insensitive zfs-data/Steam/TestCase zfs set quota=2000G zfs-data/Steam/TestCase (since we apparently can't have more than 2TB of data) install game mkdir zfs-data/Steam/testcase restart steam library still fails to load

It's a very narrow problem

[craig-sanders]

Yeah, that does seem to be a bug in the linux steam client. It's a subtle problem. Also, zfs-data/Steam/testcase is just a subdir of zfs-data/Steam and is outside of the zfs-data/Steam/TestCase dataset you created (so is unaffected by the case-insensitive setting).

I guess the steam client is trying to avoid case-sensitivity problems and finds the lowercase testcase before TestCase - probably because lowercase sorts before upper.

JKA installed OK for me when I created case-insensitive hex/var/games/steam2, chowned it to my uid/gid, and added it as a steam library. It ran OK too when I set the launch option to __GL_ExtensionStringVersion=17700 %command%, played through to the end of the first mission with no problems.

The weird thing is that, from the example Jedi Knight Academy you gave, it seems that steam itself installed the game into the all-lowercase windowssteam directory rather than WindowsSteam. That could be an error in one of steam's config files, you might be able to find which one with:

cd ~/.steam   # or wherever...
find . -maxdepth 2 -type f -exec grep -i windowssteam {} + | less

(the -maxdepth 2 stops it from grepping the gigabytes of game files themselves)

Another possibility is that the JKA game lower-cased the install directory. Have you tried installing it into zfs-data/Steam/TestCase? I don't know your setup or exactly what you did when installing JKA so I'm not 100% sure that your mkdir artificial test is relevant to real-life usage, or if it's just an example of "you can break anything if you try hard".

Maybe the problem is caused by the fact that you created a MixedCase directory for it, when the default is all lower-case. I've been making all lower-case directories and datasets for steam, so maybe that's why it works for me.

---

I wasn't aware of the 2TB limit. That's a problem for me because I have about 3.5TB of steam games on my win7 box. I guess I'll have to create multiple case-insensitive Steam Library datasets on my 4TB games pool when I get the drives for it.

[AlurianNighthawk]

Yup, JKA could be either a proton issue, a config issue, or just some casetolower function happening in the old code. I probably should file an issue against Proton for that but I've been lazy and haven't taken the time to investigate it (as I only play it once in a blue moon).

I actually played with it (the issue) a bit back when i first filed the issue here. You would think that Steam goes for the lowercase as it should sort lowercase first, but that isn't always the case (for Steam). When I tried to use my 'LinuxSteam' (as linuxsteam) as a test case it continued working just fine (no library drop on restart). So it's actually a bit more frustrating as it ends up being arbitrary and undefined behavior (as to whether or not the library is correctly imported).

My test case game was 'Defy Gravity' which I think i got for free at some point and was only 80-90MB, so quick install. It's native Linux which is why I chose it as the example (of the client having an issue). As it was the only game in the test dataset and native Linux I ruled out Proton/Windows shenanigans.

I'll acknowledge that there are ways to work around the issue and that you wouldn't usually run into this or more people would be reporting it. That being said, I clearly ran across it by accident and would have rather not.

Obviously this is not a deal breaker as you can remove the incorrect directory or cover it up by putting a dataset over it that obscures the case issue. I'm just flagging this because it's an error and at some point it should be fixed.

Heads-up on the 2TB, but I'm relatively sure WINE has the same issue. I can't think of any other reason I had to quota its save directory (Documents/My Games), but I'll grant that I did it some time ago (2 years...longer?).

[h1z1]

Different titles appear to have different degress of success. One other recurring problem is with the download path. It's a temporary directory outside the final install path.

~/.steam/steam/steamapps/downloading/

There was another one for community content. Regarding the quota I've seen installs fail with quotas larger then 200G, possibly hitting another bug with inode size.

The bugs themselves go back literally years. #5955 #4686 etc.

[AlurianNighthawk]

The issue is here about a Linux application (Steam) which is not handling case sensitivity correctly in 1 case (Library detection).

It isn't about proton (Can be replicated with native Linux games).
It isn't about zfs (Unless someone can prove to me that it doesn't happen on other case-sensitive filesystems).
It isn't about windows (God knows we could gripe about windows all day long).

Let's try and keep this clean.

[h1z1]

Let's try and keep this clean.

?

[somebody1234]

i'm having a similar problem (except for DLLs), but... oddly enough:

• everything was working before i... rebooted or something?
• launching through proton cli still works for some reason, but
• launching through steam does not. very odd