Open PSebs opened 4 years ago
So, the source of the slowdowns on default filesystems is by virtue of SEGA doing a SEGA because making games is hard?
Considering the state of this game at NA's (two) launch(es), I'm somehow not surprised that this would be an issue. But weird how it causes that much havoc on a normal case-sensitive fs - and being the only one that does, at least of my own personal library.
Can also confirm that summoning the Steam overlay causes a likewise stall every time it's used, even on a casefolded directory structure.
Replying to https://github.com/ValveSoftware/Proton/issues/4122#issuecomment-1166541144
The game is trying to load
4faefcffad1ca8a5f1e8fef157e2b542
which doesn't exist anywhere in the game data (thanks SEGA).
I asked someone knowledgeable enough about the game's internals for details on asset loading, it trying to read 4faefcffad1ca8a5f1e8fef157e2b542
could be explained by the game trying to load things that could possibly be there even if it might not be, mostly regarding outfits, but also terrains as well as it seems to load a lower LOD that doesn't exist.
As for whether the game is consistently giving out the correct filepath to read however, is rather unknown, might have to trace the game for longer possibly just to confirm that a patch is feasible.
(The game engine is very much designed with resiliency in the case that an asset goes missing or kaboom, so it might not be a surprise to see little care with it just trying to read files that does not exist)
It's quite possible that 4faefcffad1ca8a5f1e8fef157e2b542
is some piece of Japanese exclusive content that was stripped from the US client - it would take cross referencing the JP client to confirm this.
So, the source of the slowdowns on default filesystems is by virtue of SEGA doing a SEGA because making games is hard?
Considering the state of this game at NA's (two) launch(es), I'm somehow not surprised that this would be an issue. But weird how it causes that much havoc on a normal case-sensitive fs - and being the only one that does, at least of my own personal library.
Can also confirm that summoning the Steam overlay causes a likewise stall every time it's used, even on a casefolded directory structure.
The game has always used this data structure even on the JP client, and it's always been this bad. It never stopped the ARKS Layer team from patching the game data, and we've had the unofficial English patch for far longer then the game has been alive in the US.
However, this stupid file schema is (likely) the main reason that things like patching, installing, and downloading take so long, because of the huge amount of overhead from operating on so many individual files. It takes multiple hours just to allocate, download, and install the game, and it was about as bad on JP as well - so bad that the PSO2 Tweaker uses a heavily multithreaded download engine just to speed the process up.
Of course, we see the results of two inefficiently designed processes colliding with each other here.
It's quite possible that
4faefcffad1ca8a5f1e8fef157e2b542
is some piece of Japanese exclusive content that was stripped from the US client - it would take cross referencing the JP client to confirm this.
It doesn't show up in the JP client, neither does it show up in any of the past clients (including the ancient SEA client nor the Taiwan client). Whatever that file is, SEGA has simply forgotten about it. I assume it might have been from the alpha, as you could find data relating to the game's beta and alpha despite being from 2010 ~ 2012.
Whatever that file is, it's not particularly important - the game tries to open hundreds, if not thousands, of files that don't exist or are elsewhere. I've made a PR that addresses that issue.
Still not sure what's causing the freeze when opening the steam overlay, however.
It's quite possible that
4faefcffad1ca8a5f1e8fef157e2b542
is some piece of Japanese exclusive content that was stripped from the US client - it would take cross referencing the JP client to confirm this.It doesn't show up in the JP client, neither does it show up in any of the past clients (including the ancient SEA client nor the Taiwan client). Whatever that file is, SEGA has simply forgotten about it. I assume it might have been from the alpha, as you could find data relating to the game's beta and alpha despite being from 2010 ~ 2012.
As alluded to earlier, NGS and PSO2 as a whole attempt load many files that don't exist as we've seen through prehashed filename logging. And I should note MANY of these exist specifically within NGS as well. I've attached a file showing this though note that it was tested quite a while ago so I can't vouch for its validity in current patches, but it's unlikely it was changed much, if at all. megaice.txt
For example, each terrain section has 3 levels of detail that are in the files. As you would see in the file attached, the game tries to load detail level 4 for each of the 100 sectors (0 based). That's 100 read attempts on its own.
There's also the player outfits. A rarely used feature of outfits is that they can dynamically load material animations from another ICE file with the costume denoting characters swapped for 'bm'. In addition, the basewear outfits are known to trigger checks for potential alt versions in the case of certain outer wears.
There's also random one off files that haven't been added yet which get checked for.
Point is, there's a lot, lot of cases even in current where it happens and it's a pretty big deal in this case. Sega could fix this on their end potentially, but it seems more sensical to fix this and alleviate it for the myriad other games which might suffer from this issue.
Okay, looks like the fix isn't as simple as I thought it was gonna be. I do have an idea based on the Wine Wiki page.
The first option is too finicky: building in a coherent cache (we need to know when files change and make it stale) into Wine, way too much effort and prone to breaking (the wiki says we'd need kernel support for that).
The second one:
Using a case-insensitive FUSE filesystem for just the Wine folder
Is much more reasonable, since Wine already has (some) support for detecting ciopfs. So we could have Proton mount certain directories with ciopfs (perhaps only the steamapps dir is necessary), if configured to.
However there are some caveats:
But FUSE doesn't give you any advantages to just doing the mapping in Wine.
ciopfs isn't exactly suited for this use case:
To avoid any conflicts you should not manipulate the data directory directly, any change should be done over the mount point. All filenames in the data directory which aren’t all lower case are ignored.
Instead of ignoring, we would need to build an in memory mapping. Same issue as in Wine, so FUSE is actually completely unnecessary for this. So that brings us right back to building a coherent cache. Dang.
At this point, I don't know if this is an issue that could be fixed in Wine itself without some kind of refactor on how it handles these case-sense checks and balances entirely.
IMO, if this were a semi-perfect world, btrfs, ext4, f2fs, et al filesystems would all come with casefold support flags in newly created filesystems by default, and any (also fresh) Steam library folders should have that attribute checked (it's doable on the user side, and assuming the folder is clean, no reason why Steam couldn't do this automagically for the user). Hecc, they could even just add a "Windows compatibility" checkbox when creating new folders!
This might be controversial, and PSO2 (and I guess BeamNG) are the only real culprits guilty of this soft-dependency right now; but especially in the context of Windows-on-Linux gaming... I don't see the downside to making this optional feature a new default. If anything, the only problems I've ever had regarding gaming on Linux with Wine &/or Proton (especially in the case of modding games) were because of case-sensitivity and the snafus regarding that.
Then again, I just never saw the appeal or point of case-sensitivity in Linux filesystems anyways so I might just be the odd one out, this whole idea of mine is getting a touch political mayhaps... apologies, but those are just my thoughts.
Good news:
You can enable casefolding on any existing ext4 partition without wiping it as long as it's not encrypted (afaik) and your kernel supports ext4 casefolding:
Make sure your kernel supports ext4 casefolding:
$ cat /sys/fs/ext4/features/casefold
supported
Unmount the partition from it's current location:
sudo umount /mnt/Storage
Use tune2fs to enable casefolding allowance:
sudo tune2fs -O casefold /dev/sda1
mount the drive again:
sudo mount /dev/sda1 /mnt/Storage
Now, let's say you need to convert your existing steamlibrary so that you can install PSO2. First, move it to a backup:
mv SteamLibrary SteamLibrary1
Now you'll need to rebuild the basic folder structure and enable casefolding on that structure:
mkdir SteamLibrary
chattr +F SteamLibrary
mkdir steamapps
cd steamapps
mkdir -p {common,compatdata,downloading,shadercache,temp.workshop}
Now -- anything that is -moved/merged- into any of those folders will -not- have case folding, but anything -copied/downloaded- newly to them will, and the folders we created will retain casefolding.
SO let's get files moved over. First go to your backup SteamLibrary1/steamapps/ folder. You're going to select everything -except- the common folder and right click + CUT. Then go to your new SteamLibrary/steamapps/, rightclick + PASTE. Choose 'MERGE'. This will move everything over while retaining the +F flag on your new folders.
Last thing is the games themselves. Go to the backup SteamLibrary1/steamapps/common/, again -CUT- everything, and PASTE + MERGE everything into SteamLibrary/steamapps/common/
We do this separately so that nothing is tried to copy which wastes space.
Finally -- open steam -- now you can install PSO2 into /mnt/Storage/SteamLibrary and it will have case folding! No more stutter, and you didnt have to wipe your drive!
Last thing -- If you have PSO already installed here, you can copy it to another drive, delete the original, then copy it back, and it will then be copied with new casefolding attributes. You need to make sure to copy both the game files:
SteamLibrary/steamapps/common/PHANTASYSTARONLINE2_NA_STEAM/
And the appmanifest:
SteamLibrary/steamapps/appmanifest_1056640.acf
Oooo, didn't realize there's tune2fs
, I wonder if there's an equivalent for F2FS somewhere :thinking:
Recreating the folder structure using the find
command here also works wonder.
I took udf's code and improved the hack a little. Fundamentally the issue is that PSO2 is scanning the win32_na folder for the files it wants and then falling back to the standard (japanese) win32 folder for the content. If you have a handful of files this no problem, but if you're looking for over a few thousand files the manual search will just murder the performance.
Here's the patch: pso2_hack.txt
For TKG build you can add this as a user patch in the wine-tkg folder (rename extension to .mypatch), for other builds you do their specific thing. The improvement I made is that I can filter out a specific folder where the manual search isn't executed. Run the game with WINE_NO_OPEN_FILE_SEARCH="pso2_bin/data" %command% for best effect. I had some sounds missing and got a crash after hours of play, but nothing terrible. I suppose sound issues can be fixed by checking what files can't be loaded from the win32* folders and renaming them as they are searched for by the game.
Enjoy.
EDIT: sound issues were my mistake, the game is in dolby surround and I forgot to reroute the other channels to my front speakers in jack. So all sounds are there. And the game also works perfectly with just WINE_NO_OPEN_FILE_SEARCH="_na" which allows the system to fallback on the search system for the base folders (which it apparently never does because I haven't had a performance issue for over 7 hours).
Unfortunately, some of us don't make use of EXT4 (i personally use btrfs on my major Linux systems) so casefolding isn't an option. PSO2 remains inaccessible on those filesystems until mainstream releases of WINE are available with GoLd-ReAvEr/udf's patch, or some variant of such.
Unfortunately, some of us don't make use of EXT4 (i personally use btrfs on my major Linux systems) so casefolding isn't an option. PSO2 remains inaccessible on those filesystems until mainstream releases of WINE are available with GoLd-ReAvEr/udf's patch, or some variant of such.
Speaking purely anecdotally, ext4 performs a bit better than btrfs did on the same hard disk over a few other games, so there is merit to switching. But I also agree with the sentiment, considering how widespread the latter is becoming than the former, in recent years anyways.
For now (because I have my doubts such patches will make it into Wine in the short term), either make some space with a dedicated partition for it, or annoy kindly ask btrfs maintainers to consider support.
Replying to https://github.com/ValveSoftware/Proton/issues/4122#issuecomment-1172770360
btrfs has become the default filesystem in Fedora, and it's use of subvolumes is very useful for spreading space out over your entire disk without requiring multiple partitions.
the way I settled on getting this working was creating a new sparsefile, formatting it with EXT4 with the casefolding flag turned on, then setting it up with Steam as a separate library folder with attribute +F set. It's not a perfect solution and I can't verify that the performance will be perfect, but it's a good stopgap solution at least.
Weird, enabled case insensitivity on this game's folder on ZFS and still experience hiccups and freezes although it's better indeed. Was something else needed?
strace is also full of --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
, no idea if related...
@DistantThunder Wine currently doesn't check for ZFS when determining the case (in)sensitivity of a directory. ZFS also doesn't support the casefold flag (Wine is checking the same flag that lsattr shows for ext4 and f2fs).
So we need to find different way to detect a case insensitive ZFS dataset, and add the check to Wine. I had a look at the ZFS ioctl source code, and I didn't find anything that would help for this. The way zfs get casesensitivity <dataset>
works involves multiple ioctl calls that return a custom structure, and implementing that would add way too much overhead for it to be worth it.
I ran into the first real issue. The concert has a tendency to crash the game and show black on the TV. When this happens I'm also unable to participate in the urgent quest that comes after.
The concert has a tendency to crash the game and show black on the TV
To be clear, is this regarding the limited-time watch-together concert events that's currently going on, or is this with the usual random idol concerts? or does it happen with both?
I just heard it's with the PSO2 anime. This is with the TKG build, it's on right now, this exact moment.
I applied some foundation and at least the crashes are gone. Installing the media package that should not be named at least prevents the game from crashing it seems. I had sound initially but that was gone 5 minutes in or so.
About the slowdown on shows and videos, this is reproducible on Windows too, as long the game is on an exFAT partition. I contacted SEGA on 16/04/2021 about this. Soon after that they made the anti-cheat more restrict and I stopped playing, as it would always trigger when I went to character selection.
Yesterday, I tested the game again and I was able to play it (Xubuntu 22.04) using Proton Experimental.
The game and the compatdata are on NTFS. There is stuttering while the shader cache is being created and when loading certain elements. I can see the external HDD LED blinking nonstop when I'm around other players. If I set the graphics preset above 2, then it will fail to connect with a 630. After playing for some time, the experience is much better.
There's black screen on displays, but there are no FPS drops this time. I don't know when something should be playing at all.
Loading on PSO2 Classic is pretty fast, loading on NGS takes long, but not as long as on CBT (it took 12 minutes back then!). Teleporting from Retem to Central City seem to not work, the game freezes without disk activity and doesn't even send the 630, while the sound effect or the Ryuker Device and the music play. If I go back to Halphana Plains it works, but as long as I enter Central City, it freezes with similar symptoms.
I even tried running from Retem City to Central City, and as soon as I entered Central City, the game froze. I think that if I waited for like 1 hour the game would recover, as I experienced those large freezes when going to other places with players, but I don't want to really wait for 1 hour...
What I have to do is to go from Retem to Aelio (not Central City), log out, log in and then I can go to Central City.
I tried generating a log using PROTON_LOG=1
, nothing special. Just from running from Retem to Aelio, the log already had 5 GiB! When I finally reached Central City, the log had 14,9 GiB.
I think I'll have to get 4 TB of NVMe exclusively to do the same with PROTON_LOG=+all
.
I'm still early on NGS (just got on Retem), they really messed up with this Battle Points thing.
These are the mount settings I'm using for the NTFS partition on /etc/fstab
:
UUID=43754B5932F34301 /mnt/Externo ntfs-3g rw,uid=1000,gid=1000,umask=022,relatime,user_id=0,group_id=0,exec,allow_other,uhelper=udisks2,noauto,x-gvfs-show 0 0
It works as a normal removable HDD you would mount using GNOME Disks/Thunar. Obviously, my user's UID and GID are 1000.
Edit: I can confirm the show is working and the video is playing on Retem
@FurretUber Logs should be saved to a directory with compression, any compression. Text converts very well from gigabytes to a few megabytes.
I've got 322 hours in the game right now and I've seen nothing happening on this front. If there's no intention of doing any code fixes for the abhorrent search speeds or standardization fixes to deal with the case sensitivity of the standard ext* filesystems, could my patch be included in the proton release and the video issues (with the anime being played in game) be fixed so other people can also enjoy this game? I mean right now it's looking kinda silly.
I've got 322 hours in the game right now and I've seen nothing happening on this front. If there's no intention of doing any code fixes for the abhorrent search speeds or standardization fixes to deal with the case sensitivity of the standard ext* filesystems, could my patch be included in the proton release and the video issues (with the anime being played in game) be fixed so other people can also enjoy this game? I mean right now it's looking kinda silly.
You're free to test with other non-PSO2 games to make sure there's no regressions, since it is a fairly deep-rooted change you're suggesting. Otherwise, the length to which it'd take to test this change seems appropriate. Might have better luck submitting the patch to Wine upstream directly rather than a Proton-only patch.
Also, what video issues? I've not seen any problems on this front with the ~80hrs I've spent. Only crashing problems I've had all seems to be networking related. Unless I'm really missing something here...
The video issues relates to the special anime watch together event that was being held, those videos wasn't a local game asset as is the case with the concert events, instead they were being streamed remotely.
I have in-game video playback explicitly disabled so I don't really have anything to say in this regards. related comment
I don't think I've mentioned it on this issue thread but I've written a guide that summarizes all the stuff that's been discussed here in regards to casefolding
includes a step to make a loop device, which I've never tried personally, but should potentially work?
Replying to https://github.com/ValveSoftware/Proton/issues/4122#issuecomment-1203428561
or just add a SteamGameId check around the hack -- thats the easier solution here.
Something like this I think should work, @GoLD-ReaVeR?:
diff --git a/dlls/ntdll/unix/file.c b/dlls/ntdll/unix/file.c
index ad66954eb8b..70d73eb4557 100644
--- a/dlls/ntdll/unix/file.c
+++ b/dlls/ntdll/unix/file.c
@@ -3350,6 +3350,13 @@ static NTSTATUS lookup_unix_name( const WCHAR *name, int name_len, char **buffer
if (is_unix && (disposition == FILE_OPEN || disposition == FILE_OVERWRITE))
return STATUS_OBJECT_NAME_NOT_FOUND;
+ static char *skip_search = "_na";
+ const char *sgi = getenv("SteamGameId");
+ if ((!sgi) | (sgi && !strcmp(sgi, "1056640"))) {
+ if (strcasestr(unix_name, skip_search) && disposition == FILE_OPEN)
+ return STATUS_OBJECT_NAME_NOT_FOUND;
+ }
+
/* now do it component by component */
while (name_len)
a note on the code for those that may think this is incorrect:
!strcmp(sgi, "1056640")
strcmp returns 0 if the strings match and 1 if they don't match. We want them to match. 0 is determined as "false" while 1 is determined as "true" , and in boolean land you check if a boolean is false by prefacing it with !, hence:
!strcmp(sgi, "1056640")
You replaced the environment variable guard with a steam id. It would work but it would prevent users from changing the filter when necessary. I suppose it's fine for the ordinary plebeian users.
You replaced the environment variable guard with a steam id. It would work but it would prevent users from changing the filter when necessary. I suppose it's fine for the ordinary plebeian users.
how often do they need to change the filters is the question. does _na work universally or is it going to be problematic
The cleaner way to do this is use environment vars and set them using protonfixes.
The cleaner way to do this is use environment vars and set them using protonfixes.
I disagree. If only _na is ever used and it's only ever used for one game, specifically only usable within steam then there is no need for a protonfix. All that does is add a second requirement on top of the hack. otherwise I would have done it that way.
pretty sure it's best to think about PSO2 for now as it's currently the only case where this fix is needed the most, making the patch extensible for other games can come later no?
Someone mentioned BeamNG.drive also suffering from this. I'm pretty sure there are or there will be other games that could benefit from this.
Has anyone got the built-in controller on the Steam Deck to work with this game? Controller doesn't seem to get recognized. Also, if I use kbm, mouse tends to freak out.
Has anyone got the built-in controller on the Steam Deck to work with this game? Controller doesn't seem to get recognized. Also, if I use kbm, mouse tends to freak out.
Never had a problem on mine. With all default settings it should just work. Make sure you have the controller settings set for classic gamepad mode. I'll go through and double check all the settings on mine and report back.
EDIT: Verified the PSO2 install is stock on my Steam Deck. Default controller config is Gamepad, no custom proton version. Just Works(tm).
Seems the GE-Proton got broke after PSO2's September 14th maintenance. The launcher no longer is able to open when using any GE version, however the Proton Experimental/7.0 versions the launcher will open but the game still has the long loading problems the GE workaround fixed.
A few reports on ProtonDB have started appeared stating the same
I hereby state that the tkg build I'm using with my patch is still working for me.
@GloriousEggroll perhaps switching to WINE_NO_OPEN_FILE_SEARCH="pso2_bin/data" will help you. I switched to it a while ago to try to avoid some performance issues that were introduced during the wa(n)ker release and I never switched back. I tested _na just now and it won't start for me, so the problem seems to be there.
Sega's done it again. When previewing the player character in shops, scratch ticket or the look menu, the game locks up.
Occurs on both Valve and GE builds of Proton and will re-occur.
Log (Is this the right log?)
Update: I was worried it was my system and didn't want to do a drastic change but did it anyway. Switched to Pop-OS and it's now resolved on my side. I was on Kubuntu 22.04, basically stock. So things that I can think of, Mesa being out of date, AMDVLK being out of date... any ideas?
I'm not the only one I've seen with the issue, there was one other on a Discord that I frequent. I don't want to recommend distro-hopping or reinstalling OS as a solution so, I don't know. This is beyond my skill level.
can't repro (Proton Experimental; Arch Linux; linux-lqx 5.19.8-lqx1-1-lqx; Mesa 22.3.0-devel git-34a390569d) could it perhaps be graphics quality-related?
I didn't have this problem either. I'm running the game maxed and still with my own patch on a tkg build compiled a few days ago.
Just for the record, I'm running nvidia with DLSS enabled.
AMDVLK being out of date...
@jhc1993-846 so are you using AMDVLK as default? what happens if you switch to RADV?
@jhc1993-846 if you're providing new info, please make a new post so that people tracking the conversation get an update. If you edit your post it's unlikely for people to see it.
I suppose the last step of the journey is figuring out why Steam Overlay causes the game to freeze momentarily?
AFAICT this affects typing with the OSK on Steam Deck as well (which might be a huge bummer for some people as socializing is what the game is mostly about for them)
It seems with one of the last updates they did the game seems to get hung up loading into the actual game. Experimental, 7 and 8 all behave the same. It will get past the launcher, and to character selection, but trying to load into the game usually causes the loading screen to lock up.
Now i did find the game will eventually respond, but by then the game has been locked up for so long the servers will just kick you, and it'll come back. I then tried to login again...it immediately bypassed the loading screen (likely because most of it was already loaded); but i loaded into basically nothing as it slowly loads everything in one by one until it freezes again (this eventually has the same disconnect issue before it kicks you back out and repeats this on further attempts).
GE Proton 7 and 8 both seem to bypass this issue, but both still carry that Steam Overlay issue (so i just disable it). But in addition i found some audio seems to be missing (mostly anything with your own character voice attached to it, dodging, grunts, etc.. but anything that has its own separate sound like gunfire and footsteps seem to be fine).
Its been a minute since this one has been updated, but i thought i'd put this here.
the status of this game is still the same as June last year.
https://github.com/ValveSoftware/Proton/issues/4122#issuecomment-1165651616
Proton doesn't carry the patch for bypassing Wine's case-insensitivity traversal, only GE does. the other workaround that doesn't involve the patch involves creating an ext4/F2FS partition with casefolding (or configure existing partitions)
after that initial hurdle, game works flawlessly so long as it is installed on an SSD, HDD is not recommended (same as on Windows). Steam Overlay does momentarily freeze the game however.
EDIT: just realized that tomorrow marks the anniversary of this game properly at all on Linux
Ok i tried that method; Using non-GE; but the missing audio issue remains. Funny enough when adjusting the character volume slider i can hear it; but everywhere else (including the salon's preview voices section) those sounds are just gone.
For the steam overlay its a crapshoot if it works or causes some horrible error. Sometimes it works and just lags for a second like you said. Other times it gives me this error (and i also sometimes won't get past the SEGA logo on start)
Hello @Arucied, please copy the contents of Steam Runtime Diagnostics from Steam (Steam
-> Help
-> Steam Runtime Diagnostics
) and put it in a gist, then include a link to the gist in this issue report.
Here's the gist
As a small update. Wine/Wine-staging just updated for me; and it seems to have resolved that audio issue. I still need to keep the seam overlay off, as when its on there's a 50/50 shot the game won't launch; and the other where it'll give the error i linked previously.
The latest version of GE-Proton8-10 seems to break the workaround for long loading times and now has severe performance hiccups. GE-Proton8-9 and below work properly still. Edit to add context that this was on Steam Deck.
Compatibility Report
System Information
I confirm:
Symptoms
The game would not launch when you click the Start Game button in the game's own launcher. pso2.exe runs in the background as a zombie process.
Reproduction
Log
Google Drive link to steam-1056640.log, It kinda blew up to 53 MB in just a few seconds when opening up the game.