[PSOBB] Mag Label/Model Issues

[ShiftaDeband]

I have a strange issue occurring, and I'm not sure where it starts, whether that's on my end or something in the project. I want to say client, but I swear this was working before. (I just booted an older instance that was on commit f5bfd4a but it does the same thing there.)

This has to do with English files, unitxt_e, etc.

I'm seeing Mags with wrong labels, but correct mag models on files from an alleged clean 1.24.3 version, and then wrong mag models and correct labels on the English files in from the Notes folder. This only appears to affect Mags, at least with some limited testing with the $i command.

When using old 1.24.3 files, it's listed as an Ushasu but visually is a Mag:

When using the English files in the repository in notes/usbb, its listed as a Mag but visually is a Vritra:

Using $i to generate a Mag with 5/0/0/0 reproduces the behavior for each version as above. Trying to generate any other mags will also be incorrect with similar results as above.

For instance, using the 1.24.3 files:

• Soniti: Listed as ROBOCHAO, visually shows as Soniti
• Kalki: Listed as Ashvinau, visually shows as Kalki

Terminal output shows that the server has the correct names/IDs for all of these.

[PlayerInventory]    6: [+00000000] 02216900 F4011027 00000000 (00010006) 00000000 (Soniti LV105 5/100/0/0 0% 0IQ (red))
[PlayerInventory]    7: [+00000008] 020D1300 DC050000 2C016400 (00010007) 00000000 (Kalki LV19 15/0/3/1 0% 0IQ (red))

--

Background: I noticed this after a recent build when trying to resolve the issue with the client bug a few days ago. I thought that it was probably something that just got a little messed up in my patch-bb folder over time, so I completely re-did my patch-bb folder and made sure all files between the client and the server were synced, making sure files were not altered otherwise. After doing this, the issue persisted with the Mag visually being correct but listed as an Ushasu.

I then tried to use a different executable after guidance from @nolrinale (thank you again, by the way) and wanted to see if it was an issue with the client. When launching with this version, the issue persisted with the Mag visually being correct but listed as an Ushasu.

I next tried the files in the notes/usbb folder, simply replacing all other files in the patch-bb/data folder, and then got the second screenshot with the text showing 'Mag' but visually rendering as a Vritra. I checked between clients, and this behavior was the same between both the modified 1.24.3 client and the Tethealla executable.

--

I'm not sure what's going on. My understanding was that bb-data relied on the appropriate files being present in the patch-bb folder, which if not present falls back to data.gsl, which then falls back to the defaults in system. The output indicates that it is using the files from the patch-bb folder for all but PlyLevelTbl.prs, which seems correct. But, then again - the files the server is loading don't have anything to do with localization, do they?

I did just now try to remove everything in the patch-bb folder to force defaults from the system folder, then logged in, but I still have the same issues as above. The output indicates that it did use defaults.

My next steps were to look inside data.gsl, possibly repack it if needed, and see if that resolves the issue, but as far as I understand it, this shouldn't make a difference as the files are present in the /patch-bb/data folder.

After that, and if that didn't resolve the issue, I was planning on hopping back to different commits to see approximately when this started to happen. As far as I can remember, this wasn't a problem earlier this year, so something may have changed with item handling, but my memory here could be faulty. I'll probably try to commit hop later today to see if I can find anything.

Happy to provide any files, logs, etc - I'm just trying to find a better place to start. Thank you in advance for any guidance.

[nolrinale]

First lets equalize our clients, I've prepared a cleaned version of the tethealla client which i have a copy on my server you can download it from here https://kiyomizu.moe/psobb/patch/, then create with the CreateTeth tool the exe for your server with the ENG switch enabled.

Then add the usual _e files to make the client work the including unitxt_e in the notes/psobb/usbb-resources folder (not the one that says SEGABACKUP)

Let's start the local server and create a new character, you should start with a normal mag with the usual 5/0/0/0 stats

Try to manually feed the mag until lv 50 and make it evolve it's gonna take a while but I can confirm from my production server my mag evolved correctly as a hunter from a Mag to a Varuna, then to a Marutah it is currently at lv 47, just finished leveling it up to lv 50 and it became an Ila as intended.

[Matt-Swift]

Entirely client based, the issue is that your unitxt does not match with the ItemPMT file you are using - the default one with newserv has all the Episode 4 content in it. You would need to somehow track down an ItemPMT file without those items, or modify it yourself.

@nolrinale ShiftaDeband wants to use the US client, so this is not particularly helpful.

[nolrinale]

The Tethealla client (JPBB) with the ENG switch enabled is essentially the same as the USBB client just on a newer version (1.25.13)

I try to provide support the best way I can

[fuzziqersoftware]

To be clear - it's just the item names that are misbehaving, correct? Do the mags' stats/models/behaviors all appear correct in-game?

If it's just the names, then the problem is likely the client's unitxt_e.prs not matching the server's ItemPMT.prs. To generate an item's name, the client looks up the item's definition in ItemPMT, then takes the item ID from there and looks up the string with that number from the second collection in the unitxt file. Here is my decoded unitxt file; you can check if yours matches by running ./newserv decode-unicode-text-set unitxt_e.prs, and looking at the second list in the resulting JSON file (it should start with "Monomate"). I'm guessing your file will have the strings in a different order, will be missing some strings, or will have some extra strings.

Let me know if your unitxt matches; if it doesn't, I'll change the server code to use a unitxt file instead of the existing names-v4.json file, so you'll be able to properly use localized item names on the server side.

[ShiftaDeband]

First off, thank you all. Apologies for the delay, been a little under the weather but very much appreciate the thoughts here.

@nolrinale - Wow, you really went above and beyond what I was doing simply using the $i command to test this. I'm going to give that a shot, and I assume your files would fix this issue. Thank you for providing those.

I really do want to try to get this client to work as it was, but I'm beginning to see how difficult this idea is especially in the event someone was trying to support v4 across the three different localized regions at some point. The differences in the unitxt files has the potential to be a huge pain. At least in your case, you do have something that aligns correctly, which is super helpful.

--

@Matt-Swift - Thank you for the reminder about ItemPMT. At the very least, in the one instance, it looks like the items themselves are in sync, just the mags in the unitxt_e are out of order (see json file below).

Random thought, I had thought that Sonic Team was pretty picky about adding things to unitxt in any language. Meaning, they'd try to add items to either blank areas or near the bottom of the file. I don't think they were thinking about backwards compatibility, but more to avoid having to deal with issues like this, even between regions. Think having to adjust ItemPMT and unitxt, and that doesn't even account for different regions having to do the same.

It seems that assumption was incorrect. I mean, even all Blue Burst clients have legacy Dreamcast, GameCube, Xbox, etc, language in there that they just never seemed to have bothered with. Just interesting to see that as time goes on. (I say this knowing very well I have similar issues/technical debt in my own code, so I shouldn't be speaking on this. Anyway...)

--

@fuzziqersoftware - It makes sense what you're saying. I think there's a huge amount of complexity here and I understand that this isn't an easy answer.

To start off, I used newserv to parse my unitxt_e.prs file, check the diff here. I see the issue right off the bat at line 1077. For what it's worth, it wouldn't be too bad to edit the unitxt_e in my case and add blank spots for those, which should bring all rows below that up to alignment. (other sequences of \u characters should be untranslated Japanese letters, not sure if that's a limitation of my setup.)

Another program I've used (unitxt) will render the other languages. Here's my unitxt_e.json and a unitxt_cs.json, and here are those corresponding unitxt_e.prs and unitxt_cs.prs files (and unitxt_shop_cs.json and unitxt_shop_cs.prs for good measure and because I'm sure those could come in handy some day, I'm just not sure what patch these are from - hopefully the latest before it went offline.).

With that in mind, then, it makes sense that your items list would be the 'master' list, and any other translations/unitxt files need to match that pattern. This just gets more complex with trying to support multiple languages, but I think that's just how that'll need to be.

I assume you can transform the JSON back into a PRS using encode-unicode-text-set?

[nolrinale]

Im very sorry didnt really understand you wanted to specifically make ver 1.24 from USBB then latest Tethealla wont be of use for you.

If 1.24 is the goal for this issue then the unitxt and unitxt_shop will have to match what the client is expecting in each of their lines, at the end of the day is a mega array and each element will have to be aligned to a mapping we cannot really see... but the server tool to build and rebuild the unitxt will be super handy to achieve this, still is gonna be a big artisan effort.

I would take the latest unitxt specifically from the USBB client and look inside if it includes all the episode IV items including the ones from the later update after MA4 days such as Zero Divide or From the Depths, Union Guard or the TypeM weapon series means they preloaded all the required items in parity with 1.25.13 and can be a good starting point to begin debugging everything else.

I also went to check the JPBB archives and confirmed the MA4 event update began with version 1.24.0 (https://web.archive.org/web/20060725153424/http://psobb.jp/news/wis/?mode=view&id=27) so I think SOA did received this same client revision for USBB at the time too but decided to just dont release the MA4 quests and also decided to dont enable Coren either, I checked your USBB exe and can confirm the Coren functions were in there too.

However version 1.24.12 (on JPBB) is the one that had the last item drop tables changed but I think these were all done server side as this version is the one that released Black Paper 2 you can check the news update here https://web.archive.org/web/20070304163618/http://psobb.jp/news/wis/?mode=view&id=116

So I think at least functionality wise the two clients are supporting all the game systems already, is just a matter of aligning the unitxt's strings

[ShiftaDeband]

Perfect.

Here's my plan: I'll work to get the unitxt_e that I have to match for my purposes - there's already a more updated unitxt_e in this repo that will work. I'll use your _e files for consistency and for bug checking when testing functionality between clients. (For instance, if I think there's a bug, I'll re-test whatever it may be with the client you've provided.)

For now, this can be closed. I just want to say thank you for everyone's help here. You all rock and are incredible people. Have a great 2024 and happy new year!