fuzziqersoftware / newserv

Phantasy Star Online game server, proxy, and reverse-engineering tools
MIT License
167 stars 37 forks source link

[PSOBB] EnableEpisode3SendFunctionCall appears to be applying to Blue Burst clients #297

Closed ShiftaDeband closed 10 months ago

ShiftaDeband commented 10 months ago

Good evening,

I typically leave EnableEpisode3SendFunctionCall on with a welcome message, wait a moment, then press A - this typically allows for the EnableEpisode3SendFunctionCall to work without crashing the Episode III USA client. (I assume this is a side effect of the patch being applied/loaded, which I can understand why it's disabled by default.)

Anyway, the issue I'm seeing right now is two-fold. With the EnableEpisode3SendFunctionCall enabled, and since a recent commit that I need to track down, it appears that this quest/file is being loaded onto PSOBB clients, which causes them to crash during the 'Go to Lobby' and 'Disconnect' screen.

I believe this is related to this command, because without this, I can get to the above screen. I also believe I recall seeing m999999p_e.bin being sent for Episode III clients in the past to allow for patches, although I'm not sure.

For what it's worth, I'm using a modified, allegedly clean but decompressed and with Game Guard removed 1.24.3 client executable, but also tried building my own with CreateTethExe and the same issue occurs.

Here's a log of the issue:

I 21289 2023-12-27 21:31:47 - [Server] Client connected: C-5 on fd 38 via 14 (T-12004-BB_V4-bb-data1-login_server)
I 21289 2023-12-27 21:31:47 - [Commands] Sending to C-5 (version=BB command=0003 flag=00000000)
0000 | 7C 01 03 00 00 00 00 00 50 68 61 6E 74 61 73 79 | |       Phantasy
0010 | 20 53 74 61 72 20 4F 6E 6C 69 6E 65 20 42 6C 75 |  Star Online Blu
0020 | 65 20 42 75 72 73 74 20 47 61 6D 65 20 53 65 72 | e Burst Game Ser
0030 | 76 65 72 2E 20 43 6F 70 79 72 69 67 68 74 20 31 | ver. Copyright 1
0040 | 39 39 39 2D 32 30 30 34 20 53 4F 4E 49 43 54 45 | 999-2004 SONICTE
0050 | 41 4D 2E 00 00 00 00 00 00 00 00 00 00 00 00 00 | AM.             
0060 | 00 00 00 00 00 00 00 00 A4 64 DE 23 99 47 29 88 |          d # G) 
0070 | 3F 02 9A 18 D6 CA 09 2C E4 A8 04 4C 5D B4 F1 70 | ?      ,   L]  p
0080 | A9 8D 29 0F CE FF 5E AE 93 92 94 65 EE 13 20 9A |   )   ^    e    
0090 | 08 93 5D B7 E8 3E 1B 28 50 D5 75 79 79 B7 E0 6F |   ]  > (P uyy  o
00A0 | 27 61 44 A5 80 D1 F5 EC DE 2D 0A 36 E1 19 3B 20 | 'aD      - 6  ; 
00B0 | 5A AF BB 1B 6F 33 93 B2 E1 7C C4 AD 76 FE 42 CB | Z   o3   |  v B 
00C0 | 4A B8 F5 F7 06 8D C1 CE 54 68 69 73 20 73 65 72 | J       This ser
00D0 | 76 65 72 20 69 73 20 69 6E 20 6E 6F 20 77 61 79 | ver is in no way
00E0 | 20 61 66 66 69 6C 69 61 74 65 64 2C 20 73 70 6F |  affiliated, spo
00F0 | 6E 73 6F 72 65 64 2C 20 6F 72 20 73 75 70 70 6F | nsored, or suppo
0100 | 72 74 65 64 20 62 79 20 53 45 47 41 20 45 6E 74 | rted by SEGA Ent
0110 | 65 72 70 72 69 73 65 73 20 6F 72 20 53 4F 4E 49 | erprises or SONI
0120 | 43 54 45 41 4D 2E 20 54 68 65 20 70 72 65 63 65 | CTEAM. The prece
0130 | 64 69 6E 67 20 6D 65 73 73 61 67 65 20 65 78 69 | ding message exi
0140 | 73 74 73 20 6F 6E 6C 79 20 74 6F 20 72 65 6D 61 | sts only to rema
0150 | 69 6E 20 63 6F 6D 70 61 74 69 62 6C 65 20 77 69 | in compatible wi
0160 | 74 68 20 70 72 6F 67 72 61 6D 73 20 74 68 61 74 | th programs that
0170 | 20 65 78 70 65 63 74 20 69 74 2E 00             |  expect it.     
I 21289 2023-12-27 21:31:47 - [Commands] Received from C-5 (version=BB command=0093 flag=00000000)
0000 | B4 00 93 00 00 00 00 00 00 00 01 00 E5 71 D0 2F |              q /
0010 | 52 00 00 00 01 00 04 00 00 00 00 00 74 65 73 74 | R           test
0020 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0030 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0040 | 00 00 00 00 00 00 00 00 00 00 00 00 74 65 73 74 |             test
0050 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0060 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0070 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0080 | 00 00 00 00 01 01 8D 7A 9C D7 41 BB 32 AC 99 83 |        z  A 2   
0090 | 00 00 00 00 FF FF FF FF FF FF FF FF FF FF FF FF |                 
00A0 | FF FF FF FF FF FF FF FF FF FF FF FF 00 00 FF FF |                 
00B0 | 80 FF FF FF                                     |                 
I 21289 2023-12-27 21:31:47 - [Commands] Sending to C-5 (version=BB command=00E6 flag=00000000)
0000 | 44 00 E6 00 00 00 00 00 00 00 00 00 00 00 01 00 | D               
0010 | E5 71 D0 2F 00 00 00 00 32 AC 99 83 00 00 00 00 |  q /    2       
0020 | FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF |                 
0030 | FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF |                 
0040 | 01 01 00 00                                     |                 
I 21289 2023-12-27 21:31:47 - [Commands] Sending to C-5 (version=BB command=00A6 flag=00000000)
0000 | 40 00 A6 00 00 00 00 00 50 53 4F 2F 42 75 66 66 | @       PSO/Buff
0010 | 65 72 4F 76 65 72 66 6C 6F 77 00 00 00 00 00 00 | erOverflow      
0020 | 00 00 00 00 00 00 00 00 00 00 03 00 6D 39 39 39 |             m999
0030 | 39 39 39 70 5F 65 2E 62 69 6E 00 00 18 00 00 00 | 999p_e.bin      
I 21289 2023-12-27 21:31:47 - [Commands] Sending to C-5 (version=BB command=00A7 flag=00000000)
0000 | 1C 04 A7 00 00 00 00 00 6D 39 39 39 39 39 39 70 |         m999999p
0010 | 5F 65 2E 62 69 6E 00 00 48 00 00 08 AA AA AA AA | _e.bin  H       
0020 | 39 6C FF 50 3C E0 81 5F 60 E7 F4 40 38 00 00 00 | 9l P<  _`  @8   
0030 | 90 07 00 00 90 07 00 04 3C C0 00 1E 60 C6 09 60 |         <   `  `
0040 | 90 C7 00 08 90 E5 00 04 3C C0 81 5F 60 C6 F3 C0 |         <  _`   
0050 | 90 C5 00 08 90 06 00 00 48 00 00 24 00 00 00 00 |         H  $    
0060 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0070 | 81 5F F3 E8 80 59 2A C4 00 00 01 60 48 00 01 11 |  _   Y*    `H   
0080 | 7C 08 02 A6 94 21 FF C0 90 01 00 44 90 61 00 14 | |    !     D a  
0090 | 90 81 00 18 88 A4 00 01 54 A5 82 1E 64 A5 B3 00 |         T   d   
00A0 | 60 A5 0C 00 90 A1 00 08 38 C0 00 04 7C A4 34 2C | `       8   | 4,
00B0 | 90 A1 00 0C 28 05 00 00 41 82 00 7C 38 C4 00 10 |     (   A  |8   
00C0 | 7C E6 2A 14 38 E7 FF E0 81 07 00 04 28 08 00 00 | | * 8       (   
00D0 | 41 82 00 34 7D 09 03 A6 81 07 00 00 7D 08 32 14 | A  4}       } 2 
00E0 | 39 08 FF FE 7C CA 33 78 A5 28 00 02 55 29 10 3A | 9   | 3x (  U) :
00F0 | 7D 4A 4A 14 81 2A 00 00 7D 29 32 14 91 2A 00 00 | }JJ  *  })2  *  
0100 | 42 00 FF E8 3C 00 80 00 60 00 C3 24 7C C3 33 78 | B   <   `  $| 3x
0110 | 7C A4 2B 78 7C 09 03 A6 4E 80 04 21 81 07 00 10 | | +x|   N  !    
0120 | 7D 08 30 2E 7D 09 03 A6 4E 80 04 21 39 00 00 0C | } 0.}   N  !9   
0130 | 7C 61 45 2C 80 81 00 18 38 A0 00 08 7C 64 2C 2C | |aE,    8   |d,,
0140 | 38 A0 00 0C 7C 84 2C 2C 3C 00 80 10 60 00 F8 34 | 8   | ,,<   `  4
0150 | 7C 09 03 A6 4E 80 04 21 39 00 00 10 7C 61 45 2C | |   N  !9   |aE,
0160 | 80 61 00 14 80 83 00 18 80 84 00 28 7C 89 03 A6 |  a         (|   
0170 | 38 81 00 08 38 A0 00 0C 4E 80 04 21 80 01 00 44 | 8   8   N  !   D
0180 | 38 21 00 40 7C 08 03 A6 4E 80 00 20 7D 28 02 A6 | 8! @|   N   }(  
0190 | 48 00 00 05 7D 48 02 A6 39 4A FF F8 3D 80 80 00 | H   }H  9J  =   
01A0 | 61 8C BD 80 7C E9 50 50 54 E7 F0 BE 7C E9 03 A6 | a   | PPT   |   
01B0 | 39 0C FF FC 39 29 FF FC 84 09 00 04 94 08 00 04 | 9   9)          
01C0 | 42 00 FF F8 3D 20 80 00 61 29 C3 24 7D 29 03 A6 | B   =   a) $})  
01D0 | 7D 83 63 78 54 E4 10 3A 4E 80 04 21 3C A0 80 44 | } cxT  :N  !<  D
01E0 | 60 A5 F6 84 38 00 00 B2 90 05 00 00 91 85 00 0C | `   8           
01F0 | 48 00 00 11 3C 80 5F 5C 60 84 A2 97 90 81 00 14 | H   < _\`       
0200 | 3C 60 80 0F 60 63 33 38 7C 88 02 A6 80 04 00 00 | <`  `c38|       
0210 | 90 03 00 00 80 04 00 04 90 03 00 04 80 04 00 08 |                 
0220 | 90 03 00 08 38 80 00 20 7D 29 03 A6 4E 80 04 21 |     8   })  N  !
0230 | 48 00 00 11 3C 80 5F 5C 60 84 A2 97 90 81 00 60 | H   < _\`      `
0240 | 3C 60 80 0F 60 63 36 44 7C 88 02 A6 80 04 00 00 | <`  `c6D|       
0250 | 90 03 00 00 80 04 00 04 90 03 00 04 80 04 00 08 |                 
0260 | 90 03 00 08 38 80 00 20 7D 29 03 A6 4E 80 04 21 |     8   })  N  !
0270 | 3C 60 80 10 60 63 FD 8A 38 80 00 64 B0 83 00 00 | <`  `c  8  d    
0280 | 54 63 00 36 38 80 00 20 7D 29 03 A6 4E 80 04 21 | Tc 68   })  N  !
0290 | 38 60 00 00 7D 68 03 A6 4E 80 00 20 00 00 00 00 | 8`  }h  N       
02A0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
02B0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
02C0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
02D0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
02E0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
02F0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0300 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0310 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0320 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0330 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0340 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0350 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0360 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0370 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0380 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0390 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
03A0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
03B0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
03C0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
03D0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
03E0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
03F0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0400 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |                 
0410 | 00 00 00 00 00 00 00 00 84 02 00 00             |                 
I 21289 2023-12-27 21:31:47 - [Commands] Sending to C-5 (version=BB command=0007 flag=00000002)
0000 | 8C 00 07 00 02 00 00 00 11 00 00 11 FF FF FF FF |                 
0010 | 04 00 30 00 31 00 3A 00 55 00 53 00 2F 00 50 00 |   0 1 : U S / P 
0020 | 6F 00 6C 00 6C 00 75 00 78 00 00 00 00 00 00 00 | o l l u x       
0030 | 00 00 00 00 11 00 00 11 11 22 22 11 04 00 47 00 |          ""   G 
0040 | 6F 00 20 00 74 00 6F 00 20 00 6C 00 6F 00 62 00 | o   t o   l o b 
0050 | 62 00 79 00 00 00 00 00 00 00 00 00 00 00 00 00 | b y             
0060 | 11 00 00 11 11 88 88 11 04 00 44 00 69 00 73 00 |           D i s 
0070 | 63 00 6F 00 6E 00 6E 00 65 00 63 00 74 00 00 00 | c o n n e c t   
0080 | 00 00 00 00 00 00 00 00 00 00 00 00             |                 
I 21289 2023-12-27 21:31:47 - [Commands] Received from C-5 (version=BB command=00A6 flag=00000000)
0000 | 18 00 A6 00 00 00 00 00 6D 39 39 39 39 39 39 70 |         m999999p
0010 | 5F 65 2E 62 69 6E 00 00                         | _e.bin          
W 21289 2023-12-27 21:31:47 - [C-5] Unknown command: size=0010 command=00A6 flag=00000000
W 21289 2023-12-27 21:31:47 - [Server] Error processing client command: unimplemented command
I 21289 2023-12-27 21:31:47 - [Server] Client disconnected: C-5 on fd 38
I 21289 2023-12-27 21:31:47 - [C-5] Deleted
I 21289 2023-12-27 21:31:54 - Cleared 1 expired system file(s)
I 21289 2023-12-27 21:31:54 - Cleared 1 expired character file(s)
I 21289 2023-12-27 21:31:54 - Cleared 1 expired Guild Card file(s)
fuzziqersoftware commented 10 months ago

This is caused by the client flags being broken somehow. The client is sending the config data 32 AC 99 83 00 00 00 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 00 00 FF FF 80 FF FF FF, which is wrong - it should be mostly 00, not mostly FF. This has the effect of enabling all the client flags, which causes this kind of incorrect behavior. Can you set HideDownloadCommands to false in config.json and capture a full session log, starting with the patch server and ending with the crash?

ShiftaDeband commented 10 months ago

You bet. Here you go - let me know if I didn't do something correctly here.

BB-Ep3FuctionLog.txt

Thank you!

fuzziqersoftware commented 10 months ago

The client is doing something pretty strange here. Can you also upload your psobb.exe? I'd like to take a look at some of the command handlers.

ShiftaDeband commented 10 months ago

You bet. Here's that 1.24.3 executable I mentioned earlier. Please note that this does look for _e files as opposed to some other clients that replace the base/Japanese files.

For what it's worth, I'm still seeing the same results even with the most recent commit on a 'CreateTethEXE'.

I'll try to see where this was broken - I have used this client in the past without issue until last night/today.

Edit: I created a branch from commit 522c184/'use a cleaner method to deal with duplicate 97 commands' and the client works here. Going down the list since then.

ShiftaDeband commented 10 months ago

It appears to be related to the changes here: https://github.com/fuzziqersoftware/newserv/commit/84ed80365cd290320dbbf05dbeb90935dfc397e1

Using any of the commits immediately before this works. When building and running this one causes it to try to send the quest again. Here's the log with commit 84ed803: BB-NotWorking-84ed8036.txt

And the version immediately before this, 87440437: BB-Working-87440437.txt

EDIT: Just to avoid posting another comment, in the revision that things are working, I'm seeing that my mag is being referred to as an Ushasu. I was wondering if this was related, so I deleted the player and related character data from the server, restarted using the latest commit, created a new character, and I still get disconnected/sent the quest. Regardless, I'll be happy to dig into this more.

pso1703745093

EDIT2: This just looks like a desync between the server items_v4.json and my server/client items file.

fuzziqersoftware commented 10 months ago

This bug led to the discovery of a behavior I didn't previously know about: if a certain field in the login response command (E6) is zero, the client scrambles the client config. (Previously, we thought this field was team_id, but it apparently is not.) d478e9b0 implements the relevant descramble logic - please try it again with that commit.

nolrinale commented 10 months ago

I tested with the newest commit on the Tethealla client with ENG mode enabled, and on first glance I see my inventory items with their correct names and mags, i also loaded a quest and verified again and saw no issues, is there anything more specific I should look for?

ShiftaDeband commented 10 months ago

Thank you so much for all you do. The latest commit fixed my issue, I'll close this.

As for the item issue - I can investigate that more. Everything seems to line up for the most part, but I'm using outdated English (1.24.3) files, which is probably causing my other issue. For what it's worth, everything lines up with the $i command, it's just the labels themselves that aren't matching. For instance, generating a Mag creates a Ushasu, but displays as a Mag in-game.

Other examples: Team Points 10000 creates a Platinum Pen which can be exchanged for 10,000 points, which I assume was the English version name before it was likely changed at some point, and in Ephinea's example, likely a better translation from the 1.25.x game. Disks past disk 7 (which is just DISK Vol.7 in my unitxt_e) don't exist with names, and trying to play them causes silence. I never tried placing the corresponding OGG file, though, so it may still "work", but the item name will be blank.

I need to get this 1.24.3 client uploaded at some point to archive.org. The English final patch is probably still out there somewhere, but I've never had good confirmation that the alleged 1.24.3 version I've had is complete and matches 100%.

Last but not least, I have some of the PSOBB CN client files, but I am assuming that most quests, including Government, Battle, Challenge, and others are lost to time. Someone went through the effort to translate them, but they certainly are not the files from the server. And even then, the project I've seen for this just replaces the base Japanese files (ones without _e, for example) with the translated ones. (I believe the CN client used _c for these?)

More on all that later. I'm hoping someone out there has the Simplified Chinese quests and final patch files somewhere, but I'm not going to hold my breath.

Anyway, after going mad off topic -- thank you.

nolrinale commented 10 months ago

@ShiftaDeband Well regarding your last comment, the client version expects a specific unitxt for that version in the case of the final US BB 1.24.3 and the final JP BB 1.25.13 you need to provide the proper unitxt of each version as the indexes of the items vary greatly thats why you see different items and scrambled names if you attempt to load the 1.25.13 unitxt into the 1.24 client and vice versa.

It doesn't really matter if it's a unitxt_j or unitxt_e but the indexes of that file inside should be the ones the client version is expecting to find for correct operation.

I recommend to enable the ENG switch in the Tethealla client which will turn the client into pretty much the US BB client with the very latest updates from JP BB you can find all the notes and required files in the notes/psobb folder.

The hidden switch should also enable the chinese support but this version I believe was a little bit different than USBB/JPBB .... but if you enable it the client should still expect the usual files to end with _cs for correct operation in this regard.

One little trick that can be used in case the files are not found is to enable CN support in the hidden lang switch and copy the unitxt files from the base japanese version (the most updated one) and rename them to unitxt_cs for example, then the game should load just fine but all the items will be in japanese but ready to be localized into chinese.

I think the texture files such as the localized PSOBB chinese logo in the title screen, the headers in the in-game menus, or the textures of the F12 menu should all be inside the data.gsl already so you dont need to source the game should just load them as usual as soon you start the game with the CN switch enabled. But if they are not they should still be present in the chinese client with the _cs prefix added at the end usually these texture files are compatible between versions.

At the bare minimum you require these files to exist in the client folder (not counting the textures ones)

Hopefully this mess of an explanation helps you solve your doubts about the PSOBB client localization shenanigans.

ShiftaDeband commented 10 months ago

@nolrinale

This is probably the best and easiest to understand write up I’ve ever read.

Thank you, this is unbelievably helpful. Thank you so much for your assistance with all this!