How to go about testing/self hosting?

[crispbitload]

Hello! I was wondering if its possible to test out the private server software, more specifically on the vita version of the game. I saw that there is already compiled artifacts available, and also wanted to know how much progress has been made so far in the private server software.

Also, do you by any chance have a backup of the vita 6.30 update? I have only been able to find up to 5.41 archived online, and the 6.31 update prevents you from finding a server (Text box pops up about the end of service).

[AntonnMal]

Hello! Sorry for the late reply, I was busy. Of course it's possible to test the server. To start it:

• Get the master server and the ship server (either from compiling or from actions)
• Get the data directory from actions or compile yourself: cargo run --bin=data_compiler data data_dir
• Copy the config files for master server and the ship server from data directory to the executable location
• Start the master server first
• Edit the ship server cfg to point to the master server (don't use localhost if you are planning to connect with the vita)
• Start the ship server

Now, about the vita version: PSN still has 6.30, NoPayStation scripts (more specifically pyNPU.py) will help you. You'll also need the RePatch plugin. To prepare the vita version:

• Install FAGDec on your vita
• Backup the 6.31 eboot.bin from the patch folder (!)
• Temporarily replace the 6.31 eboot.bin in the patch folder with the 6.30 one
• Using FAGDec decrypt it to an ELF
• Using any hex editor replace the RSA key in the ELF file
• Using vita-elf-inject replace the bin file with the new one
• Restore the 6.31 eboot.bin
• Place the injected eboot.bin in rePatch/PCSG00141/ folder (you might also need to copy the 6.30 050 and 051 cpks, but I'm not sure)

If you don't have access to a self hosted DNS server (i.e. pihole) you'll also need to replace the IP addresses

• Extract the sy_script.fpk file
• In the compiled lua file 3B9C16C5.lua replace the IP addresses (e.g. gs001.pso2gs.net) with your desired IP address (strings are prepended with the 32 bit length (string len + null terminator) value, so don`t forget to change it)
• Pack the sy_script.fpk file back and put in the rePatch/PCSG00141/data/vita folder

About the servers progress: it's not a whole lot. Basically only lobby works semi-normally, everything else is VERY unfinished. I have basic damage calculation working, but it's not upstream, I'm wating to update the packet library, then I'll push the server changes.

[crispbitload]

Thanks for the reply and no worries!

I really appreciate the helpful instructions, and didn't know about the python script to get the updates on vita! I'll try this later on today and update you on it, I think it's really awesome your making a project like this!

[crispbitload]

Update: So I have been having some trouble with the 6.30 update files. Using pyNPS, it does find the update files for PSO2, however its the 5.42 version (A0542).

I did however find the 6.30 version via playstation's update servers, but after unpacking using pkg2zip and putting the eboot.bin in the update folder, FAGDec does not recognize the file (May be still encrypted, cant find a RIF to decrypt it via pkg2zip)

I'm stuck at this point (And am sorry for my inexperience), is there a certain way to configure pyNPS, or would it be possible to provide the pre-patched eboot? Thanks in advance.

[AntonnMal]

Vita OS is very picky about the files in the patch folder. Make sure that the patch folder doesn't contain any extra files and folders and the game patch folder itself doesn't anything extra also. If this fails you can try to copy the whole 6.30 patch folder and rebuilding the livearea cache.

Also, when copying the files to the patch folder, they should be encryped.

Running the script with python pyNPU.py -al -t PCSG00141 gives the A0630 version download link. I don't feel comfortable sharing binaries.

And am sorry for my inexperience

Don't stress about it, we've all been there

[crispbitload]

Great news! So I ended up going the nuclear route and wiping the PCSG00141 patch folder and replacing it with the 6.30 folder, which finally let me decrypt the eboot.bin!

Now my next question is about the RSA key. Where would I find the key to replace? Also, what hex values would the key be located at? Thank you again for your help (and patience with a noob like me lol).

[AntonnMal]

Finding the RSA key is quite easy: inside the new eboot.bin.elf search for the header 0x06, 0x02, 0x00, 0x00, 0x00, 0xA4, 0x00, 0x00, 0x52, 0x53, 0x41, 0x31. The key is 148 bytes in total (with header).

Now, running the server should create a keypair.pem, but it's in another format, so we'll need to convert it. The easiest way is to use openssl: openssl rsa -in keypair.pem -outform MS\ PUBLICKEYBLOB -pubout -out publickey.blob. After running this command you should get a publickey.blob file. Just open the file in a hex editor and use it to replace the RSA key in the eboot.

[crispbitload]

So I have some bad news... After applying the HEX edit to change the RSA key, one of two things happen.

1. Game crashes and errors out with error 1-12828-1 (Corrupted error)
2. Game loads, but stuck on a black screen (Music is playing)

I have tried decrypting using FAGDec (Both as a ELF and SELF), I also attempted reinstalling the patch to no avail.

https://drive.google.com/file/d/1G5iDXngzuQWXwZJxSEqdLF4QmcwZkHA9/view?usp=sharing

I have attached my patch files, along with my RSA blob as well. I'm not sure if i'm doing something wrong when modifying the ELF (Maybe wrong hex values?)

[AntonnMal]

I'll take a look at the files a bit later, but I can tell you that the black screen is a symptom of an unrecognized patch folder.

[AntonnMal]

Your ELF files look good, so there should be no issues with the binaries, so I assume the problem is with your patch folder. If FAGDec doesn't display the patch folder as an option, then the OS won't see it also.

[crispbitload]

Oh shoot, I meant my rePatch folder. I put the patched eboot in the folder, and that's when I see the black screen. Weirdly enough I do see the loading wheel and such, but after I just hear audio but no video. Without the patched eboot the game loads fine.

[AntonnMal]

That's weird. I'm unable to recreate the problem (although I'm using my own eboot). I'm gonna assume that your rePatch folder contains the data directory from 6.30. Try only leaving the eboot in the rePatch folder and see what happens.

If the eboot was corrupted I'd assume that the game would crash immediately. But it may fallback to an 1.0 version. How long is the loading wheel? If it's around 1 sec then the OS is loading the 1.0 version, if it's around 5 secs then the 6.30-6.31 version is loading.

[crispbitload]

Here's a video of what is happening when I launch the game (With the Repatch folder only containing the eboot.bin)

https://www.youtube.com/watch?v=_z3el7gWR7M

It looks as if its loading the full 6.30/6.31 patch, but weirdly I can only hear audio (And sometimes if I spam the button enough the PSN sign in even pops up weirdly enough...)

By the way, what hex editor are you using? I am using HxD, so im unsure if the editor could be the problem?

Also, when getting the 6.30 patch the way you showed earlier, is the patch supposed to be the full 16-17GB or only around 200MB?

[AntonnMal]

Ok, weird. Can you try putting the whole (decrypted) 6.30/6.31 patch folder alongside the patched eboot in the rePatch folder?

I'm using imHex.

About the patch size: IIRC only the changed files are included In the patch set, so to get the full 6.30 patch you'd need to start at the latest full patch (somewhere along v5.xx, it's the first link in the patch list)

[crispbitload]

Sorry for the late response (Hope you had a good 4th)

Ill download the entire patch from 5.XX-6.30 and decrypt the eboot again, along with using imHex instead and report back.

[crispbitload]

Just wondering, how do you compile the eboot.elf back to eboot.bin?

The way I been doing it is

1. Use FAGDec to decrypt the 6.60 eboot.bin to both SELF and ELF and transfer it over
2. Use imHex (previously used HxD) to replace the RSA key
3. After, use vita-elf-inject to remake the eboot
4. Put it in the rePatch folder

I have also attempted to use vita-unmake-fself to create an ELF from the decrypted eboot as well.

Weirdly (and I don't know if this is normal) I receive a Compressed size too big! before compile.

Along with that, ill attach two photos of before replacing the RSA key and after (To make sure im pasting it in right)

Before

After

I'm feeling either i'm not compiling it right, or maybe im accidently overwriting something key in the eboot? Maybe my RSA key wasnt converted right, im not sure.

My RSA blob

Another weird thing, after upgrading the patch folder from 6.19 to 6.30 (Using the script that's the lowest version I get, and seems pretty complete), it gets stuck in a loading loop. I plan to try just installing 6.19 and see if the patch works, and upgrade one by one. Might also decompile 6.31 and do the RSA key replacement, to rule out if either the key or my compiling is the problem.

Thanks for helping and sticking with me this far, I really appreciate your help so far. :3

[AntonnMal]

The way I'm doing the injection is:

1. Using FAGDec I get the ELF file and transfer it to my PC
2. Using VitaShell I get the decrypted bin file
3. Replace the RSA key
4. Run the injector with the decrypted bin and modified ELF files as arguments
5. Copy the resulting bin file to the rePatch folder

Your RSA key looks fine. Also did a quick check using the files you provided earlier

I used the 6.31 decrypted bin and your ELF file. Injection seems fine, will check if it runs and I'll get back.

UPD: Injecting the ELF in the 6.31 version fails with the error C1-2722-3 (SCE_KERNEL_ERROR_MODULEMGR_INVALID_TYPE). Also learned that simple replacement of the bin file in the patch folder doesn't work (can't be decrypted). However, after injecting the ELF file in the 6.30 version it works just fine (I'll be honest and say that I didn't go further than the title screen). And injecting gave no output whatsoever.

UPD2: Maybe useful info: to decrypt the patch with the VitaShell you don't need to copy the folder to the patch folder, it can be anywhere (at least I've had it in downloads)

[crispbitload]

Last thing

I noticed when starting the ship server, it gives me an IO error relating to it not being able to find the item file.

In the image I have included the error, along with my current file path. I have made sure to have the data.zip unzipped in the same DIR (Obtained from artifacts).

(I should have said this earlier...sorry for multiple posts)

[AntonnMal]

The current upstream version is quite hard to setup as it turns out. The required data structure:

Also a few configuration edits are required:

• Change registration_enabled to true in the master ship cfg
• Prepend the paths with the data/ in the ship cfg

Probably need to work on documentation.

[crispbitload]

Forgot to clarify earlier, the compression message I received when injecting the 6.60 ELF with the 6.60 eboot (decrypted).

I'll downgrade PSO2 later today and try the decrypt the eboot via vitashell method instead.

On Sat, Jul 6, 2024, 2:54 AM Anton @.***> wrote:

The current upstream version is quite hard to setup as it turns out. The required data structure: image.png (view on web) https://github.com/PhantasyServer/phantasyserver/assets/24509063/47e04982-3a4e-4bb5-ad19-bd4c1c474cae image.png (view on web) https://github.com/PhantasyServer/phantasyserver/assets/24509063/cba2cb4c-3220-44e2-a7f8-57d39c9ab737

Also a few configuration edits are required:

• Change registration_enabled to true in the master ship cfg image.png (view on web) https://github.com/PhantasyServer/phantasyserver/assets/24509063/383c5631-0991-40e5-99c6-81558249ae36
• Prepend the paths with the data/ in the ship cfg image.png (view on web) https://github.com/PhantasyServer/phantasyserver/assets/24509063/2cf96885-1fa1-4e1e-afc7-94b5e766c619

— Reply to this email directly, view it on GitHub https://github.com/PhantasyServer/phantasyserver/issues/18#issuecomment-2211724064, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDRY262HVVK3M2ZKV2GALTZK65HDAVCNFSM6AAAAABKEVVSDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJRG4ZDIMBWGQ . You are receiving this because you authored the thread.Message ID: @.***>

[crispbitload]

So after decrypting both 6.31 and 6.30, merging them (replacing 6.31 files with 6.30's) and putting them in the repatch folder...

I'm happy to say it works!! The game loads (albeit a bit slowly).

Now comes to patching the FPK file to make it connect to the server (which I think I can handle on my own)

But, im having another issue related to the private server host software.

Regardless if I set my local IP or localhost, I keep getting this error. I have tried with my firewall off as well, and I have set my PC's IP as static/reserved (Along with my vitas if that makes any difference).

[AntonnMal]

Ok, what is master_ship field in the ship config is set to?

[crispbitload]

EDIT: Ok, scratch that, its now connecting but with a different error

[AntonnMal]

This error is due to disabled ship registration. Have you set the registration_enabled in the master ship cfg?

[crispbitload]

I feel dumb...

So notepad on windows was not saving my config files...Changing to another notepad tool and editing the files again fixed the problem!

Ship is working and starting up now!

The final part, for replacing the IP address in the lua, do I include the port number or just leave the port out? Also for the 32-bit strings, what do I change exactly on that?

Thank you again, I'm in the final stretch!

[AntonnMal]

Here's how to do it (at least how I remember doing it long ago):

• For example, I want to replace the first ship IP address. From the screenshot below we can see that the length of the string is before the string itself (null terminator included).
  • Let's say that I want to replace it with 127.0.0.1. That string is 9 + 1 (null terminator) bytes long, so I'll write it down.
  • Next I'll write the address itself with the terminator.
  • Then I'll remove the now extra bytes from previous address.

[crispbitload]

So my IP being 12 characters (192.168.1.11) would equal 0C

Does this look all good? (I'm not really experienced in hex editing)

[AntonnMal]

You forgot to count the null byte, so your length will be 13 (0D).

[crispbitload]

I'm glad to say after all the work, it is now fully working and connecting!

I really appreciate all the help given, and apologize for how long this ended up being (me being a noob lol).

If you have a donation page of some sort, id love to donate (once I get paid), also is there any way I (or someone with little code experience) could help?

Finally, could the private server run on something bare/basic like a bare linux install (Think like ubuntu server and such)?

Again, I really appreciate all the help.

EDIT: The English patch fully works as well alongside the server patches, if you want you can grab it at https://arks-layer.com/pso2v.php

[AntonnMal]

Good to hear that! Gonna answer your questions:

• Donations are a no-go for a long time due to certain geopolitical reasons. And also this is kinda a hobby project.
• I guess the main ways you can help is with understanding packets or gathering EN and JP item names and descriptions from live. A lot of packets are still unknown (e.g. nothing from alliances, emergency trials, personal quarters, crafting and etc is done).
• Server can definitely be run from a basic linux install (my main dev env is linux). Artifacts should work, but IMO on linux it's better to compile yourself. You'd only need rust compiler and lld (because I've made lld the default linker in the .cargo/config.toml, can be easily reverted)

[crispbitload]

Got it! I play frequently on global, so I can def grab a lot of the English item descriptions (also have a JP account as well).

Arks-visiphone also has a good wiki on some of the items, along with the various shops and what they contain https://pso2.arks-visiphone.com/wiki/Portal:PSO2.

Lastly, how does the item formatting work in the items name json file? Along with that, is there a way to add these items to your inventory for testing?

[AntonnMal]

I don't fully know how sega's formatting works but here's an example of what it would look like in the json file:

[
  {
    "item_type": 3,
    "id": 70,
    "unk3": 0,
    "subid": 1
    "en_name": "Marbled Meat",
    "jp_name": "",
    "en_desc": "Meat that can be found on <c f0df60>every planet<c>. Very<br>soft and marbled with high-quality fat.",
    "jp_desc": ""
  }
]

Capturing packet with a proxy helps with this, since all names and descriptions are streamed from the server.

There is a command to add items to your inventory, but it's kinda scuffed: !add_item <item_type_id> <item_id> <item_subid>.

[crispbitload]

Interesting, also for commands do you type it into the server command line or ingame?

For packet capture, would the global version or JP version work best? Also, what proxy could I use to capture the packets?

Finally, where could I submit the packets?

[AntonnMal]

The commands are for ingame chat.

Here are my proxy and RSA injector. I know that they work on global, unsure about JP. For capturing any version will do.

I'd advise against submitting packets publicly since they contain Sega ID passwords and maybe other PII, but if you are really ok with that, then you can email them to me (email in profile).

[crispbitload]

Alright, thanks for letting me know that! I might try the packet capture out later on (and maybe take a Crack at the item index)

Anyway thank you a lot, I hope the project keeps going well and really appreciate all the help given!

[AntonnMal]

As the original issue was solved, I'll close this issue.