search

TekkaGB / AemulusModManager

GNU General Public License v3.0
79 stars 20 forks source link

readme

Aemulus Package Manager

Introduction

The wait is finally over! No longer will you have to manually merge conflicting bin, bmd, pm1, bf and tbl files found in different mods. This is the latest and greatest mod package manager, made specifically for Persona 4 Golden on PC, Persona 3 FES, Persona 5, and Persona 5 Strikers.

How to Use

Prerequisites

For preappfile to unpack for Persona 4 Golden, you'll need .NET Core 3.1 Desktop Runtime x64

For YACPKTool to unpack/pack cpk's for Persona 5, you'll need Microsoft Visual C++ 2010 Redistributable Package (x86)

For Persona 4 Golden on PC, make sure you've set up Reloaded-II and the P4G mod loader first and foremost.

For Persona 4 Golden on Vita, setup the proper patches for mod loading from mod.cpk/m0.cpk/m1.cpk/m2.cpk/m3.cpk

For Persona 3 FES, setup HostFS for mod loading.

For Persona 3 Portable, setup the proper patches for mod loading from bind/mod.cpk/mod1.cpk/mod2.cpk/mod3.cpk

For Persona 5, setup the proper patches for mod loading from mod.cpk. Just ignore the Setting Up Mod Compendium section since you'll be using Aemulus instead.

For Persona 5 Royal (PS4), setup the proper patches for mod loading from bind/mod.cpk/mod1.cpk/mod2.cpk/mod3.cpk

For Persona 5 Royal (Switch), nothing is need other than Aemulus!

For Persona 5 Strikers, nothing is need other than Aemulus!

For Persona Q and Q2, setup the proper patches for mod loading from mod.cpk

To easily download all prerequisites at once, I recommend using Pixelguin's All-in-One Installer.

Pointing to the Correct Output Directory

After unzipping the download, just double-click AemulusPackageManager.exe to launch the program.

The first thing you'll want to do is click the Config button on the top left. From there click the Select Output Folder button on top.

Unpacking Base Files

This feature unpacks mergeable files locally on your system. This way, Aemulus can grab the unchanged assets in files like init_free.bin immediately, which saves a lot of time in the long run when building and downloading mods.

You only need to do this once for each game you want to mod. To do so, open the Config menu and click Unpack Base Files, if the previous paths weren't setup already, it will prompt you to select the appropriate file/folder to start unpacking. You'll find the unpacked files for Aemulus in your Original/ folder.

Persona 5 Strikers doesn't need to unpack any base files but instead makes copies of the original files and rdbs that will be patched and replaced on the first run through.

Adding Packages

Once you've set up Aemulus, you can install packages either directly from GameBanana or through the "Download Packages" section built into Aemulus. You can also drag and drop zipped packages that you have manually downloaded onto the new packages button to automatically install them.

You can also click the New button on the top right to create a directory along with metadata and a preview. The directory will pop up when you click confirm and you can drop the contents of the mod inside.

Setting Up Your Loadout/Package Priority

Next, you'll need to set up your package loadout. Packages are disabled by default, so enable the ones you want by checking the box to the left of each package.

You can drag and drop mods to move them up and down in order of priority. A higher priority mod has its files merged later, meaning it will overwrite more packages and fewer packages will overwrite it.

Click the double arrow button on the top right if you'd like to switch which way the loader determines which mods are higher priority.

Remember, any mod will work with Aemulus, but the mod creator has to provide a mods.aem file for bin merging to be supported. Without that file, a package with a bin file will overwrite the file completely, so it's recommended that you put non-Aemulus mods at the bottom priority of your loadout.

Final Step - Merging and Building Your Loadout

Please note that Aemulus will completely erase the previous contents of your output folder (with the exception of Persona 5 which makes and clears a 'mod' folder in the output directory selected) when creating a loadout. Back up your current folder if you aren't sure about the changes you're making, and make sure not to use a location like Desktop for your output.

Finally, to merge all supported files and build your loadout, just click the Hammer button at the top to build. The console at the bottom will print what Aemulus manager is currently doing.

Don't worry if it seems like the console is stuck on "unpacking" something. Some files take longer than others to unpack.

A window will pop up once everything is complete. Congratulations, you're all done! Now when you run your chosen Persona game, the game will utilize your brand new loadout (that is if you setup mod loading correctly).

Sharing and Creating Loadouts

As of version 5.3.0 Aemulus supports package loadouts. These allow you to quickly switch which packages are enabled, hidden and the order of packages. These loadouts can also easily be shared and imported.

To share loadouts you can simply send anyone the loadout xml which is located in the Config/Game folder in Aemulus. Once you have a loadout xml you can import it by dragging and dropping it onto the new package button. This will add the loadout to your list, also prompting you to download any mods in the loadout that you don't already have installed (if they have a valid link).

How Bin Merging Works

The mods.aem File

Aemulus now supports loose file merging, but this section may still be useful for mods that edit bin files in other folders or if you want to add Aemulus support to a legacy mod.

In order to support merging bin files, each mod/package that edits the bin file needs a mods.aem file in its folder. This is just a text file with a changed extension that you can open with Notepad or any other text editor.

Inside is a list of all the files that the package edits. Follow these instructions when typing out the file paths: One file path per line. In the path, make sure to take out any .bin, .arc, .pak, and .spr extensions (for example, "init_free.bin" becomes "init_free"). Make sure to use '\' and not '/' between directory levels. If the file being addressed is a Texture within a .spr file, give it the .tmx extension. Also note that for the specific case of file paths inside Persona 5 event paks, if there's ../../../, just ignore that and just use the rest for the file path.

For Persona 5, I implemented custom merging for .spd files. Due to sometimes having duplicate dds names within, I implemented the naming system of [ID].dds. You can find the ID number at the top of the information shown when viewing the dds texture in Amicitia. The sprites are also mergeable using .spdspr files. Due to many of them using special encoding for Japanese names, i decided to just have them named .spdspr. Again the ID's can be found when viewing the sprite information in Amicitia.

You can find these exact paths using a tool like Amicitia. For example, SeaGuardian's Bearable Fast Forward mod's mods.aem would look like this:

And here's an example what the path looks like in Amicitia:

Notice that instead of data00004\init_free.bin\init\camp.arc\event_skip.spr\sankaku, the file path is is data00004\init_free\init\camp\event_skip\sankaku.tmx, which follows all of the rules above.

Do note that mods.aem files are unnecessary for packages that don't edit bin files. Also note that if a bin file doesn't have a path indicated inside mods.aem, it will overwrite the entire bin instead of merging.

As of v1.2, mods.aem is no longer required to merge bin files, although it is still supported for everyone who has converted. You can now just include the loose files in the same folder paths that you listed in the mods.aem file and it'll merge them over the Original bin files you unpacked on setup.

The Actual Merging Process

If you're curious how the program actually works, I'll run you through it here.

  1. Creates a list of enabled packages in the order indicated in the UI.
  2. Deletes the current mod loadout found in your Steam game directory.
  3. Goes through each packages' contents and copies and overwrites all contents (excluding mods.aem and .tblpatch files) it over to the mods directory you had to select.
  4. If there's a conflict with a bin file, it unpacks the entire bin and refers to mods.aem to copy over the loose files to the mods directory, then deletes the unpacked files.
  5. Merges all the loose files with the bins then deletes all the loose files.

Bf/Flow Merging

To create a mod that supports bf merging you replace the bf in the package with a flow file which uses hooks to change functions. For example if you had a mod which edited f007.bf, you would simply put the .flow file which you used to create the bf in the same place with the exact same name (so f007.flow in this case). You do not need to supply your own f007.bf, in fact if you do it will be replaced by an original copy anyway when building.

When Aemulus builds your mod it will see that there is a flow file and then copy the original bf from the game's unpacked files into the same folder as that flow. Atlus Script Compiler will then attempt to compile the flow, the same way you would do it manually. Also, if there are any bf files with the same name and location in the previous packages instead of copying from the original files it will copy the bf from them.

Bmd and Pm1 Merging

Bmd and Pm1 merging is done entirely automatically, meaning no additional work is required for mod authors. The way it works internally is the pm1s or bmds to be merged are compared with their original version by decompiling them into msg files. Then any messages that are different to the original are added to a list, if both files edit the same message the higher priority one will be added. Then all of the changed messages are replaced in a msg files which is finally recompiled as a bmd or pm1 to be used.

How Binary Patching Works

Newly added in v5.5.6, binary patching is a feature that takes .bp files from a binarypatches folder within a package to overwrite/append bytes to any file.

Structure of .bp Files (for modders)

An example of the setup of the file is as follows:

{
  "Version": 1,
  "Patches": [
    {
      "file": "init_free\\init\\cmpTable\\cmpConfigHelp.ctd",
      "offset": 69,
      "data": "42 00 69"
    }
  ]
}

First and foremost, make sure you have Version set to 1. Otherwise, none of the patches will be read.

Next, there is a list of Patches.

Patch

Common Issues

Binary patching only works on files that are already found in the output folder from perhaps another mod or from the original folder. If you unpacked files from before v5.5.6, you might need to unpack again. If the file that needs to be patched still isn't unpacked from the Original folder, let me know and I'll add it in

How Spd Patching Works

Spd Patching is a new feature added to v6.3.1 that allows you to add a modified texture file to a .spd and change the sprite entries of your choosing to use that new texture, all without replacing any original textures. This will allow sprite mods that modify the same texture to be completely compatible if they target different sprites

Structure of .spdp Files (for modders)

An example of the setup of the file is as follows:

{
  "Version": 1,
  "Patches": [
    {
      "SpdPath": "init\\camp\\p5camp_00SPD.spd",
      "TextureName": "p5camp_11_patch.dds",
      "TextureID": 6,
      "SpriteIDs": "233 234 235 236 237 238 239 240 654 655 658 707 708 709 710 711 712 713 714 715 716 717 718 719 "
    }
  ]
}

This example was generated with SecreC.'s Spd Patching tool

Here is the list of Patches.

Patch

Common Issues

Spd patching only works on files that are already found in the output folder from perhaps another mod or from the original folder. If you unpacked files from before v6.3.1, you will need to unpack again. If the file that needs to be patched still isn't unpacked from the Original folder, let me know and I'll add it in.

How Table Patching Works

Table patching is a feature that was carried over from Inaba Exe Patcher (formerly known as Aemulus Patcher/Exe Patcher). It takes .tblpatch files from the top layer of your Package folders to modify .tbl files found in init_free.bin for P4G, table.pac for P5, and BTL/BATTLE for P3F.

Structure of .tbp Files (for modders)

For more readablity and support of expanding .tbl files (mostly seen in Persona 5) all in one file, I added this new format. Old .tblpatch files still work for those who don't want to convert yet but I highly recommend switching over. This new file still goes in the tblpatches folder. Q/Q2 patches can only use this format.

An example is show below:

{
  "Version": 1,
  "Patches": [
    {
      "tbl": "ITEM",
      "section": 1,
      "offset": 14448,
      "data": "64 72 69 70"
    },
    {
      "tbl": "SKILL",
      "section": 0,
      "offset": 3000,
      "data": "4C 4F 4C"
    },
    {
      "tbl": "NAME",
      "section": 11,
      "index": 8,
      "name": "Black Leotard"
    },
    {
      "tbl": "NAME",
      "section": 11,
      "index": 35,
      "name": "True Self Lo[80 69]ok"
    },
    {
      "tbl": "NAME",
      "section": 11,
      "index": 143,
      "name": "New Cinema Outfit"
    },
    {
      "tbl": "NAME",
      "section": 11,
      "index": 179,
      "name": "Ultramarine Outfit"
    }
  ]
}

First and foremost, make sure you have Version set to 1. Otherwise, none of the patches will be read.

Next, there is a list of Patches. Note that NAME.TBL patches have different members from every other tbl.

Normal Patch

NAME.TBL Patch

Structure of .tblpatch Files (for modders)

First three bytes of the .tblpatch file is the tbl id, or 3 letters that indicate which tbl file it is. P4G (PC/Vita) TBLS:

P3F TBLS:

P3P TBLS:

P5/P5R TBLS:

In this example, we can see that MSG is the tbl id. So this .tblpatch will be editing MSG.TBL.

Next 8 bytes indicates the offset at which you want to start overwriting the hex. This will also be in hex rather than decimal. The offsets will be found in the individual .TBL file you identified in the first three bytes. You can use a decimal to hex converter like this one if you only know the decimal offset.

Here we can see that the offset is 00 00 00 00 00 00 02 50 (or 592 in decimal).

Finally, the rest of the bytes will be used to overwrite the hex starting at the specified contents.

In this case we'll just be using 57 65 65 62 00 00 00 00 (Weeb 00 00 00 00) to overwrite the hex found at the previously stated offset of 592 in MSG.TBL.

I recommend for you to use 010 Editor and these templates if you want to mess with .tbl files to create .tblpatch's. Do note that my examples are really small to easily fit in this description but you can overwrite as much bytes as you want so that you don't need to create too many .tblpatch files. You can also utilize (P3F, P4G, P5 only) T-Pose Ratkechi's Aemulus TBL Patcher, to easily convert your edited tbl's to .tblpatch's.

As for NAME.TBL in Persona 5 and Persona 5 Royal, the file is setup differently so that instead of the 8 bytes used for the offset, it is instead split into 1 byte for the section number and the next 2 bytes for the index. This also supports expanding more entries in NAME.TBL so feel free to use larger indices. In the future, I might refactor the rest of the tblpatches to follow this format.

For reference, here's the section numbers in NAME.TBL in Persona 5:

The Actual Table Patching Process

  1. Extracts all .tbl files if inside an archive.
  2. Goes through each package folder from the bottom up (again after the merging process) and applies each tblpatch file it finds to the .tbl file it specifies.
  3. Repacks the edited .tbl files into init_free.bin or table.pac (unneccesary for Persona 3 FES). Deletes the temporarily extracted/edited tbl files.

ACB/AWB Merging

Currently only for Q, Q2 and P4G Vita until filelists for unpacking base files can be updated. To support merging, make a folder with the archive's name (without extension) inside your package folder where the archive would go, then place your modded audio inside. For paired archives, create ONE folder with the name of the acb, and place your audio for both archives inside. Awb audio should be named in the format index_streaming.extension.

The way these are merged is that Aemulus copies the original acb and/or awb into your build directory and unpacks it, either with Sonic Audio Tools for lone acbs and paired archives, or Awb Tools for lone awbs. It then copies your modded audio into the resulting folder, repacks the folder back into an acb and/or awb and deletes the folder. Note that the merging process assumes paired awbs will always either have the same name as the acb or be named acbname_streamfiles.awb; if this isn't the case file an issue or a pr.

Loadouts

You can create loadouts per game by using the dropdown menu on the top left. Clicking Add new loadout... will create a new loadout with the option to copy over your current one. You can also right click to hide/unhide packages and use the eye button to actually hide/unhide them.

P3F PCSX2 Cheats Support

You can now add a path to the PCSX2 cheats folder in P3F's config. It will copy over all pnach files from a cheats folder within a package to the PCSX2 cheats folder. On rebuild, Aemulus clears only all of the copied over pnach files from that folder.

P3P PPSSPP Cheats Support

You can add PPSSPP cheats to the game by placing ini files in a cheats folder at the root of a package. Any cheats found in any ini files (the name of the file does not matter) will be appended to your PPSSPP cheats file which can be configured in the P3P config.

If the cheat already exists it will be replaced and its status will be preserved (if it was disabled it will stay disabled, etc). Also, existing cheats will never be removed from the file, if you want to disable a cheat do so from within PPSSPP.

Prebuild.bat Support

You can also include a Windows batch file named "prebuild.bat" inside the Aemulus package that would be ran right before copying its contents over to the output folder.

Persona 4 Golden Preappfile Append Support

For modders of Persona 4 Golden who want to utilize preappfile's append feature, simply put the appended contents inside a folder called preappfile.

Persona 5 Strikers RDB Patching Support

Thanks to Raytwo's rdb_tool, Aemulus is able to patch the rdb files to use unpacked files placed in the data file. Modding linkdata is not yet supported by Aemulus.

Process

  1. Backs up all rdb files in motor_rsc and original file files in data in the Original folder on first run through.
  2. Restores data folder to have all original files only.
  3. Replaces all rdbs with the backed up rdbs.
  4. Uses rdb_tool to patch every single rdb.

Setup of Packages

All files to be patched must be placed in a folder named data. They can go by the names 0x.file, ., and/or the actual file name. Aemulus will rename them to the proper name needed for patching.

For example, 0x948554ec.file, 948554ec.g1m, H0000_Joker.g1m are different naming conventions of the exact same file, all of which will work.

Metadata

As of version 1.3, metadata has been added to provide more info for each package. Along with name, you can now display author, version, link, description, and even a thumbnail.

Package.xml

Below is an example of the contents of a Package.xml for one of mods, Persona 5 Overhaul:

<?xml version="1.0"?>
<Metadata xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <name>Persona 5 UI Overhaul</name>
  <id>tekka.persona5uioverhaul</id>
  <author>Tekka</author>
  <version>2.19</version>
  <link>https://gamebanana.com/gamefiles/12278</link>
  <description>This mod tries to replace every single aspect of the game that I can in a Persona 5 style.</description>
</Metadata>

Creating/Editing Metadata

There are multiple ways to create/edit Package.xml files:

If you place a mod folder in the Packages directory and refresh, it will automatically create a Package.xml with the name as the name of the folder with the rest of the metadata blank. You can then edit the Name, Author, and Version by double clicking on their cells in the Package grid. You can modify all parts of the metadata by right clicking the row and selecting Edit Metadata. A window with the current metadata will pop up and you can edit whatever you like here.

I also added a New button on the top right that brings up a window for you to type in the metadata and have the Package.xml be created for you. When you type in the name and author it autofills a suggested ID but you can feel free to change it if you want. Also included in the window is a file selector for the preview. Choose a png that you want to display as the preview when using this option. When you create the package, it'll create and open a folder with the mod name and version as well as the Package.xml and preview chosen named as Preview.png inside. You can now put the contents of the mod/package inside this folder to be used in your Aemulus loadout and/or to distribute.

Versioning

If more than one Package has the same ID in their metadata, only the highest version number will show. Version numbers that are nonexistent or can't be parsed due to having letters will be counted as null which is sorted less than 0. In the case of packages with the same version number and ID, the folder last modified will be kept.

An option was also added to the config menus to delete older versions. They are kept by default and just simply hidden from view in the app.

Extracting/Zipping

You can drag folders/zips over the New Package button (Folder with +) to extract/move them into the proper directories. This is a QoL feature implemented due to many end users messing up the specific directory structure many mods should have. For modders, there's also the right click option to zip the package as a .7z to the file location of choice.

Compatibility with P4G Music Manager

Since Aemulus Package Manager deletes the entire mods directory everytime you rebuild, it also deletes the mods/SND folder which P4G Music Manager utilizes. To add compatibility I added a checkbox in the Config menu to Empty SND Folder. By default, it leaves the SND folder in tact. Enabling it will delete the SND Folder.

Launching the Game from the Manager

A new QoL feature added in v1.2 is the Launch button. This is used to be able to launch your modded game straight from the package manager after building your loadout. You can setup the paths for this to work in the config menu. Under Launch Shortcut Setup click browse to select the paths required. Once you picked valid exe's, the Launch button on the main window will now start the game for you.

Ignore.aem

Creating a new package or building an existing one will create a file called Ignore.aem. The purpose of this file is to allow mod creators to keep loose files in their mod without having those files moved and built into the mod output. To make use of this file, open the file in your favorite text editor and add the path to the file (or directory) to the first available line

  script\init.flow
  camp\cardTex\c_card1e.dds
  scheduler\hook\
  field\panel\fldPanelLmap\

For Mod Creators - Aemulus Logo Overlay

If you'd like to include an Aemulus overlay in your mod thumbnails to indicate that it supports Aemulus (which just means having a named folder and a mods.aem if necessary), you can use the file aemulus_overlay.png included in the download. Thanks to Pixelguin for designing it.

The overlay looks best on thumbnails that are 1920 x 1080.

FAQ

What makes a mod Aemulus Compatible?

All mods are compatible with Aemulus, some just might need a simple directory change. The contents of the directory should be in the root folder of the Package alongside the Package.xml and optional Preview.png

Also check to see if some of the modded files are located in your unpacked Original folder. If so, extract the loose files or create a mods.aem to make them mergeable. Otherwise, it'll just overwrite rather than merge them.

What about mods that are made for Mod Compendium?

These mods just need to have a simple directory change to make them work in Aemulus. Just drop these mods in the Packages folder and hit refresh. Aemulus will automatically convert these mods for you.

Why is my antivirus acting up?

For some reason the latest update triggered some of my testers' antivirus programs. Simply make AemulusPackageManager.exe an exception in order to use it. The code base is now open source so feel free to look through it and even build it yourself if you're still worried about the antivirus notification.