FE9 POR - changes saved to system.cmp hangs the game indefinitely

[swdtech]

Rewrite-Alpha-2-3-2 - FE9:

Any saved changes made to "system.cmp" causes the game (when using that new file) to hang indefinitely (no sound or inputs taken) after starting a new game or loading a saved game. This occurs even if the saved changes replicate the same values and/or entries as the original game.

Booting the game, the title screen, and the menu otherwise work as normal, and changes saved to any other file also work fine.

Tested with the JPN and USA versions FE9 on the Windows releases of Dolphin 5.0-16101-x64 and its latest version(2409-x64); same errors occur.

[thane98]

Are you getting this consistently?

I am able to modify system.cmp through Paragon and run things in game just fine. Sal has also used made extensive changes to system.cmp through Paragon for his recent FE9 run without hitting this.

I've occasionally run into hangs around the start menu, but it's not consistent. The testing section of the Dolphin wiki also has a report of hangs on the start menu which pre-dates Paragon's FE9 support: https://wiki.dolphin-emu.org/index.php?title=Fire_Emblem%3A_Path_of_Radiance

Disabling dual core in Dolphin's settings might help.

Open to looking at this further, but I suspect it's not caused by Paragon.

[swdtech]

This is 100% consistent for me; it will always hang in the described ways if system.cmp is modified.

Disabling dual-core (speed up) doesn't fix it for me, nor has any other setting or graphic option change I've tried. I might test it on real hardware instead later.

Dolphin throws out a bunch of warnings in the JPN version right before the point where it hangs. On the USA version, it threw out different warnings repeatedly for so long that I stopped bothering to finish going through them all(200+ warnings). Every warning in both versions related to having invalid reads on certain areas, and they're all the same regardless of how system.cmp is modified.

Also, Dolphin says that the game is still running at 60 FPS (or whatever FPS you set it to) while at this point(which is why I don't think its a crash or freeze, but I might be wrong).

JPN version:

Warning

Invalid read from 0x00000008, PC = 0x80163720

Invalid read from 0x0000000c, PC = 0x8016373c

Invalid read from 0x00000014, PC = 0x8016360c

Invalid read from 0x00000024, PC = 0x80163644

Invalid read from 0x00000018, PC = 0x8016360c

Invalid read from 0x00000028, PC = 0x80163644

Invalid read from 0x0000001c, PC = 0x8016360c

Invalid read from 0x00000020, PC = 0x8016360c

Invalid read from 0x0000002c, PC = 0x80163644

Invalid read from 0x00000000, PC = 0x801637a4

[swdtech]

I did a quick hex edit test on a copy of both the original JPN system.cmp file and the original USA system.cmp file; changed the Iron Sword from 46 Durability / 5 Might / 90 Hit% to 50 Durability / 9 Might / 85 Hit%, and the game worked just fine for both versions(and its weapon stats were correctly edited when viewed in the inventory and character combat statistics).

When doing the same edit for the JPN version but with Paragon instead(same original system.cmp file); the hex values edited are moved one byte to the right on the newly saved file, but also the value of every single other byte in the file (from what it looks like) is changed to some other value instead.

For the USA version, Paragon seems to (I think) displace the hex values of the Iron Sword (or other item edit) more significantly, and the values of the other bytes in the file are also all changed to some other value.

Despite this, Paragon still reads any contents in both system.cmp files as if this doesn't happen, while also still reading the actual changes you made and saved correctly(despite them being in different bytes).

JPN Original System.cmp (Iron Sword Durability / Might / Hit% highlighted):

JPN Hex Editing System.cmp (Iron Sword Durability / Might / Hit% edited and highlighted):

JPN Paragon Editing System.cmp (Iron Sword Durability / Might / Hit% edited and highlighted):

USA Original System.cmp (Iron Sword Durability / Might / Hit% highlighted):

USA Hex Editing System.cmp (Iron Sword Durability / Might / Hit% edited and highlighted):

USA Paragon Editing System.cmp (Iron Sword Durability / Might / Hit% edited and highlighted, I believe):

[thane98]

Thanks for looking into this further - try this build: https://www.dropbox.com/scl/fi/lz1ldrrk0gx4wzs5zyc8j/Paragon-Windows.zip?rlkey=srqmps0p7dl245tcanhxqt1nv&st=cldn8s6w&dl=1

For context, system.cmp is similar to a zip archive in that it's a compressed file containing other files. Paragon decompresses system.cmp and pulls out individual files for editing. When you save, it re-inserts changed files and then recompresses everything. This is why positions shifting in system.cmp does not break Paragon. It's also why other bytes may change when positions shift - it impacts compression.

Previously, Paragon did not generate a structurally identical FE8Data.bin. It was effectively the same, but some data was reordered. This caused the decompressed version of system.cmp to look a little different which would affect its compressed form (shown in your screenshots) as well. The build I've linked won't reorder anything.

I've confirmed that the decompressed version of system.cmp is identical when round tripped through Paragon now. The compressed version is still a little different (likely the LZ10 implementation I'm using doing something different). However, it's the same length as the original and the bytes in your example are not shifted. I tested in Dolphin 2409 and was able to start a new game and play through the first chapter just fine. No warnings / errors about bad reads in the logs either.

[swdtech]

Thanks for sending the build, so far it seems to be working fine for the USA version and all of my saved games I have for it(with the 2 previously mentioned versions of Dolphin).

Unfortunately for me it's giving me the exact same warnings and indefinite hanging as before still for the JPN version(tested with the 2 previously mentioned versions of Dolphin again).

[swdtech]

Re-did some tests and looking at the hex editor again, for the JPN version of the game I believe that the build you sent was still re-ordering data in system.cmp like it used to before(which seems to confirm what gives these errors).

Is it possible that you can apply this for the JPN version as well?

[thane98]

Yep, I will investigate this as well. I haven't done much testing with the JP version of FE9. I suspect there regional differences since FE10 had a similar problem.

Not in a good spot to work on this in the middle of the week though. Will take a look this weekend.

[swdtech]

That's fine, thanks for the work.

[thane98]

Thanks for waiting. There were structural differences in chapter data between JP and NA / EU versions of FE9. Fixed those and Paragon's now round tripping the files with 0 differences.

Let me know if this build fixes things: https://www.dropbox.com/scl/fi/jbklsefpwc33wwnfegvzn/Paragon-Windows-FE9-JP-Fix.zip?rlkey=ateirg6zmoi1wg7qe3vrv2x4g&st=19m94gdp&dl=1

[swdtech]

Tested the new build for a while on the JPN version, everything seems to work fine now(no issues or warnings/errors).

Thank you.