version collision after packing in windows

[kennyfortune]

actually i was wrong about being able to just cat the stub and the data into a combined file; as AGS tries to locate the start of the data via an "end signature" containing the offset, rather than just searching for the start signature, so doing it manually as i suggested would involve using a hex editor to change the offset 16 bytes before the end as a 4 byte little endian value to contain the size of the .exestub file. since this is impractical for casual users, i now added the -e option to agspack which does everything automatically.

Originally posted by @rofl0r in https://github.com/rofl0r/agsutils/issues/16#issuecomment-872932851

Thanks ! it works！Now it can pack correctly，but i encounter another problems， When opening the exe file, it shows that Error: Unable to load the room file 'room41.crm' Format version not supported Required format version: 23117, supported 17 -33 it got a collision of versions

[rofl0r]

oh huh. agsutils actually does a couple short-circuits and writes data in a half-ways modern format that's simpler to emit than the oldest format as well as the newest formats. since my own use-case was always to load the game file into latest stand-alone engine, this was "good enuff". however this can of course not work when using the exestub of an ancient AGS version, that's older than the format we emit. the only thing we can do about it is to either prepend the stub of a newer ags version, or emitting the data in a format 100% compatible to the used engine version, which is probably a lot of tedious work. would you mind pointing me to the specific game you're using?

[kennyfortune]

oh huh. agsutils actually does a couple short-circuits and writes data in a half-ways modern format that's simpler to emit than the oldest format as well as the newest formats. since my own use-case was always to load the game file into latest stand-alone engine, this was "good enuff". however this can of course not work when using the exestub of an ancient AGS version, that's older than the format we emit. the only thing we can do about it is to either prepend the stub of a newer ags version, or emitting the data in a format 100% compatible to the used engine version, which is probably a lot of tedious work. would you mind pointing me to the specific game you're using?

The game is called "Strangeland" which is on sale on steam. I am doing the translation work for this game. Great appreciate for your help

[rofl0r]

i acquired the game, however it uses this windows plugin https://github.com/Dualnames1/AGSWaves/blob/master/agswave.cpp which i'm currently trying to circumvent (since i dont have any windows systems to test on). i'll report back once i got it working.

[rofl0r]

i managed to reproduce the problem, and i suspect that the author of Strangeland, which is in all likelihood the same author as the above mentioned spaghetti code plugin, modified the open source AGS game engine in a proprietary way.

i applied the following patch to the engine to circumvent the room version issue:

--- a/Common/game/room_file.cpp
+++ b/Common/game/room_file.cpp
@@ -84,6 +84,8 @@ String GetRoomFileErrorText(RoomFileErrorType err)
 static HRoomFileError OpenRoomFileBase(Stream *in, RoomDataSource &src)
 {
     src.DataVersion = (RoomFileVersion)in->ReadInt16();
+    if(src.DataVersion > kRoomVersion_Current)
+src.DataVersion = kRoomVersion_Current;
     if (src.DataVersion < kRoomVersion_250b || src.DataVersion > kRoomVersion_Current)
         return new RoomFileError(kRoomFileErr_FormatNotSupported, String::FromFormat("Required format version: %d, supported %d - %d", src.DataVersion, kRoomVersion_250b, kRoomVersion_Current));
     return HRoomFileError::None();

but then we get the next issue:

Error: Unable to load the room file 'room41.crm'.
Unknown block type.
Type: 73, known range: 1 - 9..

//edit: turned out it's due to usage of huge files > 2GB.

[kennyfortune]

Thanks！ It seems that it hard to push ahead with the work. Sorry to have occupy your precious time.

[rofl0r]

hmm, maybe i'm wrong and there is one more possibility; and that is that in latest engine version layout of room files may have changed and a mistake happens when injecting .crm files. i will investigate this today.

[rofl0r]

@kennyfortune when you run the original StrangeLand.exe from a terminal window, does it display the version number of AGS used, and if not, somewhere else in an about/help/credits dialog or something like that?

[rofl0r]

i suspect it could be caused by this https://github.com/adventuregamestudio/ags/pull/1183 which was originally merged to ags4 branch which i never meant to support, but as i now found out to my surprise was backported to ags3.

[kennyfortune]

the entire message is 'An internal error has occurred. Please note down the following information. If the problem persists ,post the details 0n the AGS TechnicaI Forum. (ACI version 3．5．0．26） Error:Unable to load the room file'room41.crm'. Format version not supported. Required format version:23117，supported17一33．'

[ivan-mogilko]

i suspect it could be caused by this adventuregamestudio/ags#1183 which was originally merged to ags4 branch which i never meant to support, but as i now found out to my surprise was backported to ags3.

The serialization logic was of course backported, in case we have any more changes to the game & room formats (and simply to detect formats correctly). I thought it was your meaning that ags should now only add new data using these blocks, regardless of branch. We do not have any actual extra blocks supported yet though. EDIT: if it matters, this feature was backported in 3.5.1, so 3.5.0 (used to compile Strangeland) did not have it yet.

Running original "Strangeland" with the engine, room 41 has format 33. The format check is before any blocks are read, which may mean that in the above situation the asset (room) file offset is wrong in the pack's table of contents and the engine begins reading the room in the wrong position.

[ivan-mogilko]

@kennyfortune because I probably missed the beginning of this discussion, could you mention which files did you change in the game (which you repack)?

[kennyfortune]

@kennyfortune because I probably missed the beginning of this discussion, could you mention which files did you change in the game (which you repack)?

the '.crm' file. At the end of the crm file, there are text data. Each sentence start with '00 26' and ends with '00'. I use hexeditor to modify this text data to test whether it could be translated into another language. however the problem of packing occurs whether i edit the data or not. it may be not allied to the problem

[ivan-mogilko]

At the end of the crm file, there are text data. Each sentence start with '00 26' and ends with '00'. I use hexeditor to modify this text data to test whether it could be translated into another language.

Sorry if this is not related to the problem, but have you tried generating TRS file out of the game data? I believe there might be tools around for generating TRS from the game and then compiling TRA which you place in the game dir as a regular file; this way you don't have to edit anything in binary.

[kennyfortune]

Sorry if this is not related to the problem, but have you tried generating TRS file out of the game data? I believe there might be tools around for generating TRS from the game and then compiling TRA which you place in the game dir as a regular file; this way you don't have to edit anything in binary.

Great appreciate i will try

[rofl0r]

@kennyfortune good news, the issue is now fixed in agsutils. the issue was that the game is > 2GB, and we wrote "multi-file-lib" game data file in old version that should theoretically support up to 4GB, but AGS engine reads the 32 bit values as signed integers, so we have only 2GB available.

@ivan-mogilko thanks for your explanations, much appreciated

[ivan-mogilko]

but AGS engine reads the 32 bit values as signed integers, so we have only 2GB available.

Maybe it's possible to change this in the engine too, just in case? I recall we already did this change from signed to unsigned interpretation for a few things previously. As offsets are stored as int64_t in memory now, it should be possible to write unsigned int32 there.

EDIT: Ah, of course that won't change anything if they are using original exe.

[rofl0r]

Maybe it's possible to change this in the engine too, just in case?

yeah, that would be great

EDIT: Ah, of course that won't change anything if they are using original exe.

true, but at least games released in the future would work and/or if the utils user copies the acwin.exe stub from a more recent ags release as published on ags.co.uk.

[rofl0r]

@kennyfortune could you confirm it works now ? you might have to extract Strangeland.exe again because the old agsutils version likely corrupted game28.dta (don't forget to make a backup of the .crm you changed and copy it back in place after the extraction)

[kennyfortune]

@kennyfortune could you confirm it works now ? you might have to extract Strangeland.exe again because the old agsutils version likely corrupted game28.dta (don't forget to make a backup of the .crm you changed and copy it back in place after the extraction)

I am very sorry that I just saw your message. It now works, Thanks for your help!

[rofl0r]

@ivan-mogilko: unrelated to this issue; i'm now completely banned from ags org. are you aware of that, and do you have anything to tell me ?