search

C7-Game / Prototype

An early-stage, open-source 4X strategy game
https://c7-game.github.io/
MIT License
34 stars 9 forks source link

02 MP Fall of Rome.biq Not Openable When Decompressed in C7 #121

Open QuintillusCFC opened 2 years ago

QuintillusCFC commented 2 years ago

Some Civ3 scenarios aren't openable, for example 3 MP Fall of Rome.biq. At first I suspected variations between version 12.06 and 12.08 of the BIQ format, as that scenario is version 12.06, however, this appears not to be the case. I decompressed it with a Java decompression program, and C7 could read the decompressed version, strongly suggesting the problem is in the decompression program we are using, or perhaps how we are using it. This scenario fails with the following message:

System.Net.Sockets.SocketException (0x80004005): An existing connection was forcibly closed by the remote host
   at Mono.Debugger.Soft.TcpConnection.TransportReceive(Byte[] buf, Int32 buf_offset, Int32 len)
   at Mono.Debugger.Soft.Connection.Receive(Byte[] buf, Int32 buf_offset, Int32 len)
   at Mono.Debugger.Soft.Connection.ReadPacket()
   at Mono.Debugger.Soft.Connection.ReceivePacket()
   at Mono.Debugger.Soft.Connection.receiver_thread_main()

It almost consistently fails at the following line in the unit section:

Unit = new UNIT[count];

Count is 211, which seems normal.

The weird thing is that in the cases where it doesn't fail when I step the code forward from that line, Godot crashes while it is paused at that line, with the same SocketException. I'm not sure why yet, and it's really strange for a program to crash while it's paused at a breakpoint. But something is going on.

Our decompression library's source is at https://github.com/jamestelfer/Blast . It does mention that it is probably the least elegant way to implement a decompressor, and its summary says it could use polish, so perhaps this scenario hits an edge case. I'm wondering if there isn't a resource that's getting closed under our feet somehow.

The decompression format (PKWare Data Compression Library [DCL], not to be confused with PKWARE ZIP) is documented in this Newsgroup posting (HTTP portal link, should we need to dive into the details. Mark Adler's original C version is at https://github.com/madler/zlib/tree/master/contrib/blast .

(I'm pretty sure this can affect SAV files as well, but have mainly been testing with BIQ files as I have a wider variety of those)

QuintillusCFC commented 2 years ago

I also sometimes get this with 02 MP Rise of Rome.biq, but haven't been able to pin down another one, although some do fail to open for other reasons, without crashing Godot (I think those include BIQs without a custom map). It appears that I am able (more likely to be able?) to open 02 MP Rise of Rome.biq if I open another file first, but that is not the case with 03 MP Fall of Rome.biq; it always crashes. When Rise does crash, it crashes at the same place in the load that Fall does.

So maybe there's a race condition too, or a data-not-initialized problem.

Overall, not a high priority, but something I want to keep on the radar as my sample size is still small and something that only works some of the time can be frustrating. If it's only these two files, that's one thing, but it might be a lot of other ones too, that just aren't in my default Civ3 install directory (I have several Civ3 install directories).

QuintillusCFC commented 2 years ago

The plot thickens. I added a line to Util.cs so that when it decompresses a BIQ, it saves it to a local file. This let me grab the decompressed-by-blast version, and compare it with the decompressed-by-Java version that C7 happily opens in the hex editor. They're identical, byte-for-byte. And sure enough, C7 opens the one I intercepted from its own decompressor without issue.

I also found that after opening the decompressed one, it opened the compressed one once, but then crashed... but this time got farther, printing out:

RelativeModPath

Process finished with exit code -1,073,741,819.

Though this time it didn't print anything about a socket error.

Something is definitely fishy, and I'm still suspecting there's something to do with resources being freed while they can still be referenced. But I'm not sure what, yet. Also wondering if Godot can generate crash logs... seeing the underlying C error might give a hint as to what's going on.