temisu / ancient

Decompression routines for ancient formats
BSD 2-Clause "Simplified" License
208 stars 14 forks source link

Consider implementing some self-extracting executables #65

Closed neuromancer closed 5 months ago

neuromancer commented 5 months ago

Hi!,

I'm working for the ScummVM project to re-implement the freescape engine. This engine was used to produce a variety of classic videos games such as Driller, Total Eclipse and Castle Master in different computers from the 80s, including AtariST and Amiga.

There are many cases of compressed executable for the AtariST/Amiga releases of the freescape games. For instance, there is a self-extracting AtariST Driller demo that we cannot run since it seems to be packed with some unknown algorithm. Could you please consider implementing some code to perform decompressions of self-extracting executables like this one or at least, taking a look and point us into the right direction on how to do it.

Thanks!

temisu commented 5 months ago

Hi,

The compression in that file is Jam-packer (like the hexdump implies.) The Jam Packer 4.0 has 4 different compression schemes and it looks like it is using the LZH-variant. The other variants include Pack-Ice which itself has multiple variants.

I've been planning to add Pack-Ice and Bytekiller variants for some time, and then implementing Jam Packer will become a real possibility. These are formats that I generally care about.

However, 2 things that I have not been implementing in ancient-library are archive-support and executable packers.

The reason for not implementing archive-support is mostly about time on my part - it would require a magnitude more effort and there are a lot of archival tools, some of which support ancient formats as well (unar, lhasa, deark etc.) Maybe I will change my mind someday if it is useful to someone, but definitely not in near future.

And then there are the executable packers. The reason for not supporting these boils down to 2 reasons:

  1. They are lossy formats. They are made for a specific system, they contain sometimes executables relocated to fixed address, combine sections etc. Putting that toothpaste back to the tube is really hard. If you can't create a working uncompressed executable out of that the process is a lot less useful.
  2. There are incredibly lot of variants to identify them properly. Any tool trying to implement any reasonable subset of them is fighting a losing battle.

Now, onto your specific case. I understand you do not necessarily need a working executable if you are planning to use the file in scummvm, thus this project is more of a digital archeology than tooling.

The contents of that file can be extracted by stripping away the decompressor and starting the file with part that starts with "LZH" then running the JAM Packer (http://umich.edu/~archive/atari/Archivers/jampack4.lzh) in Hatari and getting the file out of it. Whether it is usable for scummvm is a question but definitely it is not executable anymore.

If that does not work, I can try to help but that's not really in scope for this project. Just email me "tz at iki dot fi" and we can continue discussion offline