maestun / alis

Silmarils game engine re-implementation
MIT License
18 stars 5 forks source link

Ishar 1 Mac Color and Ishar 2-3 Mac version Broken #36

Closed TiCoKH closed 1 week ago

TiCoKH commented 2 weeks ago

I have Mac version of Ishar 1-2-3 and looks like only Ishar 1 BW version works. Ishar 1 Mac Color not recognized but start and black screen, Ishar 2-3 Mac version exits not recognized and '[FATAL] [N/I] cnul: NULL opcode called'.

Probably something with a MAIN script offset, because not identical as Falcon version.

Should be update in ReadMe to Broken.

skruug commented 2 weeks ago

I have all three Mac versions and all of them are working properly. Please re-run games with -d and output redirected to file. Post resulting logs here so we can try to fix it.

Cosmicore commented 2 weeks ago

Hi,

I can confirm that Mac versions of Ishar 1/Monochrome (MO files), Ishar 1-3/Color (FO files) working.

I have just tried them on the latest version 0.1.035. Alis is compiled for Windows. What is your system/version of alis?

TiCoKH commented 2 weeks ago

MacIshar3-WSL-Ubuntu22.04.log MacIshar2-WSL-Ubuntu22.04.log MacIshar1Color-WSL-Ubuntu22.04.log

I upload what I see. By the way I copy game files with BasiliskII shared folder. Is possible these file get corrupted if I copied form Mac filesystem, but under BasiliskII and vMac games working.

Cosmicore commented 2 weeks ago

Use the -d option on the command line to generate full logs, please.

./alis ../data/ishar1-macc/ -d

Now only Warning level messages are logged, but this way every opcode that passed through the virtual machine will be seen. Thank you!

Is possible these file get corrupted if I copied form Mac filesystem, but under BasiliskII and vMac games working.

Yes, it can happen. We will see in the logs if the code diverges from what it should be.

TiCoKH commented 2 weeks ago

Ishar1MacColor-debug-log.zip

I tried to get file from Mac filesytem with MacDump too but result was same, not corrupted by BasiliskII.

What I suspect I have a different version then what you have from Mac Ishar games. Looks like in this version MAIN.FO is total uncompressed. Maybe all other files too. But Alis engine is try to run decode on this files too.

If you interested I can share my version.

Cosmicore commented 2 weeks ago

Our versions are definitely different. Alis unpacks a file only when the file itself says that it is packed. There is no other way to determine this algorithmically, unless we write a special parser that will evaluate it probabilistically (using probabilistic method), by some features that should be in a packed file and vice versa. As far as I know, there is no such function among development requests.

I assume that your main.fo file is unpacked, but the header says that it is packed. As you can see, the header says that the unpacked file should be smaller than it already is in reality. That is, it was unpacked by someone, but not quite right. The script cannot load logo.fo because the script id ( = 0x0040) is incorrect (0x2e46).

Your log:

main.fo [00060b]0005dd: 45: cload ###  0x0040
Loading script file: ../data/ishar1-macc/logo.fo
Loading...

 main.fo ID 00 AT 02be30
*logo.fo ID 2e46 AT 0329f8
Initialized script 'logo.fo' (ID = 0x2e46)
DATA at address 0x329f8 - 0xa7641

main.fo [000616]0005e8: 40: clive ###  0x0040
<...>
main.fo [00537a]00534c: 42: cstop ###
-- CSTOP --
 RUNNING main.fo [00, 01]
main.fo [00063a]00060c: 86: cmouse ###

Mine:

main.fo [00060b]0005dd: 45: cload ###  0x0040
Loading script file: Ishar1\logo.fo
Loading...

 main.fo ID 00 AT 02be30
*logo.fo ID 40 AT 0329f8
Initialized script 'logo.fo' (ID = 0x40)
DATA at address 0x329f8 - 0x3c8b0

main.fo [000616]0005e8: 40: clive ###  0x0040
...

main.fo [00537a]00534c: 42: cstop ### 
-- CSTOP --
 RUNNING logo.fo [00, 01] 
logo.fo [00001e]000000: 0a: cjmp24 ###  0x000010 [jmp dn +0x000010]

And this strange value 0x2e46 seems to be part of the logo.fo filename (i.e. .f) that got in instead of the real script id.

Try to remember if anyone has tried to unpack these files before?

Are you on Discord?

Cosmicore commented 2 weeks ago

Of course, it could be some bug in alis too that's causing it to get this nonsense on your system. It doesn't even matter if it's due to a broken script. It is desirable to understand the cause.

skruug commented 1 week ago

Hi, we would like to solve your issue, to do that we need some more informations. It looks like there are some unexpected values in script header. Could you please open main and logo scripts in hex editor and send us the first 30 bytes?

TiCoKH commented 1 week ago

The first 0x80 bytes just looks like some Mac resource header: 00 07 4D 41 49 4E 2E 46 4F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 20 20 20 20 20 20 20 05 00 00 00 00 00 00 00 00 00 00 00 69 E3 00 00 0A 6E B2 7D A3 95 B2 7D A3 9D 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 81 81 52 F3 00 00

And just after looks like similar then other alis MAIN header. 81 00 6B DE 00 00 00 3C 00 32 00 3D 09 00 00 00 5D C0 00 00 01 F4 82 00 01 1C 82 00 01 16 82 00 02 52 81 82 00 02 03 21 82 00 0C 53 66 00 20 42

I shared all MacDumps in my OneDrive folder I hope you can access with this link: https://1drv.ms/f/c/81380babc51338d0/ErEXexGUOq5Nl8duNUZ6rxwBbRKe9VXCy5PBJlvAIZitAQ?e=OL2Bfp

skruug commented 1 week ago

There are 128 bytes of unexpected data in front of Ishar 1 scripts. Looks like some meta data, maybe macintosh resource fork or something like that? I suspect that its byproduct of used method of dumping those files.

TiCoKH commented 1 week ago

I created a script what remove first 128byte from all files and that case all 3 games works. Looks like Mac file dump is put some extra metadata what caused files don't work on other os.

Cosmicore commented 1 week ago

@TiCoKH Thanks for the information for testing.

Yes, the dump objects contain an additional header at the beginning and an icon (!) at the end, as @skruug has determined.

I see that your Ishar1MacColor-debug-log.txt was running a different data_path. Maybe we can test it too if you just gzip it as it is. Thanks.

Cosmicore commented 1 week ago

Okay, I think I got it. In log Ishar1MacColor-debug-log.txt you were running a folder where main.fo was without that unnecessary 128 byte header, but with an unnecessary icon at the end of the file. Got the same result as in your log.

TiCoKH commented 1 week ago

I found not all files have icon metadata at the end. I needs to do metadata remover python bit smarter to could check icon data. But looks like alis could get over if extra data at file end.

skruug commented 1 week ago

Not a bug in Alis.