Pyrdacor / Ambermoon

Resources for the incredible Amiga game Ambermoon
91 stars 8 forks source link

/ ! \ data files from v1.08 up to v1.10 are faulty / ! \ #15

Closed dlfrsilver closed 3 years ago

dlfrsilver commented 3 years ago

Pyrdacor, i made different tests, and i have found where the problem comes from.

Basically, v1.07 from Thalion webshrine works fine on A500 512 chip +512 fast.

Version v1.08, v1.09, v1.10 just fail on A500 configuration.

I did a simple test : load the program from disk with disk A v1.07, then use disks B to I from the v1.10 set containing your data files modified.

It crash no matter what. So, there are 2 news, one good and one bad.

The good news is that all the modifications you did to fix the game data files are OK, no problem there.

The bad news is that you changed the data files byte structure (at least text data files like xMap_Texts.amb for instance).

How did i noticed that ? Simple :

I use the automatic Ambermoon data files ressourcer that CFOU, my partner in crime programmed for me in 2013 when i did the translation in french of the game.

The data files are crippled, because the ressourcer once i process the files from files 'xMap_Texts.amb' for example, outputs all the texts with the first character and the last character chopped off.

The problem never happen with v1.07 data files.

The game parser is very unforgiving on that matter. I seem to remember that you shorten some bytes in the text data files, but the parser seems to not like it......

What should we do now ?

Pyrdacor commented 3 years ago

Did you try the 1.10 version with 1.07 text files? If this is really the problem, it should work then.

Pyrdacor commented 3 years ago

But I wonder if this is really the cause. Emulating with WinUAE works fine. I don't think the text files are treated differently in emulation and real Amiga.

dlfrsilver commented 3 years ago

What i mean is that you remove some bytes you thought were useless, but used by the parser.

As i explained, since you did not made any serious changes to the program code, and from my own discovery, my data file ressourcer should ressource data files from v1.08 to v1.10 without errors (it works with v1.07 data files).

When i ressourced your data files, the first character of a text bank and the last one are always cut off. This means that the data file structure has been changed or modified.

And the proof is in the pudding : I tested v1.10 program with v1.07 text files, and..... it works !

My ressourcer can't be faulty, because the programmer is one of the best on amiga, and he spent quite a bit of time to produce this tool, and I tested extensively the tool because it was my tool helper for translating.

The ressourcer takes decompressed amb extracted files, and then put them back as *.asm files. Then i have also the assembler that will re-assemble the ASM file to data file. My french version works on A500 and A1200 without crashing.

I suggest to stop the patching for the moment. we must ensure that the data files generated are correct. Otherwise we will go towards bigger problems later.

Pyrdacor commented 3 years ago

Maybe it's about word boundaries? Often load routines might read such data as words or dwords for speed. Can you give me an example which texts are crippled? Are there texts that are not crippled and if so do they have a length of a multiple of 2 or 4?

dlfrsilver commented 3 years ago

here is what a ressourced data file looks like with my ressourcer :

BOPT    uo+
BOPT    ue-
BOPT    ua+

OUTPUT  0B6.amb

;Source Created by "Ambermoon Langage File ReSourcer" V1  
;A Tool done by CFou! on June 2013

Start dc.w (BankListEnd-BankList)/2 BankList _B00 dc.w Bank00End-Bank00+1 ;$004D BankListEnd

Bank00 dc.b $20,"VOUS POUVEZ LIRE SUR L'ECRITEAU DE PIERRE:^^ ",$22,"LABYRINTHE DES GNOMES DE L'ILE DE MERA",$22,"",$20,0 Bank00End dc.b $20 END

now compare with the same file, but generated from your versions data files :

BOPT    uo+
BOPT    ue-
BOPT    ua+

OUTPUT  0B6.amb

;Source Created by "Ambermoon Langage File ReSourcer" V1  
;A Tool done by CFou! on June 2013

Start dc.w (BankListEnd-BankList)/2 BankList _B00 dc.w Bank00End-Bank00+1 ;$004D BankListEnd

Bank00 dc.b $20,"N THE STONE SIGN YOU CAN READ:^^ ",$22,"LABYRINTH OF THE GNOMES OF MERA'S ISLAND",$22,",$20,0 Bank00End dc.b $20 END

dlfrsilver commented 3 years ago

It's not about word boundaries. It's about text structures that are broken !

All the xMap_texts.amb file internal data files are crippled, all of them !

Look :

image

each text is starting by the byte '20'. This byte is missing from each data files made by your tools. This is part of the data files structure.

And the ending text banks bytes are missing too : the ending byte is not '00', but '20 00 20 00'

The parser is lost......

Pyrdacor commented 3 years ago

Can you try with the files from here? https://github.com/Pyrdacor/Ambermoon/tree/master/Disks/Bugfixing/English/Amberfiles

I just adjusted the text importer and generated the text files again.

dlfrsilver commented 3 years ago

sorry, exactly the same problem, the missing bytes are still missing.

The error possibly comes from either the LOB compressor or your inserter/extractor tool.

a1exh commented 3 years ago

The original v1.07 xMap_,text.amb sub files regularly contain bytes at the end of them which are not described by the header. It has been proved previously these bytes are not used by the Amiga text parser because they have been removed by patches made by myself and Meynaf.

Can you provide a save game or a step by step description on how to trigger a crash.

Pyrdacor commented 3 years ago

I already fixed it I guess but can't push to github at the moment :/

Pyrdacor commented 3 years ago

Ok now the files are uploaded. @dlfrsilver can you now please test with the files. Sorry for the trouble.

dlfrsilver commented 3 years ago

ok, let's me pick the files and test.

@AlexH : there is no save to provide. Please look either above or on EAB forum what i did exactly.

The version A500 crash with v1.08 - 1.10 data files. the only thing that has really changed is the text data file structure.

The game crash just before displaying the text when you are in front of the the grand father.

dlfrsilver commented 3 years ago

@Pyrdacor : the problem is now fixed.

Next step is to make ADF of v1.10 again to test.

Pyrdacor commented 3 years ago

I won't have much time for the next few days. But then I'll do so.

dlfrsilver commented 3 years ago

Pyrdacor, i need to have all data files reviewed. not just the texts files. It still crash from ADF for the A500 version.

Pyrdacor commented 3 years ago

I never provided ADFs for 1.10. I thought it worked with 1.10 plus text files from 1.07.

dlfrsilver commented 3 years ago

I built a set of adf from your files, v1. 10

And then I replaced the texts files from the latest you made.

Le dim. 12 sept. 2021 à 23:37, Pyrdacor @.***> a écrit :

I never provided ADFs for 1.10. I thought it worked with 1.10 plus text files from 1.07.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-917714942, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYT43I6EFGPFJHX4RMDUBUMTHANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Pyrdacor commented 3 years ago

But it works with the text files from 1.07 you said so why should we review the other files?

dlfrsilver commented 3 years ago

because you fixed only some files (texts ones).

As i said previously Robert, the current french version i have doesn't crash on A500.

I did also a try with v1.10 version under whdload (excellent tool to find if there is any problem on a game).

v1.10 crash whdload with an illegal instruction.

Pyrdacor commented 3 years ago

I guess we have to improve our communication. I thought you tested my 1.10 with only the text files from 1.07 and it worked.

I didn't change other data structures afaik. And I still wonder how data changes can influence crashes on the A500 while everything works on the A1200. I mean they won't have two implementations for different systems. Or is it related to AM2_CPU vs AM2_BLIT? Then maybe we should look for an issue there. I am not sure if something was changed there before my time.

dlfrsilver commented 3 years ago

The a500 version is am2_blit.

The major difference is that original v1.07 data files pass with V1. 10 program. Something is wrong somewhere......

I will ask for help to cfou, my partner.

Le lun. 13 sept. 2021 à 00:03, Pyrdacor @.***> a écrit :

I guess we have to improve our communication. I thought you tested my 1.10 with only the text files from 1.07 and it worked.

I didn't change other data structures afaik. And I still wonder how data changes can influence crashes on the A500 while everything works on the A1200. I meant they won't have two implementations for different systems. Or is it related to AM2_CPU vs AM2_BLIT? Then maybe we should look for an issue there. I am not sure if something was changed there before my time.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-917718284, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYROOIVEBWDKMJLRXGTUBUPTLANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Pyrdacor commented 3 years ago

Can you try to use 1.10, copy only all the text files (including the dictionary file) from 1.07 over?

dlfrsilver commented 3 years ago

Once I finish my toeic training, I will take care of this.

Le lun. 13 sept. 2021 à 09:32, Pyrdacor @.***> a écrit :

Can you try to use 1.10, copy only all the text files (including the dictionary file) from 1.07 over?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-917920036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYQX2UYM5KJSHV6B7NLUBWSIXANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Pyrdacor commented 3 years ago

I think the best approach is something like this:

Take the 1.07 version which works. Then add 1 file after the other from the 1.10 patch (https://github.com/Pyrdacor/Ambermoon/blob/master/Disks/Patches/PyrdacorFixEnglish1.10.lha). This contains every changed file. When it crashes after adding a file, you know that this file causes trouble.

You might want to skip the text files which I fixed in bugfix folder first.

dlfrsilver commented 3 years ago

I'll take care of this once all my planning is done.

Le lun. 13 sept. 2021 à 13:28, Pyrdacor @.***> a écrit :

I think the best approach is something like this:

Take the 1.07 version which works. Then add 1 file after the other from the 1.10 patch ( https://github.com/Pyrdacor/Ambermoon/blob/master/Disks/Patches/PyrdacorFixEnglish1.10.lha). This contains every changed file. When it crashes after adding a file, you know that this file causes trouble.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-918099913, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYQPVQRD7G77ABABQO3UBXN6NANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

dlfrsilver commented 3 years ago

Ok, I have done a simple test :

I took v1.07, and simply replace the file 1Map_texts.amb v1.07 by your version v1.10. This was enough to make the game crash.

a1exh commented 3 years ago

I took v1.07, and simply replace the file 1Map_texts.amb v1.07 by your version v1.10. This was enough to make the game crash.

What was your A500 setup? Are you playing from floppy disk? Hard disk? Kickstart version? RAM?

It crashes in the same place? When you start a new adventure on the first screen when it is displaying the text from Grandfather?

Pyrdacor commented 3 years ago

Ok, I have done a simple test :

I took v1.07, and simply replace the file 1Map_texts.amb v1.07 by your version v1.10. This was enough to make the game crash.

Why you would do this? Those are already fixed. Take the fixed file and progress with the others which I didn't recently fixed.

dlfrsilver commented 3 years ago

Why would i do that ? Simply to test and point out that the game crash.

What i did is simple :

1) copy the file 1Map_texts.amb from v1.10 on disk D (version v1.07) as you request above.

2) load the game with A500 configuration and 2 drives (the game is a nightmare to load with 1 drive).

3) the game will ask you disk D and G alternatively, and then once loading has finished, the game will crash.

dlfrsilver commented 3 years ago

This is precisely what I did.

You asked me to replace one file at a time and check if it works or if it crashes.

The answer is it crashes.....

Le lun. 13 sept. 2021 à 17:14, Pyrdacor @.***> a écrit :

Ok, I have done a simple test :

I took v1.07, and simply replace the file 1Map_texts.amb v1.07 by your version v1.10. This was enough to make the game crash.

Why you would do this? Those are already fixed. Take the fixed file and progress with the others which I didn't recently fixed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-918295791, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYUBYPRMQ4L3LBAPHSTUBYIMPANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Pyrdacor commented 3 years ago

I said before.

You might want to skip the text files which I fixed in bugfix folder first.

So we want to know what other files cause a crash. If you want to test that the text files still cause a crash, then at least test with the bugfixed ones. Everything else makes no sense.

dlfrsilver commented 3 years ago

I just replaced the files in disk D and G, which are the ones loaded before it crashes on the grand father screen.

Same as before, it crashes.

Robert, are we sure that the lob packing is correct, both directions : compressing and decompressing?

I have phoned Cfou, asking for his help. I generated a whdload report file when it crashed and explained him I need his expertise on the amiga program side.

Stay tuned.

Le lun. 13 sept. 2021 à 18:08, Pyrdacor @.***> a écrit :

I said before.

You might want to skip the text files which I fixed in bugfix folder first.

So we want to know what other files cause a crash. Of you want to test that the text files still cause a crash, then at least test with the bugfixed ones. Everything else makes no sense.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-918347528, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYRLIAIZD2BDGRHPEJTUBYOZVANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

a1exh commented 3 years ago

copy the file 1Map_texts.amb from v1.10 on disk D (version v1.07) as you request above.

1Map_texts.amb is not on disk D. It's on disk C.

The game crash just before displaying the text when you are in front of the the grand father.

I don't think 1Map_texts.amb is used at the start of the game so where does it crash when all you are changing is 1Map_texts.amb?

dlfrsilver commented 3 years ago

Robert, I have an idea to corner if the compression is the cause of the problem. Can you build me the texts files without compression? The amiga executables are able to load each data files without compression. Thanks. Envoyé depuis mon mobile Huawei-------- Message original --------Objet : Re: [Pyrdacor/Ambermoon] / ! \ data files from v1.08 up to v1.10 are faulty / ! \ (#15)De : Pyrdacor À : Pyrdacor/Ambermoon Cc : dlfrsilver ,Mention I said before.

You might want to skip the text files which I fixed in bugfix folder first.

So we want to know what other files cause a crash. Of you want to test that the text files still cause a crash, then at least test with the bugfixed ones. Everything else makes no sense.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

dlfrsilver commented 3 years ago

copy the file 1Map_texts.amb from v1.10 on disk D (version v1.07) as you request above.

1Map_texts.amb is not on disk D. It's on disk C.

The game crash just before displaying the text when you are in front of the the grand father.

I don't think 1Map_texts.amb is used at the start of the game so where does it crash when all you are changing is 1Map_texts.amb?

Yes this is correct. Disk D contains the data files with 2 prefix .amb. the other disk used is disk G. I changed the texts files on those, and the game crash.

Pyrdacor commented 3 years ago

This is all a bit messy. We should test this in an organized fashion.

If you want to test if it is the compression then take a file which is not a text file. Because this way you have two potentional crash reasons. Take a map data file etc.

I don't think the LOB compression is the problem. It works fine on A1200.

So please let's test some use cases to track the crash cause down.

Then we can proceed.

Don't replace multiple files at once and don't test the unfixed text files at first.

dlfrsilver commented 3 years ago

Don't mix the version for A500 and the version for A1200. each works differently.

dlfrsilver commented 3 years ago

This is all a bit messy. We should test this in an organized fashion.

=> this is what i did already. And the issue is always the same.

If you want to test if it is the compression then take a file which is not a text file. Because this way you have two potentional crash reasons. Take a map data file etc.

I don't think the LOB compression is the problem. It works fine on A1200.

=> AM2_BLIT and AM2_CPU do not operate the same way.

So please let's test some use cases to track the crash cause down.

=> it crashes.

=> i followed carefully your instructions, i picked the files you asked me to pick in your bugfixes folder.

What you need to understand Robert, is that i can test any file i want, map_data, map_texts, whatever on disks D,G, and instead of asking for disk I (contains the music file), the screen remains black, the track counter stop on track 8, and it bombs.

We need an Amiga programmer or a whdload coder to help us.

Then we can proceed.

Don't replace multiple files at once and don't test the unfixed text files at first.

Le lun. 13 sept. 2021 à 19:47, Pyrdacor @.***> a écrit :

This is all a bit messy. We should test this in an organized fashion.

If you want to test if it is the compression then take a file which is not a text file. Because this way you have two potentional crash reasons. Take a map data file etc.

I don't think the LOB compression is the problem. It works fine on A1200.

So please let's test some use cases to track the crash cause down.

  • Test 1 single file like 2Map_data.amb. It will be used at the start of a new game. If this does not crash, the LOB is fine.
  • Test 1 single text file like 2Map_text.amb. Take the one from my Bugfixes folder (mentioned above). If it does not work, something is missing for the text parser maybe. If it works you might test with the unfixed text file from 1.10 to get sure.

Then we can proceed.

Don't replace multiple files at once and don't test the unfixed text files at first.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-918428297, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYSZ3NQA6WPNGFGLBDDUBY2LJANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Pyrdacor commented 3 years ago

If you want to test an uncompressed text container you can use the tool AmbermoonPack. First decompress with AmbermoonPack.exe UNPACK 2Map_texts.amb 2Map_texts and then pack as AMBR with AmbermoonPack.exe AMBR 2Map_texts 2Map_texts.amb. I am only at my mobile phone till Thursday so I can't help you there.

a1exh commented 3 years ago

@Pyrdacor you added a new text block to 2Map_texts.amb for your "Anti-cheat". Perhaps this extra texts means it now exceeds the Amiga RAM limit?

The failure is only seen on the A500 1MB config, the absolute minimum RAM configuration.

Maybe not if this really affects versions before 1.10 as this new text-block wasn't added until 1.10?

Pyrdacor commented 3 years ago

I also added a few map events (14 bytes per event). I didn't know that the game was already at the limit for the A500.

But as you said I would expect 1.08 to work then.

Of course we also fixed some texts and added or changed some words there as well.

Pyrdacor commented 3 years ago

Would also be interesting to test a Lyramion world map as it only uses the 1-prefix files which should have only very few texts and not so much changes in general.

One could play with 1.07 until leaving grandpa's house. Then continue from there with 1.10 or 1.08.

@a1exh Can I test this with WinUAE as well or do I need something else to reproduce the crash?

a1exh commented 3 years ago

@dlfrsilver gave just enough information here (in legible form) for me to reproduce it.

WinUAE 4.4.0 Quickstart A500 1.3 ROM, OCS, 512KB Chip + 512 KB Slow RAM (most common) 2 Floppy drives (DF0, DF1) enabled (this affects RAM) ambermoon_english_1.09_adf.zip Start with disk A and B in DF0: and DF1: respectively Change disk A for disk J when prompted Change disk B for disk A when prompted Change disk A for disk G when prompted Create you character Change disk J for disk D when prompted CRASH

a1exh commented 3 years ago

It still crashes if you put emulated floppy disk speed up to 800% (at least you don't have to wait as long)

a1exh commented 3 years ago

Increasing the memory of the emulated A500 it doesn't stop it crashing.

It crashes in an A500 0.5MB+4MB configuration when run from HDD too (so no need to spend time changing floppy disks).

Pyrdacor commented 3 years ago

Alright but it seems to work on A600 with 2MB.

a1exh commented 3 years ago

Alright but it seems to work on A600 with 2MB.

Not for me. (From floppy disk)

dlfrsilver commented 3 years ago

the game asks for disk D. Strange you didn't mentioned it.

of course, Two drives, internal + external. No one would be crazy enough to play the game with 1 drive.

And, to be honest, 1Mb is just too short for such a big game.

The crazy thing is that my french version works on A500 but hard drive installed.

It's very possible that we have a bug inside the A500 executable, that only triggers on A500 and not on A600.

Le mar. 14 sept. 2021 à 12:40, Alex Holland @.***> a écrit :

@dlfrsilver https://github.com/dlfrsilver did not given quite enough information here (in legible form) for me to reproduce it.

WinUAE 4.4.0 Quickstart A500 1.3 ROM, OCS, 512KB Chip + 512 KB Slow RAM (most common) 2 Floppy drives (DF0, DF1) enabled (this affects RAM) ambermoon_english_1.09_adf.zip Start with disk A and B in DF0: and DF1: respectively Change disk A for disk J when prompted Change disk B for disk A when prompted Change disk B for disk G when prompted Create you character Change disk G for disk J when prompted CRASH

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-919031665, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYTGKRKZQILAOGVGS2LUB4RDJANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

a1exh commented 3 years ago

the game asks for disk D. Strange you didn't mentioned it.

I did.

It's` very possible that we have a bug inside the A500 executable, that only triggers on A500 and not on A600.

No. It doesn't work under emulation with A600 (running from FDD) either.

Pyrdacor commented 3 years ago

Hexae just contacted me. He suggested not to use imploder for the executables as there are some issues with 060 CPUs. I don't know if this can be related. Maybe we should try with uncompressed executables?