/ ! \ data files from v1.08 up to v1.10 are faulty / ! \

[dlfrsilver]

Pyrdacor, i made different tests, and i have found where the problem comes from.

Basically, v1.07 from Thalion webshrine works fine on A500 512 chip +512 fast.

Version v1.08, v1.09, v1.10 just fail on A500 configuration.

I did a simple test : load the program from disk with disk A v1.07, then use disks B to I from the v1.10 set containing your data files modified.

It crash no matter what. So, there are 2 news, one good and one bad.

The good news is that all the modifications you did to fix the game data files are OK, no problem there.

The bad news is that you changed the data files byte structure (at least text data files like xMap_Texts.amb for instance).

How did i noticed that ? Simple :

I use the automatic Ambermoon data files ressourcer that CFOU, my partner in crime programmed for me in 2013 when i did the translation in french of the game.

The data files are crippled, because the ressourcer once i process the files from files 'xMap_Texts.amb' for example, outputs all the texts with the first character and the last character chopped off.

The problem never happen with v1.07 data files.

The game parser is very unforgiving on that matter. I seem to remember that you shorten some bytes in the text data files, but the parser seems to not like it......

What should we do now ?

[a1exh]

That might solve the 060 CPU compatibility but I can't imagine it will solve the A500/68000 issue.

Like you said we have to step-by-step replace all the files in a v1.07 installation until we get it to crash at the start of the game. Once we know which file it is we can look at individual sub-files. Or the LOB decompression like @dlfrsilver suggested.

[a1exh]

Test 1 single text file like 2Map_text.amb

I tried a fresh HDD install of v1.07 and (of course it works).

I replaced just 2Map_texts.amb with the latest version in this repo.

CRASH

First decompress with AmbermoonPack.exe UNPACK 2Map_texts.amb 2Map_texts and then pack as AMBR with AmbermoonPack.exe AMBR 2Map_texts 2Map_texts.amb

I tried this and no more crash.

Put back latest 2Map_texts.amb from here and it CRASHES.

Does that mean it is the LOB compression?

(Amiga configuration A500 1MB Chip RAM + 0.5MB Fast RAM)

[dlfrsilver]

I think you nailed it Alex.

The lob compression is at fault. It s very probable that it is not fully understood.

The lob packer v3.07 is cryptic, I have never been able to repack a file in the format and compression the amiga version can decrunch.

Le mer. 15 sept. 2021 à 15:55, Alex Holland @.***> a écrit :

Test 1 single text file like 2Map_text.amb

I tried a fresh HDD install of v1.07 and (of course it works).

I replaced just 2Map_texts.amb with the latest version in this repo.

CRASH

First decompress with AmbermoonPack.exe UNPACK 2Map_texts.amb 2Map_texts and then pack as AMBR with AmbermoonPack.exe AMBR 2Map_texts 2Map_texts.amb

I tried this and no more crash.

Put back latest 2Map_texts.amb from here https://github.com/Pyrdacor/Ambermoon/blob/master/Disks/Bugfixing/English/Amberfiles/2Map_texts.amb and it CRASHES.

Does that mean it is the LOB compression?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920041227, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYRJAEGWOGMEEPTWV5DUCCQVJANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[a1exh]

It will be something very subtle that @Pyradcor might be able to fix.

Files compressed with his LOB packer seem to work with the version of decompressor in AM2_CPU.

[Pyrdacor]

It would be interesting if this happens with all LOB compressed files.

In the end it could be something like too many encoded matches. Maybe the original LOB compressor just stops after some number of matches. It might also be a bug in the executable. I don't think it's related to the LOB algorithm as it works flawlessly for all files with the A1200. I mean the files are the same for all Amigas so the algorithm also has to be similar.

It's either a bug in AM2_BLIT decompressor or there are too many encoded matches for the amount of RAM / AM2_BLIT decompressor.

If other AMNP files work (even if only one works) it is the latter. Otherwise most likely the former.

[dlfrsilver]

@AlexH : You were right about running the game on A600. I tested my french version that was not crunched with LOB packer.

with v1.10 LOB packed, it crashes.....

The problem is why is it crashing with AM2_BLIT (A500), and not AM2_CPU (A1200) ?

Le mer. 15 sept. 2021 à 16:04, Alex Holland @.***> a écrit :

It will be something very subtle that @Pyradcor might be able to fix.

Files compressed with his LOB packer work with the decompressor in AM2_CPU.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920049124, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYRZXSHLDW7OURVVB33UCCRXNANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[dlfrsilver]

It happens with all LOB compressed files. I did multiple tries, whatever the file changed, it bombs.

I guess the LOB unpack routine in AM2_BLIT differs from AM2_CPU.

The problem is that i don't know where it's located inside the code..... Let me try to find it !

Le mer. 15 sept. 2021 à 16:19, Pyrdacor @.***> a écrit :

It would be interesting if this happens with all LOB compressed files.

In the end it could be something like too many encoded matches. Maybe the original LOB compressor just stops after some number of matches. It might also be a bug in the executable. I don't think it's related to the LOB algorithm as it works flawlessly for all files with the A1200. I mean the files are the same for all Amigas so the algorithm also has to be similar.

It's either a bug in AM2_BLIT decompressor or there are too many encoded matches for the amount of RAM / AM2_BLIT decompressor.

If other AMNP files work (even if only one works) it is the latter. Otherwise most likely the former.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920062953, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYX4D43AZ546JOSOKOLUCCTQNANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[dlfrsilver]

oh god..... the COMPRESSOR routine (unpacker) is quite big :

here is a small part of it :

COMPRESSORTYPE.MSG db 'AMBR' dl lbC02555A db 'AMPC' dl lbC02566C db 'AMNC' dl lbC02583E db 'AMNP' dl lbC025860

lbC02555A MOVEM.L D0-D3/D7/A0/A1/A4,-(SP) TST.L ($12,A5) BNE.B lbC0255A6 TST.B (lbB06C338).L BEQ.B lbC02558A MOVE.L (6,A5),D1 MOVE.L (14,A5),D2 MOVEQ #0,D3 MOVE.L A6,-(SP) MOVE.L (lbB068130).L,A6 JSR (-$42,A6) MOVE.L (SP)+,A6 TST.L D0 BMI.W lbC025628

lbC025628 JSR (lbC025FF4).L

AMNP :

lbC025860 MOVEM.L D0-D3/D7/A0/A1/A4,-(SP) TST.L ($12,A5) BNE.B lbC0258AE TST.B (lbB06C338).L BEQ.B lbC025890 MOVE.L (6,A5),D1 MOVE.L (14,A5),D2 MOVEQ #0,D3 MOVE.L A6,-(SP) MOVE.L (lbB068130).L,A6 JSR (-$42,A6) MOVE.L (SP)+,A6 TST.L D0 BMI.W lbC025A18

lbC0258AE MOVE.W (10,A5),D0 JSR (lbC02611C).L MOVE.L (6,A5),D1 MOVE.L (14,A5),D2 MOVEQ #0,D3 MOVE.L A6,-(SP) MOVE.L (lbB068130).L,A6 JSR (-$42,A6) MOVE.L (SP)+,A6 TST.L D0 BMI.W lbC025A18 LEA (lbB06C32A).L,A4 MOVE.L (6,A5),D1 MOVE.L A4,D2 MOVEQ #8,D3 MOVE.L A6,-(SP) MOVE.L (lbB068130).L,A6 JSR (-$2A,A6) MOVE.L (SP)+,A6 CMP.L #$FFFFFFFF,D0 BEQ.W lbC025A18 CMP.L D0,D3 BEQ.B lbC02590C lbC025900 MOVE.W #$1F4,(lbB06C214).L BRA.W lbC025A1E

lbC025890 MOVEQ #0,D0 MOVE.W (10,A5),D1 MOVE.W (12,A5),D2 JSR (lbC024060).L JSR (lbC024010).L MOVE.B D0,($16,A5) BRA.W lbC025A12

lbC02590C CMP.L #$14C4F42,(A4) ; 014C4F42 = LOB header ! BEQ.W lbC025980 MOVE.L ($12,A5),D0 SUBQ.L #4,D0 MOVE.W (10,A5),D1 MOVE.W (12,A5),D2 JSR (lbC024060).L MOVE.B D0,($16,A5) JSR (lbC024B40).L MOVE.L D0,($18,A5) MOVE.L D0,A0 ADDQ.L #4,A4 MOVE.L (A4),(A0)+ MOVE.L (6,A5),D1 MOVE.L A0,D2 MOVE.L ($12,A5),D3 SUBQ.L #8,D3 MOVE.L A6,-(SP) MOVE.L (lbB068130).L,A6 JSR (-$2A,A6) MOVE.L (SP)+,A6 CMP.L #$FFFFFFFF,D0 BEQ.W lbC025A18 CMP.L D0,D3 BNE.B lbC025900 MOVE.L D7,-(SP) MOVE.W (12,A5),D0 MOVE.L D3,D7 ADDQ.L #4,D7 MOVE.L ($18,A5),A0 JSR (lbC026776).L MOVE.L (SP)+,D7 BRA.W lbC025A08

lbC025980 MOVE.L (4,A4),D0 AND.L #$FFFFFF,D0 MOVE.L D0,D7 ADD.L (lbB06B948).L,D0 MOVE.W (10,A5),D1 MOVE.W (12,A5),D2 JSR (lbC024060).L MOVE.B D0,($16,A5) JSR (lbC024B40).L MOVE.L D0,($18,A5) MOVE.L D0,A0 MOVE.L (A4)+,(A0)+ MOVE.L (A4)+,(A0)+ MOVE.L (6,A5),D1 MOVE.L A0,D2 MOVE.L ($12,A5),D3 SUBQ.L #8,D3 MOVE.L A6,-(SP) MOVE.L (lbB068130).L,A6 JSR (-$2A,A6) MOVE.L (SP)+,A6 CMP.L #$FFFFFFFF,D0 BEQ.W lbC025A18 CMP.L D0,D3 BNE.W lbC025900 MOVE.L D7,-(SP) MOVE.W (12,A5),D0 MOVE.L D3,D7 MOVE.L ($18,A5),A0 ADDQ.L #8,A0 JSR (lbC026776).L MOVE.L (SP)+,D7 SUBQ.L #8,A0 JSR (lbC02613E).L MOVE.B ($16,A5),D0 MOVE.L D7,D1 JSR (lbC023F5C).L

Le mer. 15 sept. 2021 à 16:22, Denis Lechevalier @.***> a écrit :

It happens with all LOB compressed files. I did multiple tries, whatever the file changed, it bombs.

I guess the LOB unpack routine in AM2_BLIT differs from AM2_CPU.

The problem is that i don't know where it's located inside the code..... Let me try to find it !

Le mer. 15 sept. 2021 à 16:19, Pyrdacor @.***> a écrit :

It would be interesting if this happens with all LOB compressed files.

In the end it could be something like too many encoded matches. Maybe the original LOB compressor just stops after some number of matches. It might also be a bug in the executable. I don't think it's related to the LOB algorithm as it works flawlessly for all files with the A1200. I mean the files are the same for all Amigas so the algorithm also has to be similar.

It's either a bug in AM2_BLIT decompressor or there are too many encoded matches for the amount of RAM / AM2_BLIT decompressor.

If other AMNP files work (even if only one works) it is the latter. Otherwise most likely the former.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920062953, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYX4D43AZ546JOSOKOLUCCTQNANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[Pyrdacor]

The decompression has to look into the already decoded data for matched data. Maybe the data is read in some chunks of size x. If we compress something at the edge it might crash as it can not look into the last chunk anymore. Maybe the A1200 uses a larger buffer where those edge cases do not happen or only in fewer cases. For example if the buffer is large enough to hold the complete file this issue wouldn't matter.

I think something like this might be the problem. But then it should not happen for very small files like 1Map_texts.amb. Did you really test to only replace this single file?

[dlfrsilver]

yes, i did multiple tries (the files used before the grandpa), any file changed make the program bomb.

And yes, i have found something nasty..... the program in the unpack routine performs checks on hunk parts located at the end of itself..... This smells very bad !

Le mer. 15 sept. 2021 à 16:37, Pyrdacor @.***> a écrit :

The decompression has to look into the already decoded data for matched data. Maybe the data is read in some chunks of size x. If we compress something at the edge it might crash as it can not look into the last chunk anymore. Maybe the A1200 uses a larger buffer where those edge cases do not happen or only in fewer cases. For example if the buffer is large enough to hold the complete file this issue wouldn't matter.

I think something like this might be the problem. But then it should not happen for very small files like 1Map_texts.amb. Did you really test to only replace this single file?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920078941, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYWC75MM3QIYKX5G2ALUCCVS5ANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[a1exh]

it should not happen for very small files like 1Map_texts.amb. Did you really test to only replace this single file?

Are you are asking me? No. I replaced 2Map_texts.amb that contains the data at the very start of a new game.

I can try JUST 1Map_texts.amb if you give me an example of where in the game this data is used.

[a1exh]

@Pyrdacor when you first created your LOB packer last year and I trialled it we discussed over email about how it didn't produce 1:1 identical files to the originals when packing the same data.

The compression rate is very similar to those of the original compressed files. I use a Trie for compression which should allow almost optimal compression. I don't know what the original LOB compressor used

I wonder if we can work out the changes required to create 1:1 identical compressed files?

[Pyrdacor]

It is used for the Lyramion world maps. At least one text is used at the sign post east of Spannenberg.

[Pyrdacor]

@Pyrdacor when you first created your LOB packer last year and I trialled it we discussed over email about how it didn't produce 1:1 identical files to the originals when packing the same data.

The compression rate is very similar to those of the original compressed files. I use a Trie for compression which should allow almost optimal compression. I don't know what the original LOB compressor used

I wonder if we can work out the changes required to create 1:1 identical compressed files?

This is my plan B. Plan A would be to have a look at the 68k code in ghidra.

I can also try to limit the maximum match length and amount of allowed matches. It depends on the test with 1Map_texts.amb.

[a1exh]

It depends on the test with 1Map_texts.amb

With just 1Map_texts.amb changed it crashes as soon as you leave Grandfather's House. Put back v1.07 1Map_texts.amb and it doesn't crash.

[dlfrsilver]

Pyrdacor, can you ask for help from Kermit?

The problem with original lob packer, is that you have to crunch each file individually, you can set the crunch setting and then process all the files from 2map_texts.amb for example.

Le mer. 15 sept. 2021 à 17:30, Pyrdacor @.***> a écrit :

@Pyrdacor https://github.com/Pyrdacor when you first created your LOB packer last year and I trialled it we discussed over email about how it didn't produce 1:1 identical files to the originals when packing the same data.

The compression rate is very similar to those of the original compressed files. I use a Trie for compression which should allow almost optimal compression. I don't know what the original LOB compressor used

I wonder if we can work out the changes required to create 1:1 identical compressed files?

This is my plan B. Plan A would be to have a look at the 68k code in ghidra.

I can also try to limit the maximum match length and amount of allowed matches. It depends on the test with 1Map_texts.amb.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920125114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYT5FL6PVMEPZOGUQ3TUCC3YPANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[dlfrsilver]

Alex, after all these years, i have found the scheme of compression used in Ambermoon !

It's the scheme 'MSS" (choice F6) in the Lob Packer v3.7 !

Let me compress the files from Pyrdacor, archive 1Map_texts.amb for you to test !

Le mer. 15 sept. 2021 à 17:54, Denis Lechevalier @.***> a écrit :

Pyrdacor, can you ask for help from Kermit?

The problem with original lob packer, is that you have to crunch each file individually, you can set the crunch setting and then process all the files from 2map_texts.amb for example.

Le mer. 15 sept. 2021 à 17:30, Pyrdacor @.***> a écrit :

@Pyrdacor https://github.com/Pyrdacor when you first created your LOB packer last year and I trialled it we discussed over email about how it didn't produce 1:1 identical files to the originals when packing the same data.

The compression rate is very similar to those of the original compressed files. I use a Trie for compression which should allow almost optimal compression. I don't know what the original LOB compressor used

I wonder if we can work out the changes required to create 1:1 identical compressed files?

This is my plan B. Plan A would be to have a look at the 68k code in ghidra.

I can also try to limit the maximum match length and amount of allowed matches. It depends on the test with 1Map_texts.amb.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920125114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYT5FL6PVMEPZOGUQ3TUCC3YPANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[a1exh]

@nicodex mentioned that Ambermoon used "LOB Type 6" on his website in 2018. I didn't know what that meant.

[Pyrdacor]

Pyrdacor, can you ask for help from Kermit? The problem with original lob packer, is that you have to crunch each file individually, you can set the crunch setting and then process all the files from 2map_texts.amb for example. Le mer. 15 sept. 2021 à 17:30, Pyrdacor @.**> a écrit : … @Pyrdacor https://github.com/Pyrdacor when you first created your LOB packer last year and I trialled it we discussed over email about how it didn't produce 1:1 identical files to the originals when packing the same data. The compression rate is very similar* to those of the original compressed files. I use a Trie for compression which should allow almost optimal compression. I don't know what the original LOB compressor used I wonder if we can work out the changes required to create 1:1 identical compressed files? This is my plan B. Plan A would be to have a look at the 68k code in ghidra. I can also try to limit the maximum match length and amount of allowed matches. It depends on the test with 1Map_texts.amb. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYT5FL6PVMEPZOGUQ3TUCC3YPANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

I wasn't at my PC yet. Let me first have a look at it myself tomorrow. But of course @kermitfrog can have a look if he has some time for it.

[Pyrdacor]

@dlfrsilver Is there some setting visible for the scheme? Like "max match length", "chunk/block size" etc?

[dlfrsilver]

Type 6 is MSS compression.

Another discovery, the encoding done by Robert when packing and inserting the subfiles inside the amb files is broken. The metallic tools extract.amb can't extract correctly the files.

Let me try another method, I'll come back with a file.

Le mer. 15 sept. 2021 à 18:32, Pyrdacor @.***> a écrit :

Pyrdacor, can you ask for help from Kermit? The problem with original lob packer, is that you have to crunch each file individually, you can set the crunch setting and then process all the files from 2maptexts.amb for example. Le mer. 15 sept. 2021 à 17:30, Pyrdacor @.***> a écrit : … <#m-3308864336086492026_> @Pyrdacor https://github.com/Pyrdacor https://github.com/Pyrdacor when you first created your LOB packer last year and I trialled it we discussed over email about how it didn't produce 1:1 identical files to the originals when packing the same data. The compression rate is very similar to those of the original compressed files. I use a Trie for compression which should allow almost optimal compression. I don't know what the original LOB compressor used I wonder if we can work out the changes required to create 1:1 identical compressed files? This is my plan B. Plan A would be to have a look at the 68k code in ghidra. I can also try to limit the maximum match length and amount of allowed matches. It depends on the test with 1Map_texts.amb. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment) https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920125114>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYT5FL6PVMEPZOGUQ3TUCC3YPANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

I wasn't at my PC yet. Let me first have a look at it myself tomorrow. But of course @kermitfrog https://github.com/kermitfrog can have a look if he has some time for it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920175527, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYS7U7X2H7YQTZQ73ZDUCDDCBANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[a1exh]

Another discovery, the encoding done by Robert when packing and inserting the subfiles inside the amb files is broken. The metallic tools extract.amb can't extract correctly the files.

They have worked for me using Pyradcor files for a long time. I can't say I've tried the latest files.

[a1exh]

The latest 1Map_texts.amb splits fine using Metallic's "extractamb" tool.

[a1exh]

Interesting that if I emulate a 68000 processor and use the tool XFDdecrunch (An Amiga util capable of LOB depacking) to depack files packed using @Pyrdacor LOB packer it crashes just like AM2_BLT

But with a 68020+ CPU XFDdecrunch works fine.

[dlfrsilver]

Please try again alex this 1Map_texts.amb build from the new v1.10 made by Pyrdacor, correctly extracted, and properly packed.

I can already tell you that the repack through AmbermoonPack was faulty. Lob Packer was unable to decrunch some files.

Le mer. 15 sept. 2021 à 18:50, Alex Holland @.***> a écrit :

Interesting that if I emulate a 68000 processor and use the tool XFDdecrunch (An Amiga util capable of LOB depacking) to depack files packed using @Pyrdacor https://github.com/Pyrdacor LOB packer it crashes just like Ambermoon exe.

But with a 68020+ CPU it works fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920191313, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYXAHIT4CMX6T2TBDO3UCDFD3ANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[dlfrsilver]

@AlexH @Pyrdacor

I have also noticed that once crunched, the content of the subfile has a specific pattern.

example from 1Map_texts.amb

014C4F42 .LOB 0600005400000058FF ...T...X. 00010050204F4E20FF ...P ON . 5448452053544F4E7F000649474E20594F55FF THE STON...IGN YOU. 2043414E20524541FF CAN REA. 443A5E5E20224C41FF D:^^ "LA. 425952494E544820DF4F46022D474E4F4D45BF53010E4D4552412753FF BYRINTH .OF.-GNOME.S..MERA'S. 2049534C414E4422E0200020 ISLAND". .

Those patterns were completely missing from Pyrdacor's files. in fact, the text must be visible partly when crunched. In Pyrdacor's files, the text was completely invisible (= wrong scheme of compression or error somewhere).

I will also have to check if all the data files use the MSS scheme or another one.

Le mer. 15 sept. 2021 à 18:55, Denis Lechevalier @.***> a écrit :

Please try again alex this 1Map_texts.amb build from the new v1.10 made by Pyrdacor, correctly extracted, and properly packed.

I can already tell you that the repack through AmbermoonPack was faulty. Lob Packer was unable to decrunch some files.

Le mer. 15 sept. 2021 à 18:50, Alex Holland @.***> a écrit :

Interesting that if I emulate a 68000 processor and use the tool XFDdecrunch (An Amiga util capable of LOB depacking) to depack files packed using @Pyrdacor https://github.com/Pyrdacor LOB packer it crashes just like Ambermoon exe.

But with a 68020+ CPU it works fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920191313, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYXAHIT4CMX6T2TBDO3UCDFD3ANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[a1exh]

Please try again alex this 1Map_texts.amb build from the new v1.10 made by Pyrdacor, correctly extracted, and properly packed.

???

[dlfrsilver]

Please try again the game with the file 1map_texts.amb i joined with the previous email.

This file has been redone and properly crunched with the LOB packer v3.7 from the version v1.10 made by Pyrdacor.

Please validate : is it still crashing or not ??

Le mer. 15 sept. 2021 à 19:19, Alex Holland @.***> a écrit :

Please try again alex this 1Map_texts.amb build from the new v1.10 made by Pyrdacor, correctly extracted, and properly packed.

???

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920215209, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYVSLO2C4U2EQNEM3ZDUCDIRTANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[dlfrsilver]

Ok, i understand, i'm unable to join the file as zip :(

[dlfrsilver]

Please Alex, use the 1map_texts.amb file below :

https://we.tl/t-ECJdVl9SL6

[dlfrsilver]

Please do the test again, like you did before, and tell me if it crashes.

[Pyrdacor]

@dlfrsilver Do you know what MSS means?

[dlfrsilver]

unfortunately, not. I know the LZSS coding, the huffman coding, the RLE coding, but i don't know what MSS means :(

[nicodex]

crashes will happen on MC68000 if any of the sizes (compressed, raw) are odd. See the Notes section in my wiki https://gitlab.com/ambermoon/research/-/wikis/compression

[dlfrsilver]

@Pyrdacor, considering what i found, i will need us to do the following :

Apart 1map_texts.amb, i will start again from v1.07, and we will then take care of each floppy content.

For convenience, i will need you to send me a zip containing all the subfiles you modified, and you will name the archive the same as the archive i will need to crunch and encode (ex: 2Map_texts.amb.zip for 2Map_texts.amb, etc).

I will send you back the correctly encoded archives one by one for inclusion.

[Pyrdacor]

crashes will happen on MC68000 if any of the sizes (compressed, raw) are odd. See the Notes section in my wiki https://gitlab.com/ambermoon/research/-/wikis/compression

Also found something similar just yet on Wikipedia:

The previous 68000 and 68010 processors could only access word (16-bit) and long word (32-bit) data in memory if it were word-aligned (located at an even address). The 68020 has no alignment restrictions on data access.

[dlfrsilver]

crashes will happen on MC68000 if any of the sizes (compressed, raw) are odd. See the Notes section in my wiki https://gitlab.com/ambermoon/research/-/wikis/compression

Hi Nicodex.

The problem here is that the content i compared with the original LOB encoding doesn't match. I have found errors all the way through by doing compare between files (crunched with LOB, and crunched by AmbermoonPack.exe).

[dlfrsilver]

crashes will happen on MC68000 if any of the sizes (compressed, raw) are odd. See the Notes section in my wiki https://gitlab.com/ambermoon/research/-/wikis/compression

Also found something similar just yet on Wikipedia:

The previous 68000 and 68010 processors could only access word (16-bit) and long word (32-bit) data in memory if it were word-aligned (located at an even address). The 68020 has no alignment restrictions on data access.

I wait for AlexH feedback, i joined the file 1map_texts.amb for him to test.

[Pyrdacor]

crashes will happen on MC68000 if any of the sizes (compressed, raw) are odd. See the Notes section in my wiki https://gitlab.com/ambermoon/research/-/wikis/compression

Hi Nicodex.

The problem here is that the content i compared with the original LOB encoding doesn't match. I have found errors all the way through by doing compare between files (crunched with LOB, and crunched by AmbermoonPack.exe).

That doesn't matter. Compressors can compress differently as long as the result can be decompressed by a decompressor. This is totally fine. The problem is the word alignment.

[dlfrsilver]

here is visually what a correctly crunched LOB packed file look like :

[nicodex]

here is visually what a correctly crunched LOB packed file look like :

Doesn't matter, I developed a compression function that haven't been beaten, yet. This reduces memory, copy, and extraction overhead. For the copy operation it would be even better to longword align the streams (MC6020+), or to not compress some streams at all.

[Pyrdacor]

@dlfrsilver Nico already decoded everything in detail and he is skilled in m68k. So we can count on what he found. Please read his link carefully.

I will adjust my packer to take care of it tomorrow and then we can test again.

@nicodex Could you provide the additional buffer sizes or the offset in the executables? That would be awesome.

[dlfrsilver]

next look at this compare : On the left, the file 1map_texts.amb correctly encoded with InsertAMB after inserting the files crunched with MSS encoding with LOB packer v3.7 on the left, and on the right, the same archive encoded with AmbermoonPack.exe with missing 'LOB' header :

[dlfrsilver]

Doesn't matter, I developed a compression function that haven't been beaten, yet. This reduces memory, copy, and extraction overhead. For the copy operation it would be even better to longword align the streams (MC6020+), or to not compress some streams at all.

The goal is to make the game able to run from floppies, like the original. And basically, your enhancements are fine for a windows PC version, but here i'm writing about the Amiga version running on original hardware.

I know that Ambermoon is very picky on modifications. So i will ensure that the game works like originally intended.

[nicodex]

@nicodex Could you provide the additional buffer sizes or the offset in the executables? That would be awesome.

CmpSpace in this table: https://gitlab.com/ambermoon/research/-/wikis/files#file-ids (0 = not compressible)

[Pyrdacor]

@dlfrsilver Do you even read what I write? The compressed output doesn't have to be equal. Every compressor can work and compress differently and still it can be valid compressed data.

[kermitfrog]

Had to have a look at this thread, as my email folder was flooding..

I don't have much time to help right now (maybe tomorrow; more likely saturday - will join if you haven't solved it by then). The alignment issue seems a good canditate for the cause, but I have one more (less likely) idea: could it be the kickstart version? Maybe the LOB compressor uses some system call from there, that is buggy in some versions?

[nicodex]

The goal is to make the game able to run from floppies, like the original.

With a better compression routine you could even put more data on the floppies...

https://gitlab.com/ambermoon/research/-/blob/master/src/am2crunch.c#L235 (I recommend to also read all the comments in am2crunch.h - cmpSpace-check and longword-alignment is not yet implemented, got hooked by other things)

[dlfrsilver]

true Nico, but this is another subject :)

It goes beyond our need :)

Le mer. 15 sept. 2021 à 20:06, Nico Bendlin @.***> a écrit :

The goal is to make the game able to run from floppies, like the original.

With a better compression routine you could even put more data on the floppies...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Pyrdacor/Ambermoon/issues/15#issuecomment-920257630, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYX5MC7BUKUG6HMKLT3UCDOBPANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[Pyrdacor]

true Nico, but this is another subject :) It goes beyond our need :) Le mer. 15 sept. 2021 à 20:06, Nico Bendlin @.***> a écrit : … The goal is to make the game able to run from floppies, like the original. With a better compression routine you could even put more data on the floppies... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVPMYYX5MC7BUKUG6HMKLT3UCDOBPANCNFSM5D4LBNNA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Please be patient and let me adjust the packer tomorrow. Then we will see if it works, ok?