Lemmings 2: No Memory Available of required type

[jschwartzenberg]

Describe the bug

To Reproduce Run the game.

Attach program/game binaries or provide an URL l2.tar.gz

Attach the log boot.log

A regression? unknown

[stsp]

fdpp? $_dos_up=(off)?

[jschwartzenberg]

Using MS-DOS 6.22 or FreeDOS 1.3 the game instead hangs at start-up. $_dos_up=(off) doesn't make a difference.

[stsp]

Not sure what you mean. I just started the game with default config (fdpp) and it works perfectly here.

[jschwartzenberg]

The version that I uploaded here? ~Maybe I have some settings lying around indeed.~ Nope I'm running with the default settings.

[stsp]

Yes the version you uploaded here. Are you on a latest devel?

[stsp]

Oh interesting! It works only when started with -K.

[stsp]

Or you need lh l2 - then it works.

[stsp]

So really out of memory? Come on. :)

[stsp]

Using MS-DOS 6.22 or FreeDOS 1.3 the game instead hangs at start-up.

Is that freedos built with fat32? I've found a wrong ifdef that breaks lemmings if built w/o fat32, though fdpp was built with fat32 so unaffected.

[stsp]

ms-dos doesn't seem to set the error code on int25. This game only works perfectly with opendos.

[stsp]

The problem is in auto-loadfix. It allocates 64K and then the game OOMs. But w/o auto-loadfix it crashes. So one way to get it running is to unpack the file. Also reducing the loadfix area makes it running. @ecm-pushbx any hopes to reduce the loadfix area? How much does ms-dos's loadfix allocates?

[stsp]

Hmm, we also need to not enable loadfix if hma is disabled.

[stsp]

Can we use loadhigh as loadfix? I think we should do exactly that. @ecm-pushbx what do you think?

[stsp]

Does anyone remember the fix that "fixed" all loadfix problems? What it was and does it still work? Can't find one. :(

[ecm-pushbx]

The loadfix (auto or explicit) shouldn't allocate a full 64 KiB, just enough to fill the first 64 KiB. But there probably should be a way to disable auto-loadfix. Perhaps an environment variable.

[stsp]

Disabling doesnt help, its needed. Can we use loadhigh as loadfix?

[stsp]

OK i replace loadfix with loadhigh. @jschwartzenberg please re-test those loadfix-demanding games.

[ecm-pushbx]

OK i replace loadfix with loadhigh. @jschwartzenberg please re-test those loadfix-demanding games.

I don't understand the sense of this. If you do not manage to find a large enough UMB then this will not do the loadfix part (reserving the first 64 KiB of memory) any longer, so the program can end up loaded wrongly.

[stsp]

Explicit loadfix was not touched, and is still working as before. Auto-loadfix is a bit specific to dosemu2. Under dosemu2 we have the abundant amount of UMBs, and those "packed file corrupt" progs are usually old and small. There are 2 ways of improving from current behavior: we can use the 0x40 strategy instead of 0x80 to force UMB-only. Then if the prog doesn't fit, comcom32 should handle the failure by doing the normal loadfix. Or we can combine loadhigh with loadfix in a hope that UMB will give the prog more free space than loadfix will take. This looks quite simple and I already know that lemmings2 works under such scheme (loadfix+lh).

[stsp]

Note that the "breakage" of auto-loadfix is times, times more probable to happen simply because the packed file is not the main executable. We have many of such failures already. Whereas there are probably no real progs that suffer from that new loadhigh strategy. So I wonder if to worry?

[stsp]

I.e. my point is that most important to not touch the explicit loadfix. Auto-loadfix is just a bonus that breaks a lot on its own anyway. My change most likely didn't add to that breakages, but it fixed the real case and added mem for everyone.

[jschwartzenberg]

OK i replace loadfix with loadhigh. @jschwartzenberg please re-test those loadfix-demanding games.

Would it be possible to have a message shown when this auto-loadfix is applied? It would make it much easier to keep track of when it's applied.

[stsp]

You'll likely miss it anyway as its not visible as soon as the game started.

[jschwartzenberg]

Maybe a sleep could be put in, at least in some debug mode? Then I could go over all the stuff I have and quickly check.

[stsp]

But what does this give? Isn't it enough to see that the game crashed?

[stsp]

Or wrote "packed file is corrupt"

[jschwartzenberg]

I can only test everything at this point as I don't know which programs specifically rely on this fix apart from the ones we tracked here. If I get some feedback on when the autofix is triggered, I could keep track of all the programs that rely on it also for future testing.

[stsp]

comcom32.exe.gz OK, take this command.com

[jschwartzenberg]

Results of testing so far:

ldfixaut (loadfix triggered automatically)
├── airbuc12
├── angel
├── arena
├── av8b
├── battles
├── brix
├── cannon
├── caw2
├── cd_man2
├── chess
├── conan
├── coridor7
├── gob3
└── problem (having issues)
    ├── alone1
    ├── alone2
    ├── bananoid
    ├── batman
    ├── beathous
    ├── beer
    ├── bloodmon
    ├── bumpy
    ├── civ
    └── crow
ldfixman (still needs manual `loadfix`)
├── a10
└── andretti

The games that are triggered by auto loadfix but have issues: alone1.tar.gz alone2.tar.gz bananoid.tar.gz batman.tar.gz beer.tar.gz bloodmon.tar.gz bumpy.tar.gz civ.tar.gz crow.tar.gz I'm quite sure some of these, such as beer are regressions.

Let me know if you're curious about the ones that still need manually specifying loadfix too.

[stsp]

If they are a regressions, please test them with comcom32 prior to lemmings2 fix. Thanks for such a big testing.

[jschwartzenberg]

That would be commit 7435653957ab6130ed54a3749699f0adbbf363ee? You're welcome!!

[stsp]

Yes, that commit.

[jschwartzenberg]

With that commit alone1 - still broken alone2 - works bananoid - works batman - works beer - works bloodmon - works bumpy - still broken civ - works (but seems slow and has audio issues that I remember were not there in the past, unrelated issue it seems) crow - still broken

So it seems quite some games were negatively affected by that change.

[stsp]

So Ecm was right that this regresses.

[stsp]

I applied the different l2 fix. You'd need to update fdpp for it to work.

[stsp]

@bartoldeman @ecm-pushbx a question: how standard is the freedosish value 0x80 in AL when calling dos_exec 4bh? On freedos it means load high, but RBIL doesn't seem to define that flag. If its specific to freedos, I'll change the semantic.

[ecm-pushbx]

It is specific to the FreeDOS kernel yes.

[stsp]

So in other DOSes if you loadhigh some command.com, the UMB will remain linked in? Or command.com itself clears the link?

[stsp]

In fdpp that flag clears the UMB link after loadhigh, rather than to restore to prev value, as in freedos. But I wonder how other DOSes deal with UMB being linked in after loadhigh - that's definitely a problem.

[ecm-pushbx]

If you run lh command.com (not sure that's what you mean) then I think the UMB link is enabled for the entire lifetime of that application unless someone explicitly disables it. If you run lh test.com from the shell, then the shell restores the UMB link state after test.com terminates.

[stsp]

OK, in this case having a flag 0x80 to disable UMB link is a good idea. Not sure why freedos implements it as a full-featured loadhigh, rather than just "disable UMB link", and of course comcom32 disables UMB link on start. So I think we are good at that field.

[jschwartzenberg]

So Ecm was right that this regresses.

Yeah, although as you can see quite a few games still worked with your variant too. Would it make sense to see if the new fix regresses any of those games that worked with your variant?

[stsp]

I think its just a matter of starting the game, so should be fairly quick. :)

[jschwartzenberg]

Yeah most of them work fine. This one seems to have a new issue though (maybe a separate bug?): coridor7.tar.gz

[stsp]

What is it?

[jschwartzenberg]

DOSEMU exits with: ERROR: general protection at 0x7fdde94e3200: 0 Here's the log: boot.log

[stsp]

Its after that last comcom32 change right?

[stsp]

I mean. a regression of that particular change?

[jschwartzenberg]

Yeah, at least I believe it worked when I was testing with the earlier situation.

[stsp]

Should now be fixed. Had to revert some Tom&Jerry fix, for which there seems to be no ticket filled.