search

joncampbell123 / dosbox-x

DOSBox-X fork of the DOSBox project
GNU General Public License v2.0
2.53k stars 369 forks source link

New SFT code causes Windows 3.1 to crash on exit with Invalid Opcode exception (executing random code in HMA). #5030

Closed joncampbell123 closed 3 weeks ago

joncampbell123 commented 3 weeks ago

Describe the bug

A recent change to better emulate the SFT table was pulled in that later turned out to cause Windows 3.1 to crash on exit. Restoring the old code fixes the crash.

Steps to reproduce the behaviour

  1. Change the #if 1 in src/dos/dos_classes.cpp to a #if 0, so it executes the newer SFT code. Compile.
  2. Start Windows 3.1 in 386 enhanced mode (which it should be default)
  3. Exit Windows 3.1. Observe the log file fill with unhandled "invalid opcode" exception messages
  4. Change the #if 0 back to an #if 1. Compile.
  5. Start Windows 3.1, then exit. Observe no crash.

Expected behavior

No response

What operating system(s) this bug have occurred on?

Linux x86_64

What version(s) of DOSBox-X have this bug?

git merge request May 29th, "allocate enough memory to store the System File Table"

Used configuration

No response

Output log

No response

Additional information

No response

Have you checked that no similar bug report(s) exist?

Code of Conduct & Contributing Guidelines

maron2000 commented 3 weeks ago

Sorry for the regressions.

Related threads https://github.com/dosbox-staging/dosbox-staging/issues/3694 Issue #5004

joncampbell123 commented 3 weeks ago

I just tried an experiment with the old code, and also examined how real MS-DOS stores the SFT.

I've noticed that the pointer to the first SFT table is always in the same segment as the "info block" aka "list of lists" and Windows 3.1 crashes like this on shutdown if the segments don't match. The SFTs next in the linked list don't have to be the same segment and they often aren't. In fact, if you examine the SFTs in a Windows 95 "dos box" the last link of the SFT chain is in the UMB area!

The first SFT in real MS-DOS is always 5 entries large and apparently contains SFTs for CON, AUX, PRN, etc. Obviously if it were any larger it wouldn't fit within the 0x200 (0x20 paragraphs) allotted to INFOBLOCK_SEG.

joncampbell123 commented 3 weeks ago

Another note: Even though DOSBox and DOSBox-X write 0xFFFF:0xFFFF for the end of the SFT chain, real MS-DOS appears to write 0x0006:0xFFFF. According to the Ralf Brown Interrupt List this is valid because only the offset field needs to be 0xFFFF to end the chain. http://www.ctyme.com/intr/rb-2983.htm

joncampbell123 commented 3 weeks ago

Uh... even if I arrange the tables normally, Windows 3.1 doesn't like it.

The magic offsets in the old code seem to work because they line up the second SFT in a way that Windows 3.1 can see "CON" line up with what the RBIL documents as the FCB-like copy of the name in the SFT. If I zero that out, then suddenly Windows 3.1 decides that it doesn't like the MS-DOS version.

Clearly writing more or less SFT is not as straightforward, but anyone who's dabbled in MS-DOS and Windows 3.1 should know Microsoft is long infamous for magic hacky values layouts and seemingly nonsensical requirements like that that break mysteriously if anything doesn't match some strange criteria.

joncampbell123 commented 3 weeks ago

If real MS-DOS serves as an example then perhaps your SFT code might work best with Windows 3.1 if it were to write dummy SFT entries for CON, AUX, PRN, etc. like those files are open somewhere.

grapeli commented 3 weeks ago

Maybe such a big change is simply unnecessary. Dunkle Schatten 2 works fine up to version dosbox-x-v2023.03.31 freezes in dosbox-x-v2023.05.01. Unfortunately, I don't currently have time to check exactly which commit changed this behavior.

edit: Freezes since this change f11408c5a4 that fixed this issue #3626.

joncampbell123 commented 3 weeks ago

That would suggest then that if you were to change the "files=" and "fcbs=" settings in dosbox.conf to smaller values that the game might work?

I'm noticing that the change to 0x20 paragraphs puts the other half of the SFT in the empty space of the CON driver and whatever follows. That's not good, because if someday we or some program touches the SFTs it could corrupt other unrelated tables.

I'm going to continue testing how I can get it to match what I observe MS-DOS doing without causing Windows 3.1 to crash.

joncampbell123 commented 3 weeks ago

Huh... if the SFTs in MS-DOS 5.0 are any indication, the default SFT limit is 8 file handles!?

I see only the main SFT for 5 entries and the second SFT for 3 entries.

If I add FILES=32 to CONFIG.SYS and reboot, the first has 5 and the second has 27.

I can see why Windows 3.1 is so interested in the SFT, it literally needs a larger SFT just to run! I doubt Windows 3.1 could run with only 8 file handles!

joncampbell123 commented 3 weeks ago

I got it. Windows 3.1 requires the string "CON" to appear three times in the SFT table. It doesn't care exactly where, it just expects it to appear 3 times. I was able to fix the code to put the fixed strings in the first SFT and dynamically allocate the second SFT.

More importantly, the previous code that let SFT tables overlap things like the FCB SFT and the CON driver has been fixed not to do that anymore.

Please try the latest commit with any games you believe had issues due to SFT tables.

maron2000 commented 3 weeks ago

Dunkle Schatten 2 mentioned in #5004 is working without freezing.

dunkle