Wing Commander 4 does not work even with `ems=false` set (as previous versions do)

[DSN9990]

Code of Conduct & Contributing Guidelines

• [X] I agree to follow the code of conduct and the contributing guidelines.

Have you checked that no other similar bug report(s) already exists?

• [X] I have searched and didn't find any similar issues.

What operating system(s) this bug have occurred on?

Windows 8.1 x64

What version(s) of DOSBox-X have this bug?

0.83.23

Describe the bug

Wing Commander 4 doesn't seem to work anymore even with ems=false set in the config file. The game just stucks at a black screen for a few seconds after clicking "Save Settings and Run WC4" and causes DOSBox-X to reboot. Also tried setting allow lmsw to exit protected mode to false, same effect, crashes anyway. I remember exactly that a few months ago (back in late July 2021) the game behaved and worked correctly with ems=false without rebooting.

Expected behavior

Game loading correctly without rebooting DOSBox-X.

Steps to reproduce the behaviour

Set ems=false in config file Run WC4.EXE or INSTALL.BAT from the game's first CD Configure the game, including video, sound and click "Save Settings and Run WC4" Observe the black screen for a few seconds and notice the emulator reboots

Used configuration

dosbox-x.txt

Emulator log

Logging: opened logfile 'wc4.log' successfully. All further logging will go to this file.
DOSBox-X version 0.83.23 (Windows SDL2)
Windows keyboard layout ID is 0x0416
Host keyboard layout is now us (US English)
Mapper keyboard layout is now us (US English)
SDL2 reports desktop display mode 1280 x 1024
The default output for the video system: direct3d
Configured windowposition: 
SDL: Current window pixel format: SDL_PIXELFORMAT_RGB888
SDL: You are running in 24 bpp mode, this will slow down things!
Screen report: Method 'Win98base' (1280.000 x 1024.000 pixels) at (0.000 x 0.000) (-1.000 x -1.000 mm) (-0.039 x -0.039 in) (-1.000 x -1.000 DPI)
ISA BCLK: 8333333.333Hz (25000000/3)
monopal: green, 
Active save slot: 1 
USING AVI+ZMBV
Max 3670016 sz 16384
Final 16384
MIDI:Opened device:win32
Pentium CMPXCHG8B emulation is enabled
 Extracting: Program_Name
 Extracting: Time_Stamp
 Extracting: Save_Remark
VOODOO LFB now at d0000000
Serial1: BASE 3f8h
Serial2: BASE 2f8h
Parallel1: BASE 378h
MPU-401 Registering I/O ports as if IBM PC MPU-401 at base 330h
Screen report: Method 'Win98base' (1280.000 x 1024.000 pixels) at (0.000 x 0.000) (-1.000 x -1.000 mm) (-0.039 x -0.039 in) (-1.000 x -1.000 DPI)
Allocated APM BIOS pm entry point at f000:ce20
Writing code to fce20
Screen report: Method 'Win98base' (1280.000 x 1024.000 pixels) at (0.000 x 0.000) (-1.000 x -1.000 mm) (-0.039 x -0.039 in) (-1.000 x -1.000 DPI)
Windows: IID_ITaskbarList3 is available
ISA Plug & Play BIOS enabled
VGA ROM BIOS init callback
Screen report: Method 'Win98base' (1280.000 x 1024.000 pixels) at (0.000 x 0.000) (-1.000 x -1.000 mm) (-0.039 x -0.039 in) (-1.000 x -1.000 DPI)
pixratio 1.000, dw false, dh false
Aspect ratio: 640 x 480  xToY=1.333 yToX=0.750
Screen report: Method 'Win98base' (1280.000 x 1024.000 pixels) at (0.000 x 0.000) (-1.000 x -1.000 mm) (-0.039 x -0.039 in) (-1.000 x -1.000 DPI)
WARNING: No translation support (to host) for code page 0
DOS keyboard layout loaded with main language code BR for layout br
XMS: 50 handles allocated for use by the DOS environment
COMMAND.COM env size:             720 bytes
COMMAND.COM environment block:    0x0701 sz=0x002d
COMMAND.COM main body (PSP):      0x072f sz=0x009a
COMMAND.COM stack:                0x0749
pixratio 1.350, dw false, dh false
Aspect ratio: 720 x 540  xToY=1.333 yToX=0.750
Screen report: Method 'Win98base' (1280.000 x 1024.000 pixels) at (0.000 x 0.000) (-1.000 x -1.000 mm) (-0.039 x -0.039 in) (-1.000 x -1.000 DPI)
IDE: index 0 slave=0
1736987914 ERROR BIOS:INT15:Unknown call ax=1209
1737004076 ERROR MOUSE:Unhandled videomode 69 on reset
pixratio 1.000, dw false, dh false
Aspect ratio: 640 x 480  xToY=1.333 yToX=0.750
Screen report: Method 'Win98base' (1280.000 x 1024.000 pixels) at (0.000 x 0.000) (-1.000 x -1.000 mm) (-0.039 x -0.039 in) (-1.000 x -1.000 DPI)
1737004185 ERROR MOUSE:Unhandled videomode 69 on reset
1737004236 ERROR MOUSE:Unhandled videomode 69 on reset
1751398584 ERROR MOUSE:Unhandled videomode 69 on reset
1751449451 ERROR MOUSE:Unhandled videomode 69 on reset
1751449547 ERROR MOUSE:Unhandled videomode 69 on reset
1751454832 ERROR MOUSE:Unhandled videomode 69 on reset
1751454919 ERROR MOUSE:Unhandled videomode 69 on reset
2077781726 ERROR BIOS:INT15:Unknown call ax=8080
2077781958 ERROR BIOS:INT16:Unhandled call 86
BIOS INT17: Unhandled call AH=86 DX=   0
Restart by INT 18h requested
Removing UMB block 0xcc00-0xefff
Rebooting the system

Additional context

Trying to start the game for the 2nd time (right after the restart) will just hang DOSBox-X completely with a blinking cursor, displaying a ERROR BIOS:INT15:Unknown call ax=1209 error message on the log file.

[Wengier]

I wonder which previous DOSBox-X release you tried previously? And where did you download WC4?

[maron2000]

I had to restart Dosbox-X after setup, but it worked by disabling EMS.

[DSN9990]

I wonder which previous DOSBox-X release you tried previously? And where did you download WC4?

If I recall correctly, I have used a beta version of 0.83.16 you provided. I downloaded WC4 from here: https://archive.org/details/wing-commander-iv-the-price-of-freedom-usa-v-1.1/

[DSN9990]

I had to restart Dosbox-X after setup, but it worked by disabling EMS.

Yeah, you are right, it indeed works by running WC4.EXE after restarting DOSBox-X once it is set up, but I expected it to work without errors when clicking the "Save Settings and Run WC4" button in the setup program as the previous versions did. So I'm thinking there may be some recent change that's causing these issues, anyway.

[grapeli]

If I recall correctly, I have used a beta version of 0.83.16 you provided.

I checked against the following dosbox-x versions 0.83.[12,13,14,15,16,17,18,19,20,21,22,23]. Additionally, under a few much older ones. In none of them, the game does not start directly from the installer (Save Settings and Run WC4). Maybe you used a specific configuration (there is a key in it).

Is that such a big problem? It starts correctly after selecting an option - "Save Settings and Exit" and typed WC4.EXE[Enter].

After all, probably no one decides to reinstall it each time. You only need to install it once. ems=false is also required under dosbox-SVN.

edit: Under dosbox-SVN it is not required to disable ems.

https://user-images.githubusercontent.com/452325/160190454-e62eae07-e76e-42b2-95fd-83a45c88d87c.mp4

[grapeli]

WC4 runs directly from the installer in dosbox-x d1f2b1c from 2014-07-19. Only with ems=true. With ems=false it restarts dosbox-x.

[rderooy]

Does it work properly if you set cputype=ppro_slow or cputype=pentium_ii?

[DSN9990]

WC4 runs directly from the installer in dosbox-x d1f2b1c from 2014-07-19. Only with ems=true. With ems=false it restarts dosbox-x.

How it is supposed to work with EMS enabled in this commit? So far, it is not expected to work with ems=true, however. Just tried it right now with EMS enabled, it just reboots DOSBox-X as soon as I run WC4.EXE. Can you do some bisecting to figure out which commit broke this?

[grapeli]

The explanation may be simple. The 2014 dosbox-x code did not differ much from -SVN.

Since then, a lot of changes have come (thousands of changes to the code). git bisect in this case can be a very time consuming activity. The result does not always have to be unequivocal.

[DSN9990]

Does it work properly if you set cputype=ppro_slow or cputype=pentium_ii?

Same result, ppro_slow and pentium_ii (without EMS, of course) will not work either, they just trigger the same restart I mentioned above. Since both cputypes yield the same issue, neither pentium_iii will work at all.

[grapeli]

@rderooy WC4 works with pentium, pentium_mmx, ppro_slow, pentium_ii.

The problem is of a different kind. It doesn't run directly (only) from the installer, you have to exit (I posted video). The installer currently restarts dosbox-x with ems=true. Must be running with ems=false (no problem for me).

[DSN9990]

@grapeli Are you sure this kind of issue does not occur on real hardware? If so, can you try it in PCem to see how it behaves? I managed to run it within the install program through 86Box (which is PCem's fork) a few days ago and I got this weird crash:

[grapeli]

@grapeli Are you sure this kind of issue does not occur on real hardware?

I do not have the appropriate equipment to check it anymore. Unless you can include my laptop from 2010 from which I am sending this post.

git bisect 9c6c9a2 bad EMS: EMM386 emulation is based on 2ba114f bad Update VCPI emulation to make use of the I/O permission bitmap. d02eef9 bad remove seemingly nonsensical masking of the task register value 790877c good MS A20 gate fix for virtual 8086 mode. It turns out the A20 gate was turned off b76e978 good EMM debug information 43f3572 good TODO list 73b338c good svn mainline:

I found the culprit. So the first bad commit - d02eef9.

I brought it back in the latest code with a little change (not sure if it is correct). The effect of this is a correctly working WC4 installer with the default ems=true settings, and that the game starts directly from the installer after selecting "Save Settings and Run WC4".

--- src/ints/ems.cpp
+++ src/ints/ems.cpp
@@ -1205,6 +1205,10 @@
                                CPU_SET_CRX(0, new_cr0);
                                CPU_SET_CRX(3, new_cr3);

+                               PhysPt tbaddr=new_gdt_base+(new_tr&0xfff8)+5;
+                               Bitu tb=mem_readb(tbaddr);
+                               mem_writeb(tbaddr, tb&0xfd);
+
                                /* Load tables and initialize segment registers */
                                CPU_LGDT(new_gdt_limit, new_gdt_base);
                                CPU_LIDT(new_idt_limit, new_idt_base);

[DSN9990]

It's worth mentioning that WC3 doesn't crash/trigger reboot when launching it from the setup program (WC3.EXE), unlike WC4:

https://user-images.githubusercontent.com/69558812/160239748-588fe99a-1fe0-4414-82c6-1a3ac2839d15.mp4

I'm quite sure WC3 uses an earlier version of WC4's engine, since the in-game graphics look very similar to WC4 anyway.

[maron2000]

I haven't tested the patch to WC4, but I think it better use uint8_t(unsigned char) instead of Bitu, since compilers may submit warnings.

[grapeli]

I haven't tested the patch to WC4, but I think it better use uint8_t(unsigned char) instead of Bitu, since compilers may submit warnings.

Very likely. I wasn't sure. I am not a programmer.

[DSN9990]

@Wengier Can you give me a compiled DOSBox-X SDL2 build? I really want to check these changes out, just for testing.

[Wengier]

@DSN9990 You did not mention which Windows SDL2 build (MinGW or Visual Studio, 32-bit or 64-bit) you want, but auto-compiled Windows builds (both SDL1 and SDL2) are available from:

32-bit MinGW builds:

https://github.com/joncampbell123/dosbox-x/suites/5819032449/artifacts/194984792

32-bit Visual Studio builds:

https://github.com/joncampbell123/dosbox-x/suites/5819032448/artifacts/194978484

64-bit MinGW builds:

https://github.com/joncampbell123/dosbox-x/suites/5819032450/artifacts/194986212

64-bit Visual Studio builds:

https://github.com/joncampbell123/dosbox-x/suites/5819032453/artifacts/194979117

[DSN9990]

@Wengier Thanks! I've tested SDL2 x64 MinGW dosbox-x and both WC3 and WC4 seem to work fine now (mostly, I guess) with ems=true setting, although, at least in WC3, I got a different but unrelated issue (that is, crackling/stuttering audio in the opening intro even with cycles=max set). Not sure if this is EMS related, actually. So maybe this issue can be closed now?

[Cerrseien]

@DSN9990 Sound crackling is caused by DOSBox-X running too fast. I have set cycles=74000, and the crackling's gone.

[DSN9990]

@DSN9990 Sound crackling is caused by DOSBox-X running too fast. I have set cycles=74000, and the crackling's gone.

cycles=74000 does not fix the sound issue. It still produces crackled audio in the intro. Maybe this is the result of a misconfiguration in either DOSBox-X or the game's sound setup? I don't think this is related to the sound setup, since I've set up the sound card in the game's installer correctly according to DOSBox-X's default SB16 settings.

[DSN9990]

Also to note is that cycles=74000 produces heavy stuttering in the music/sound test of the setup program, while cycles=max produces clearer audio.

[Cerrseien]

@DSN9990 Is it possible that your sound crackling is not like my sound crackling? I've just tested the audio on my WC3 installation, first with 74000 cycles (74000.flac), then with max cycles (max.flac): tracks.zip

[DSN9990]

As I said above this audio problem is highly unrelated to this thread, since it was about WC4 crashing when launching itself from the installer, so I think it is better to open a new issue for discussing this, anyway.

[DSN9990]

Strangely, I have now recorded part of the intro's sound output using DOSBox-X's built-in function and the sound popping disappeared entirely. What the heck? @Cerrseien Please check out my recorded file, you can hear flawlessly the clear audio: wc3_001.zip. The recording is in .WAV format because I used the "Record audio to WAV" menu item (under "Capture").

[DSN9990]

Disabling EMS also does not fix the issue, and there is another one weird issue regarding crackling sound: after the opening intro ends, and I start up at the Flight Deck (the main screen), you can again notice heavy stuttering audio playing in the background much like the sound test, unless I use cycles=max, however if I press Alt-X to display the confirmation prompt (Y,N) to exit to DOS, the audio sounds flawlessly and there are no popping issues. What does this mean anyway???