DOS extender issue with System Shock Portable

[offbyone]

Hoo boy, where to begin.

So, I'm running System Shock Portable (http://www.systemshock.org/index.php?topic=211.0) in Boxer 1.3.2 (and I've tried 1.4pre) and periodically it wouldn't start, a situation which got progressively worse -- no idea how it would have changed -- until today, when it wouldn't run at all.

I'm running it on a last-model Macbook Pro 17", OSX 10.8.2

Here's the console.app log for a failed start in 1.3.2:

http://pastebin.com/ZcaFfWhu

[offbyone]

I should note that I believe it to be a DOS extender issue because in 1.4pre I have seen DOX extender errors, but in 1.3.2 the screen just ends up blank, never starting the game at all.

[alinebee]

First question: have you done anything to the structure or configuration of SSP, beyond importing it?

When importing the SSP package, Boxer will glom onto the DOSBox configuration included in the package, which will launch CDSHOCK.EXE. None of SSP's extra stuff (its launcher/mod setup menu etc.) will be used. If you have manually installed any of these mods after importing, or done anything else to the imported gamebox, please let me know.

I haven't been able to replicate the assertion so far. How far does the game get with its startup process before crashing? If the behaviour has been progressively deteriorating, this may indicate that it's to do with savegame data or other file modifications since importing - in which case I'll probably need a copy of your gamebox to replicate it.

[offbyone]

I can arrange that; how would you prefer I provide that?

[alinebee]

Dropbox is the best way: if you have a dropbox account yourself then you can zip the gamebox and send me a public link to it over email, otherwise I can share a folder from my own dropbox for you to put it in.

Incidentally, if you're using Boxer 1.4pre then I just pushed a big improvement to how emulator crashes like this one are handled. Boxer will now present a friendly warning and give you a chance to submit an error report, which opens a page for a new issue here pre-filled with a lot more detail than you get from the Console (and whose contents you can copy-paste into an existing issue instead if you like). This won't catch emulation hangs yet, nor crashes that take the rest of Boxer with them, but it's a start.

[offbyone]

I don't think I need to send the whole gamebox; I've narrowed down the reproduction case by reimporting the SS-Portable game and making a single change to a file (which is just the game settings file):

± diff System\ Shock.boxer.bad/S\ RES.harddisk/CYB.CFG System\ Shock.boxer.new/S\ RES.harddisk/CYB.CFG
1c1,17
< midi_list lowres_cutscenes midi_io -midi_irq -inp6d -digi_list joystick 13midi_card 1midi_dma -midi_drq -cd_data_dir_path .\DATdigi_io -digi_irq -digi_card digi_dma -language digi_drq -1
\ No newline at end of file
---
> midi_list 7
> lowres_cutscenes 0
> midi_io -1
> midi_irq -1
> inp6d -1
> digi_list 0
> joystick 131
> midi_card 12
> midi_dma -1
> midi_drq -1
> cd_data_dir_path .\DATA
> digi_io -1
> digi_irq -1
> digi_card 1
> digi_dma -1
> language 0
> digi_drq -1

The working file has the lower (right) format, the broken game box's CYB.CFG has the single-line list of flags format (the left)

[alinebee]

Oho, that's intriguing - what program was used to write to the CYB.CFG file?

Also, were you getting these hangs in 1.3.2 before you started trying out 1.4pre? (There's the remote possibility the mangling could be done by some bug in 1.4's write shadowing.)

[offbyone]

I'm not sure, to be honest; I think it started with 1.4pre? But I'm not completely sure.

What I do know is that there's gamestate for SS in my Application Support folder, consisting of the DOS extender error file and ... CYB.CFG, which contains this:

± cat "./Gamebox States/9A959C73-1F6E-4E34-8F8A-B7966F3ABFF7/Current.boxerstate/S RES.harddisk/CYB.CFG"
midi_list lowres_cutscenes midi_io -midi_irq -inp6d -digi_list joystick 13midi_card 1midi_dma -midi_drq -cd_data_dir_path .\DATdigi_io -digi_irq -digi_card digi_dma -language digi_drq -%

For what it's worth, the CW.ERR file is:

CauseWay DOS Extender v3.25 Copyright 1992-96 Michael Devore.
All rights reserved.

Exception: 0E, Error code: 0004

EAX=00000000 EBX=0000000A ECX=2D20616D EDX=00725828 ESI=645F6964 
EDI=696D0832 EBP=006EF93C ESP=006EF788 EIP=696D0831 EFL=00003214 

CS=01A7-FFBCA000 DS=01B7-FFBCA000 ES=01B7-FFBCA000 
FS=0000-xxxxxxxx GS=01BF-xxxxxxxx SS=01B7-FFBCA000 

CR0=00000000 CR2=00000000 CR3=00000000 TR=0000

Info flags=00008042 

Program Linear Load Address: 00436000

[offbyone]

My guess is that I tweaked a game setting - possibly sound, possibly something else - and the game wrote that out.

As a test, I changed CYB.CFG in my vanilla System Shock gamebox (not SS-portable, just plan SS) and .. a crash too. Obviously not with CauseWay DOS extender -- SS uses DOS4GW -- but it's still crashing. So that config is the key.

[alinebee]

OK, that's great - this should definitely be enough for me to pinpoint the bug.

In the meantime, could you start with a fresh import of the SSP version and, under 1.3.2 only, try to get the game itself to modify CYB.CFG? e.g. by mucking around in the options menu and/or re-running the installer to change some sound config? If you can confirm that the game has changed the file under 1.3.2, but the linebreaks are left intact, then that points the finger at 1.4.

[offbyone]

I'll do that tomorrow; gotta sleep for work :smile:

[alinebee]

OK, then I'll probably check myself before then once I get home tonight! Thanks for your work narrowing this down, it's possible you've uncovered a serious bug in Boxer's write shadowing.

[alinebee]

OK, so here's my findings:

1. Holy cow this is unstable, straight out of the box both in 1.3.2 and 1.4pre. I'm guessing either the replacement DOS extender or something else they've patched on isn't fully compatible with DOSBox's emulation. I'd strongly recommend sticking with the original CD version of the game instead of using this one; I've never seen crashes and hangs like these when testing the original. This general crashiness is almost certainly out of Boxer's scope, though I'll test it on DOSBox 0.74 later to confirm that it behaves as poorly there.
2. There is indeed a bug with Boxer 1.4's write shadowing. The exact symptom I found was this: whenever the game crashes, it makes a duplicate copy of its own CYB.CFG file, (presumably) modifies the duplicate, deletes the original CYB.CFG, then renames the duplicate back back to CYB.CFG. But on 1.4pre with shadowing enabled, the final rename was never being performed: leaving the game without a configuration file, meaning it would no longer start up the next time. The game isn't even trying to rename the file when shadowing is enabled, and I'm still investigating why.

Incidentally, I haven't been able to replicate the contents of CYB.CFG getting mangled in either Boxer version.

[offbyone]

Wow. That's a hell of a comedy of errors. Okay, I'll stick with the DOS4GW extender version now, and leave the Portable version aside.

For what it's worth, I think the Portable version ships with its own DOSBox embedded; I assume that works :smirk:

[alinebee]

The portable version ships with two Windows builds of DOSBox actually, along with VDMSound and some other workaround utilities for running the game natively in XP. I wouldn't put money on their bundled DOSBox behaving any more stably out of the box though - there's a complex frontend program that SSP uses to enable DOSBox and other patches you select, so it's possible that the game files just aren't in a suitable state until that has been run (and, natch, it's a windows executable that won't run in Boxer).

[offbyone]

Looks like. Want me to close this up, or are you going to track it as the write shadowing issue?

[alinebee]

Closing this now, having just tracked down and fixed the write shadowing bug. It turned out that Boxer's write shadowing was making the DOS drive report that the deletion of a file had failed even though it hadn't, and this was causing the game to skip the final move step when it was replacing its own config file after crashing.

That aside, I tested the SSP gamebox in vanilla DOSBox 0.74 and it produces the same unreliable behaviour as in Boxer: so, not a Boxer bug. (I also got the mangled config file on one occasion, but again this seems to be something that game is doing when it crashes, and not a Boxer bug.)