Betrayal at Krondor

[JorisVanEijden]

Betrayal at Krondor (GOG CD version 1.02) does not work yet.

It works in dosbox and dosbox-x. In Aeon it works until the 1st "book" with the introduction text.

This is not really a bug report, more a log/discussion about getting it to work in Spice86.

What I've done so far:

• Implemented instructions: SHLD, MOVSX, MOVZX, LEAVE
• Stubbed ShowMouseCursor and HideMouseCursor in MouseInt33Handler
• Stubbed CharacterGenerator in VideoBiosInt10Handler
• Added DosInt2fHandler with stubbed SendDeviceDriverRequest

I will create Pull requests for these.

Now I've dealt with the first seven crashes. The program now terminates with the message: "You do not have enough memory to run Betrayal at Krondor. Try using the Make Boot Disk option from INSTALL."

This is because the largest free memory block that "dos" reports (275.504 bytes) is less than the 600.000 bytes the program checks for.

Am I crazy trying to get this particular game to run in Spice86? Is anyone interested in working on this game as well?

To be continued..

[maximilien-noal]

The game is from 1993 and requires a 386. So it probably requires protected mode.

The CD version would require to implement MSCDEX support, bin/cue image mounting and reading, and CDDA playback.

Also does it require EMS or XMS ? As it is unimplemented. (except in the more/graphics_modes branch, but that branch is rather stale atm).

While all of this is certainly possible, it's a lot of work. Not crazy, just large.

Also, the game seems to have a 3d view. This may be not as fast as in dosbox because the cpu is an interpreter. There is no JIT and that's on purpose. But performance problems need profiler data to be resolved first. We'll see.

Protected mode (if the game uses it) would also probably require a lot of work.

We talked a lot about supporting it or not with @kevinferrare, and the consensus is that on the emulation side of things (devices, cpu, dos,...) it's doable. Complicated, but doable.

A good first step would be to ensure that 32 bit CPU instructions are correct. There is a set of tests from qemu that 'only' need to be translated to C#.

I ran into the issue about the dos memory manager. This class only needs some work in order to fix that.

Edit: previously here I was saying that Protected Mode was making the override of ASM code with C# code impossible. Forget about that. @kevinferrare told me a memory extender is not a problem. :)

Overall, it's a lot of work. But very possible. As you can see we've been focused on Dune. Making more games work is very welcome ! :)

[JorisVanEijden]

Fortunately it does not require protected mode (https://www.vogons.org/viewtopic.php?p=75987#p75987)

It also installs to hdd and runs fine without the CD (only required for optional CDDA) so other than a stub telling it there is no CD drive present, no CD support is needed.

It does require EMS or XMS though, cause it needs at least 2mb ram. Which is the point I'm at now.

It also requires a bit more substantial implementation of a vga card. It runs in display mode 0x13, but it uses a lot of fiddling with the bit masks for writing to the 4 "planes" of video memory.

[maximilien-noal]

Fortunately it does not require protected mode (https://www.vogons.org/viewtopic.php?p=75987#p75987)

Excellent !

It also installs to hdd and runs fine without the CD (only required for optional CDDA) so other than a stub telling it there is no CD drive present, no CD support is needed.

That's very good.

It does require EMS or XMS though, cause it needs at least 2mb ram. Which is the point I'm at now.

This is implemented in the more/graphics_mode branch, along with more than 1 MB memory support.

I can't recall if I fixed the dos memory manager algorithm used when finding memory blocs in that branch however. I don't even know if the code is really at fault, or if it just didn't have enough memory.

It also requires a bit more substantial implementation of a vga card. It runs in display mode 0x13, but it uses a lot of fiddling with the bit masks for writing to the 4 "planes" of video memory.

This is implemented in the more/graphics_mode branch.

The code comes from Aeon. If the game bugs out with Aeon, well this branch might have imported that bug. It all depends where the bug comes from. If it's a CPU bug from Aeon, then we're fine. If it's in the xms or ems memory manager, then it's not.

It just needs to be rebased on master, and probably some bug fixes.

I left quite a few TODOs in there. Especially about removing pointers. It works with it (after long hours of debugging) but I'd rather avoid them if it is to be merged in the master branch.

Anyway pointers are a low priority problem as long as they work.

The branch needs missing DOS and CPU functions in order to test more advanced VGA games (like this one). In turn, those same games also demand a more advanced VGA card implementation. Kind of a egg vs chicken problem. In other words, this branch will help.

[maximilien-noal]

Now I've dealt with the first seven crashes. The program now terminates with the message: "You do not have enough memory to run Betrayal at Krondor. Try using the Make Boot Disk option from INSTALL."

This is because the largest free memory block that "dos" reports (275.504 bytes) is less than the 600.000 bytes the program checks for.

The amount of main memory is now configurable, but a XMS/EMS manager is still missing on the master branch.

[maximilien-noal]

The branch more/graphics_modes has been rebased on master.

I took a look at DOSBox Staging's (in case the one from Aeon is bugged) XMS handler. It seems to be fairly straightforward. OTOH, the EMS handler is far more complicated.

@JorisVanEijden Does the game require XMS or EMS ?

If it"s EMS, it's way more involded than XMS to implement.

[JorisVanEijden]

From http://www.sierrahelp.com/Games/KrondorSeries/BaKHelp.html

Betrayal at Krondor's original requirements:

    MS-DOS 5.0 or better, Windows 3.0 MME or higher for CD-ROM version Extras
    386SX minimum
    2x CD-ROM (CD-ROM Version)
    2 MB RAM/1MB available EMS, 4 MB RAM (CD-ROM Version)
    15 MB Hard Drive Space, 3 MB Hard Drive Space (CD-ROM Version)
    VGA
    Recommended: Mouse, 15 MB Hard Drive Space (CD-ROM Version)
    Supports: SoundBlaster, AdLib, MT-32 and General MIDI

Which suggests EMS.

https://arstechnica.com/civis/threads/finally-foolproof-way-to-make-betrayal-at-krondor-work-on-any-2k-xp-machine.594875/ mentions disabling XMS, so that also suggests EMS.

I will take a look at the more/graphics_mode branch. Sounds like it may contain a lot of the VGA stuff I have been implementing in my own branch :)

Since Aeon has a more complete emulation layer, I looked at creating a sort of hybrid of Aeon and Spice86. But their design philosophies are too opposed I think. Aeon's focus is on getting the code as low-level as possible for high performance, while Spice86's focus is on getting the code as high-level as possible for porting to high-level languages. Since my long-term goal is a C# port (and possibly a Unity one) with easy modding support, I feel Spice86 is the better fit. I am very glad you are open and willing to continue Spice86 development in this direction!

Maybe we can divide some of the focus? That one person works on EMS and another on VGA for example? I only have my evenings and part of the weekends to spend on this, but so far I'm enjoying the excercise.

[maximilien-noal]

Indeed, I was thinking of taking the EMS and XMS handlers from more/graphics_modes into a separate branch and PR. :)

Compared to Aeon, the VGA code is also missing some event handlers for when the video mode changed. It means that any video buffer on the UI side needs to be destroyed and re-created.

[maximilien-noal]

EMS is now nearing completion, and will be part of the main branch soon enough.

I took a lot of missing functions from JDOSBox.

@JorisVanEijden: Do you need help on the VGA part ?

I was thinking of adding it bit by bit, instead of trying to have everything at once, like I did in the more/graphics_modes branch. It lead to unmaintainable code. Although, it was a good exercise and example.

[JorisVanEijden]

Thank you. I was not aware that the VGA part in the graphics_modes branch was an experiment. It felt like it was almost ready to merge.

Meanwhile I have mainly been exploring the binary with IDA pro, identifying functions and structures and such.

I guess i"ll have to implement at least some of the code to test the EMS, since krondor does a bit of video capabilites detection before getting to the memory checking part.

[maximilien-noal]

It's still very valuable, and you can modify it and make it mergeable. It was my goal at one point. But that's a huge undertaking.

Firstly, the xms and EMS parts in that branch are outdated and do not work.

More over, more/graphics_modes contains text modes, cga and ega, in other words a lot of cruft unrelated for Krondor.

But also mode X and also more VGA modes code that is highly reusable. Perhaps in a new branch ?

Considering we both lack free time, reducing the scope of PRs is a good idea.

I'm currently reading the code for ems in the 86box emulator in order to understand it more.

Lost Éden requires EMS and already kinda works. Dune can also use it and has just a small bug with the current implementation. :)

[JorisVanEijden]

With EMS the program no longer exists with a "not enough memory" error. Next hurdles:

• Weird white bars show instead of the text "Loading Betrayal at Krondor... please wait."
• The IMUL instruction 66 0F AF is not implemented

[maximilien-noal]

I wish to discuss together the current (as of today) situation on the feature/ems branch.

Dosbox uses [https://wiki.osdev.org/Paging](x86 Paging) when mapping and unmapping an EMS page. And, even Jemm386 uses x86 Paging when mapping EMM pages.

My implementation is so close to being correct. But implementing a MMU is a huge task. I fear it will make things even more complexe... Not only for emulation but also for overrides.

In dosbox itself Paging (not EMM pages, x86 pages) is a ton of code and really makes memory access less readable. Now part of it because dosbox is badly written, but even dosbox staging didn't fix that part of the code.

It also makes segmented addressing obsolete... What does this mean for spice86 user overrides and the ease of writing them ?

I also read that the CPU goes into protected mode when retrieving the EMM page content, and goes back to real mode so DOS is happy.

I do none of that CPU switching. I don't think it's that bad, especially since I don't implement the VCPI parts, but still.

Right now I copy from (unmapping) or to (mapping) the HMA (starting at segment E000) but Dune, will it works better, still has bugs and eventually hangs up.

Maybe instead of copying I should make memory return the EMM page content when it is mapped, and let the original content be when it is unmapped.

If this works it should be close to what an EMS board is doing, before software implementations replaced them (so called 'LIMulators' after the LIM specs).

EMS is confusing. I'm so close yet so far. Also I wish to find LIM 3.2 specs but I can only find LIM 4.0... And it doesn't say anyway how the internals of an EMS board should implement mapping/unmapping (the only crucial part that I'm missing a correct implementation of) only how the public API should behave.

I found a PDF named 'DOS: beyond 640k (2nd édition' ): http://vtda.org/books/Computing/OperatingSystems/DOS_Beyond_640K_2nd_edition.pdf

Maybe it will make this less confusing.

[JorisVanEijden]

A lot of progress hasbeen made here. The video system has improved to a point that the game starts, the intro plays, the main menu works and you can start a new game. The current issues to work on next are:

• [x] - There's a graphical glitch in the 2nd lightning bolt in the intro animation
• [x] - The mouse does nothing. The mouse cursor shows but does not move, clicks are not registerd. (Keyboard navigation works.)
• [x] - The checkboxes/radios in setting menu are invisible. (they work, you just can;t see them)
• [x] - The parchment pages of text have either the top or bottom edge graphic smack in the middle of the page.
• [x] - The UI background graphics are all garbled. The main window is clear, as is the text overlayed.
• [x] - The 3D view shows objects and trees as garbled graphics. Only the terrain works.

[JorisVanEijden]

The latest version of the EMS branch fixes all the graphical issues!

[maximilien-noal]

I long suspected it was coming from ems. Glad to read it !

[maximilien-noal]

EMS has been merged into master via a squash merge. All feedback has been implemented. Thank you to you both @kevinferrare and @JorisVanEijden ! :)

[JorisVanEijden]

The new VGA card is done, except for bugfixing. So I'm going to start on the mouse now.