search

joncampbell123 / dosbox-x

DOSBox-X fork of the DOSBox project
GNU General Public License v2.0
2.57k stars 375 forks source link

Linux KVM (x86 virtualization) support #1089

Open joncampbell123 opened 5 years ago

joncampbell123 commented 5 years ago

Is your feature request related to a problem? Please describe. Add a core to DOSBox-X if possible that runs the guest OS using the virtualization extensions of the processor.

Linux has a KVM virtualization interface for this purpose.

joncampbell123 commented 4 years ago

Here's a good example of a hypervisor, to base Linux KVM support on.

https://github.com/david942j/kvm-kernel-example/blob/master/hypervisor/hypervisor.c

jschwartzenberg commented 4 years ago

DOSEMU2 also supports KVM: https://github.com/stsp/dosemu2/blob/42bc36c8adee504db92e5de9c6db95633c76813d/src/base/emu-i386/kvm.c

It would be very interesting to see this supported in DOSBox-X. It would probably enable very good Windows 9x support. DOSEMU2 runs up to Windows 3.x and QEMU with KVM has difficulty with Windows 9x (was quite unstable when I tried it).

joncampbell123 commented 4 years ago

I want to see KVM support in DOSBox-X too, though I don't know how to program it yet and I don't have the free time to try right now. If implemented it would probably far surpass the dynamic core.

Then for Windows users it would have to support the same through HyperV.

jschwartzenberg commented 4 years ago

Maybe you could ask @stsp or @bartoldeman for info on KVM. There is probably more info to exchange regarding DOSEMU2 and DOSBox-X development :)

AlbertHamik commented 3 years ago

I'm curious if any progress was ever made into figuring this out. I've actually been using DOSEMU2 to speedrun a certain game, 1998's Gunmetal, that is well documented for it's myriad of technical issues but chief among them is it's problems with CPU timing on modern systems. I've had nothing but terrible luck trying to get it to properly run in any version of DOSBox, SVN or not.

After getting a lead from someone else to try out CPU virtualization for VM software, I finally settled on DOSEMU2, which has generally been fine though I'm limited to just my one game with it. Trying to run other games in DOSEMU2 is random chance and I really lucked out with the one game I was desperately needing something like DOSEMU2 for.

It would be amazing if not only did DOSBox-X support KVM, but also whatever comparative options exist in Windows (WHPX, HAXM, etc.). I'm looking for a way of making Gunmetal easier for others to check out, and currently if I can see if they fixed the issue with PS/2 input, QEMU looks to be my current best option for multiplatform support, to make it foolproof for others. But also yeah, CPU virtualization on top of DOSBox-X's accuracy and performance focused feature set would be great, far superior to stuff like QEMU that is very limited, or PCem which is insanely demanding.

jschwartzenberg commented 3 years ago

@AlbertHamik did you report the issues with other games in dosemu2? Reports to the @dosemu2 issue tracker would be useful!

AlbertHamik commented 3 years ago

Well one game I know isn't dosemu2's fault. Will get to that in time. I've been appreciative of the help from those developers over in the dosemu2 community.

Torinde commented 1 year ago

https://github.com/joncampbell123/dosbox-x/issues/1184#issuecomment-513786620 discussion to consider virt86

Torinde commented 1 year ago

@joncampbell123 @StrikerX3 it seems virt86 is exactly what's needed to get hypervisor core in DOSbox-X and it supports most OS supported by DOSbox-X - Windows, macOS, Linux.

What will take to plumb those together? Can you please give some guidance?

StrikerX3 commented 1 year ago

I'm not too familiar with DOSBox-X's codebase, however it should be reasonably straightforward to integrate. Since DOSBox-X doesn't use CMake, you have two options: either use the binary distribution (following the build instructions in the README; the 1.1.0 release is outdated) or add virt86 as a submodule and include the source files manually into your build system.

virt86 is composed of the following modules:

Note that the modules are conditionally compiled based on the target operating system, as described in the modules/CMakeLists.txt file:

virt86 makes use of a few C++17 features such as std::optional and nested namespaces.

For general usage guidance, you can check virt86's Getting Started wiki page and the demo projects in virt86-demos. The basic demo covers nearly every feature in the library.

Torinde commented 1 year ago

Thank you, @StrikerX3 ! Agree it sounds straightforward... although I'll need to learn a lot more before I can embark on that. Hopefully somebody more experienced gives it a try as well...

stsp commented 1 year ago

If implemented it would probably far surpass the dynamic core.

In our experience with dosemu2, this is not always the case.

  1. I haven't found a way to accelerate planar VGA modes with kvm. Likely there is no such way.
  2. Some work-loads https://github.com/dosemu2/dosemu2/discussions/1875 just generate too many vmexits for kvm to stay competitive.
  3. Many AMD chipsets appear to have some "firmware bug" that forces linux to refuse TSC as a system clock. Linux then falls back to the (very slow?) hpet, and KVM for some reason reads system time on every invocation. And in that case things really slow down. I happened to have the notebook with that "firmware bug", so to get the good performance out of KVM I need to reboot with "tsc=reliable" in kernel's command line.

This is JFYI. Of course KVM is still very much worth adding. But its not a silver bullet, very unfortunately.

Torinde commented 1 year ago

@volkertb, I saw your post about the same option - can you implement (part of) the above?

I was actually about to open a feature request here for an optional hypervisor-backed CPU core that would leverage hardware-assisted virtualization (if available) instead of emulating a CPU. After all, on an x86-based machine, how could complete CPU emulation ever be faster than virtualization, with only stufff like VGA and sound cards having to be emulated?

And please see QEMU-3dfx mod and #3405, which are related to the other idea in your post:

(And in the case of VESA mode, even the former could be paravirtualized, leaving basically only the sound card to be emulated in software.)

volkertb commented 1 year ago

@Torinde I assume you're quoting me on a message I posted in a thread on the dosbox-staging project, right?

Anyway, I merely requested such functionality. I currently lack the skills and experience to implement something like that myself, if that was your question. :sweat_smile:

Also, @AlbertHamik and @StrikerX3, I saw you both mentioning HAXM here, but that project has been discontinued by Intel, likely because each still-supported major OS already comes with its own integrated type-2 hypervisor these days. Best to focus on KVM on Linux, HVF on macOS, WHPX on Windows, and maybe bhyve on FreeBSD. (Perhaps also Xen as a type-1 solution, if there is enough added value to support that too?)

I would really love to see an open-source hardware-assisted virtualization solution that was optimized for old x86 games (booter, DOS and early Windows). Basically a combination of VirtualBox and DOSBox. VirtualDOSBox, perhaps? :wink:

The closest we have right now, as others have mentioned here, is dosemu2, but that one doesn't offer sound card emulation that is as good as that in DOSBox and its derivatives. Also, its run-time compatibility isn't as good.

There is the VMusic project by javispedro, which is a VirtualBox extension that adds emulation support for popular DOS-era sound hardware (OPL3 FM, MPU-401 MIDI). However, I noticed some performance issues while running some games under VirtualBox with that extension pack enabled. Nonetheless, it seems fairly promising. But of course, it doesn't address the emulation quality and performance of legacy VGA emulation in VirtualBox.

An actual gaming-optimized hypervisor solution would still be desirable. That is, at least as long as the x86 architecture remains dominant in the market, and modern x86 CPUs retain their ability to natively run 16-bit and 32-bit code, at least within hardware-assisted guest VMs. I agree that in the long run, this will cease to be the case, at which point all of this will no longer be relevant. I guess many developers of DOSBox and derivative projects are already disinterested in hardware-assisted x86 virtualization support for exactly this reason. :man_shrugging:

StrikerX3 commented 1 year ago

I saw you both mentioning HAXM here, but that project has been discontinued by Intel, likely because each still-supported major OS already comes with its own integrated type-2 hypervisor these days. Best to focus on KVM on Linux, HVF on macOS, WHPX on Windows, and maybe bhyve on FreeBSD. (Perhaps also Xen as a type-1 solution, if there is enough added value to support that too?)

Yeah, I'm aware of that. When virt86 was initially developed, HAXM was still relevant and was the main hypervisor platform used by the official Android Emulator. virt86 does support KVM and WHPX as well, but it lacks support for HVF as I don't have a macOS system. Help would be greatly appreciated there. Also, virt86 still lacks support for some important hypervisor functions, such as APIC emulation, IOMMU (direct hardware access) and TSC scaling.

I would really love to see an open-source hardware-assisted virtualization solution that was optimized for old x86 games (booter, DOS and early Windows). Basically a combination of VirtualBox and DOSBox. VirtualDOSBox, perhaps? 😉

I've been thinking of developing a simpler version of that (mostly focusing on simple command-line-based DOS apps) under the virt86-demos project as a demonstration of the library's capabilities, though if there's real interest it would probably make sense to turn that into a real project. However, I currently lack the free time to manage a project of this scale.

Torinde commented 10 months ago

SoftGPU driver requires high performance in the guest, so may benefit from virt86 core.