Cannot use Kernel Debugger together with VMWare video driver

[volo-droid]

After entering the Kernel Debugger (either manually or after a kernel panic) the screen freezes, if Haiku is using VMWare video driver. Blindly typing exit on the keyboard (followed by the Enter keypress) brings us back to the OS, and everything works as before. Is it an expected behaviour, and if so, is there any workaround to it?

[diversys]

Yes, it's a dup of #9.

[volo-droid]

@diversys Thanks for the info. Could you actually reopen https://dev.haiku-os.org/ticket/994 ? As the bug is still there, so it's weird seeing the issue closed with the resolution invalid. If it cannot be solved before the R1 that's fine, but IMHO then it should be kept open and assigned to either the R1.1 milestone or "Milestone Unscheduled".

[diversys]

This is a bug in vmware graphics driver which is not part of Haiku. That's why it was moved to #9 in the first place.

[volo-droid]

I see your point. But is it possible to fix the issue in the VmWare driver itself without modifying Haiku's KDL internals? There were a few suggestions in the ticket, which led me to think it could be fixed only by altering kernel/debug/… parts:

No, KDL runs with interrupts off, and that also includes timer interrupts. Is it possible to let VMware update the screen automatically again somehow (like in VESA mode)? Another possibility would be to alter kernel/debug/frame_buffer_console.cpp to tell VMware when to update the screen. A cleaner solution to this would be to have a special frame buffer module and have kernel/debug/blue_screen.cpp use that one instead, when the VMware driver is running.

[diversys]

Maybe but vmware_addons are in maintenance mode. If you know how to fix it please do so and I'd be happy to merge your fix :)

[volo-droid]

I don't think I know C++/Haiku well enough at the moment to fix the video driver. Nevertheless, I was mainly interested in KDL in context of fixing vmware_fs for x86-64, but I'm done with that already, and have it working on my machine. I'll create a PR soon 😉

[diversys]

Check this out https://github.com/s40in/VMwareAddons/commit/693803cb4fa6151e9f8353a6f797f8eba224a2f2.

[volo-droid]

Nah, those changes will fix the build process, but the resulting driver will be unusable. Haiku drivers must not depend on libstdc++. I made a similar mistake yesterday 😄 I'll publish the solution I've come too asap, just let me cleanup my code a little bit.

[volo-droid]

@diversys here you are https://github.com/HaikuArchives/VMwareAddons/pull/33

[diversys]

Cool, thanks! :)