Please add support for nested virtualization

danielticle commented 4 years ago

Hello,

Could you please provide more details about the scenarios you have in mind for this request? Also do you have some use-cases that you may have considered?

Thanks

CySHell commented 4 years ago

Hey,

I'd like to test the HV implementation and extend it with my own modules, and for that I need to test it running a VM that itself is running VMWare workstation.

danielticle commented 4 years ago

From what you mentioned above it seems to me that you want to use napocahv in vmware. in this case vmware vm's must be configured to support nested virtualization in the first place. keep in mind that it might not be a full set of virtualization features implemented by vmware for nested support. If you have a Windows VM (legacy BIOS not EFI) in VMWare Player/Workstation then there are good chances that a successfull build of napocahv will work in that VM

stonedreamforest commented 3 years ago

i tested it in vm16. it's not work. when i reboot show dark screen...

This is my test state on the real machine


Command: queryhv
Napoca is configured!
Napoca is running! Boot mode = 1
Command: protect firefox 4 0
Adding protection for process
WinguestSetProtectedProcess failed with status = UNKNOWN WINGUEST STATUS (0xe1000501)

How to solve this error

The biggest obstacle is the inability to use dual-machine debugging. If i can use windbg and vmware, it will be very good

danielticle commented 3 years ago

@stonedreamforest there are several things we need to sort out:

the behavior you described (with the black screen) indicates that your VM is a UEFI based VM; if so then:
- there is a file on the efi partition: EFI\NapocaHv\config.cfg
- in that file there is the following setting: CfgDebugGopOutput=0
- change that to CfgDebugGopOutput=1
- this will activate early loader logs for efi platform.
- my guess (as I have seen it on earlier versions of vmware players) is that the vmware efi firmware does not support a efi protocol that we use to start-up the AP processors
for the protect command failure:
- process name must contain also the .exe part; in your case: firefox.exe
- for a valid combination of flags look at: https://github.com/hvmi/hvmi/blob/cd4562a909257d7ae053510e4df156aab0c66f44/include/public/intro_types.h#L386

Thanks for your interest. Please let me know if there is anything else I can help you with.

stonedreamforest commented 3 years ago

@danielticle

still black screen

CfgActiveOsLoad=1,
CfgAllowedRetries=3,
CfgBypassHv=0,
CfgDebugBypassOnErrors=1,
CfgDebugConfirmErrors=0,
CfgDebugConfirmHv=0,
CfgDebugConfirmOs=0,
CfgDebugDumpCrashLog=1,
CfgDebugDumpEnvironmentVariables=0,
CfgDebugDumpLastLog=1,
CfgDebugDumpRuntimeTables=0,
CfgDebugDumpVersion=1,
CfgDebugExceptionHandling=0,
CfgDebugGopOutput=1,
CfgDebugHaltOnErrors=0,
CfgDebugSimulateSecondLoad=0,
CfgFeedbackBufferSize=0x800000,
CfgFilesEfiHv="",
CfgFilesEfiHvEnabled=0,
CfgFilesHvLog="EFI\\NapocaHv\\HvLog.bin",
CfgFilesOs="EFI\\Microsoft\\Boot\\BOOTMGFW.EFI",
CfgFilesOsBackup="EFI\\Boot\\bootx64.efi",
CfgUserInterractionAllowKeyboardBypass=1,
CfgUserInterractionBypassMessage="Press ESC to start without system virtualization",
CfgUserInterractionBypassOnScanCode=23,
CfgUserInterractionTimeOutInSeconds=5

still unsuccessful


Command: protect firefox.exe 8 0
Adding protection for process
WinguestSetProtectedProcess failed with status = UNKNOWN WINGUEST STATUS (0xe1000501)
Command:

danielticle commented 3 years ago

@stonedreamforest I will try to reproduce the issue with vmware 16 and see if the problem is the unsupported efi protocol. As a workaround for the vmware could you please use a vm that uses the legacy firmware (not EFI based).

For the second issue (process protection):

in the client app run: "queryintro" command; this will tell us if the introspection engine is enabled and active.
also please attach a serial cable if possible and run the following command: "config enable serial" - see https://github.com/napocahv/napoca#debugging
i need some kind of logs to see why this is happening to you

PS: The introspection engine is provided (opensource) with support for Windows 7 (x86 and x64) ONLY! If you use any other windows version on a physical machine, you will end up with the hypervisor up and running but no introspection activated / enabled. Lets sort this out first (lets see first if this is the issue you have) and then I will redirect you to correct team to guide/help you add support for additional operating systems.

stonedreamforest commented 3 years ago

@danielticle i test it on win10 18362.295(vm and real machine), Maybe this is the reason why I run it failed.

danielticle commented 3 years ago

@stonedreamforest There is a document that describes how one can add support for different OSes. You may find it in the following link: https://hvmi.readthedocs.io/en/latest/chapters/5-os-support-mechanism.html#automatically-adding-support-for-an-os

stonedreamforest commented 3 years ago

@danielticle i still can't run it in win7(vm), but not disaplay black screen

C:\dacia\install>winguest_sample

Welcome to the NAPOCA Hypervisor sample configuration utility
Type 'help' for available commands.
Command: drvconnect
Connecting to driver
Command: queryhv
Napoca is configured!
Napoca is NOT running!
Command:

danielticle commented 3 years ago

@stonedreamforest I am still looking into this issue. Meanwhile could you please attach a COM port to the VM (in Virtual Machine Settings) and use a file as destination for data, configure the hypervizor using: "config enable serial" command as described in debugging section of the readme.md file and provide the log please.

stonedreamforest commented 3 years ago

@danielticle

C:\dacia\install>winguest_sample

Welcome to the NAPOCA Hypervisor sample configuration utility
Type 'help' for available commands.
Command: drvconnect
Connecting to driver
Command: queryhv
Napoca is configured!
Napoca is NOT running!
Command: config enable serial
Configuring
Command: help
The following commands are available:
 * help: Print available commands and detailed descriptions for them. Try 'help
help' for more information.
 * drvinstall: Install the driver.
 * drvuninstall: Uninstall the driver.
 * drvconnect: Connect to the kernel mode component (driver).
 * drvdisconnect: Disconnect from the kernel mode component (driver)
 * setpath: Set paths to various folders required to operate correctly.
 * missingfeatures: Retrieves the missing (necessary for configuration) features
 mask.
 * config: Configure/Deconfigure the hypervisor.
 * queryhv: Check if Napoca HV is active.
 * queryintro: Check if the introspection engine is active.
 * enfeedback: Enable feedback generation on disk and records a callback to disp
lay in the console.
 * feedback: Allow printing the alerts received from the introspection in the ap
plication console.
 * setfailcnt: Configures the boot failsafe mechanism.
 * resetfailcnt: Reset the boot failsafe counter.
 * protect: Add a process to be protected by the Introspection engine.
 * unprotect: Remove a protected process from the Introspection engine.
 * updateflags: Update the introspection flags.
 * exit: Close the application.
Command:

Is that right? I am a bit confused about the debugging part of the readme file.

This is my virtual machine configuration.

.encoding = "GBK"
config.version = "8"
virtualHW.version = "18"
mks.enable3d = "TRUE"
pciBridge0.present = "TRUE"
pciBridge4.present = "TRUE"
pciBridge4.virtualDev = "pcieRootPort"
pciBridge4.functions = "8"
pciBridge5.present = "TRUE"
pciBridge5.virtualDev = "pcieRootPort"
pciBridge5.functions = "8"
pciBridge6.present = "TRUE"
pciBridge6.virtualDev = "pcieRootPort"
pciBridge6.functions = "8"
pciBridge7.present = "TRUE"
pciBridge7.virtualDev = "pcieRootPort"
pciBridge7.functions = "8"
vmci0.present = "TRUE"
hpet0.present = "TRUE"
nvram = "Windows 7 x64 sp1.nvram"
virtualHW.productCompatibility = "hosted"
powerType.powerOff = "soft"
powerType.powerOn = "soft"
powerType.suspend = "soft"
powerType.reset = "soft"
displayName = "Windows 7 x64 sp1"
usb.vbluetooth.startConnected = "TRUE"
guestOS = "windows7-64"
tools.syncTime = "FALSE"
sound.autoDetect = "TRUE"
sound.virtualDev = "hdaudio"
sound.fileName = "-1"
sound.present = "TRUE"
memsize = "4096"
mem.hotadd = "TRUE"
scsi0.virtualDev = "lsisas1068"
scsi0.present = "TRUE"
sata0.present = "TRUE"
scsi0:0.fileName = "Windows 7 x64 sp1.vmdk"
scsi0:0.present = "TRUE"
sata0:1.deviceType = "cdrom-raw"
sata0:1.fileName = "auto detect"
sata0:1.present = "TRUE"
usb.present = "TRUE"
ehci.present = "TRUE"
svga.graphicsMemoryKB = "8388608"
ethernet0.connectionType = "nat"
ethernet0.addressType = "generated"
ethernet0.virtualDev = "e1000"
serial0.fileType = "thinprint"
serial0.fileName = "thinprint"
floppy0.fileName = "autoinst.flp"
ethernet0.present = "TRUE"
serial0.present = "TRUE"
extendedConfigFile = "Windows 7 x64 sp1.vmxf"
numa.autosize.cookie = "10012"
numa.autosize.vcpu.maxPerVirtualNode = "1"
uuid.bios = "56 4d 0f ea 4f e3 be 10-90 88 87 10 1a 93 57 cb"
uuid.location = "56 4d 0f ea 4f e3 be 10-90 88 87 10 1a 93 57 cb"
scsi0:0.redo = ""
pciBridge0.pciSlotNumber = "17"
pciBridge4.pciSlotNumber = "21"
pciBridge5.pciSlotNumber = "22"
pciBridge6.pciSlotNumber = "23"
pciBridge7.pciSlotNumber = "24"
scsi0.pciSlotNumber = "160"
usb.pciSlotNumber = "32"
ethernet0.pciSlotNumber = "33"
sound.pciSlotNumber = "34"
ehci.pciSlotNumber = "35"
vmci0.pciSlotNumber = "36"
sata0.pciSlotNumber = "37"
scsi0.sasWWID = "50 05 05 6a 4f e3 be 10"
svga.vramSize = "268435456"
vmotion.checkpointFBSize = "8388608"
vmotion.checkpointSVGAPrimarySize = "268435456"
vmotion.svga.mobMaxSize = "1073741824"
vmotion.svga.graphicsMemoryKB = "8388608"
vmotion.svga.supports3D = "1"
vmotion.svga.baseCapsLevel = "9"
vmotion.svga.maxPointSize = "1"
vmotion.svga.maxTextureSize = "16384"
vmotion.svga.maxVolumeExtent = "2048"
vmotion.svga.maxTextureAnisotropy = "16"
vmotion.svga.lineStipple = "0"
vmotion.svga.dxMaxConstantBuffers = "14"
vmotion.svga.dxProvokingVertex = "0"
vmotion.svga.sm41 = "1"
vmotion.svga.multisample2x = "1"
vmotion.svga.multisample4x = "1"
vmotion.svga.msFullQuality = "1"
vmotion.svga.logicOps = "1"
vmotion.svga.bc67 = "9"
vmotion.svga.sm5 = "1"
vmotion.svga.multisample8x = "1"
vmotion.svga.logicBlendOps = "1"
ethernet0.generatedAddress = "00:0c:29:93:57:cb"
ethernet0.generatedAddressOffset = "0"
vmci0.id = "445863883"
monitor.phys_bits_used = "45"
cleanShutdown = "FALSE"
softPowerOff = "FALSE"
usb:1.speed = "2"
usb:1.present = "TRUE"
usb:1.deviceType = "hub"
usb:1.port = "1"
usb:1.parent = "-1"
tools.remindInstall = "FALSE"
sata0:1.autodetect = "TRUE"
sata0:1.startConnected = "FALSE"
floppy0.autodetect = "TRUE"
floppy0.startConnected = "FALSE"
toolsInstallManager.lastInstallError = "0"
toolsInstallManager.updateCounter = "3"
floppy0.present = "FALSE"
svga.guestBackedPrimaryAware = "TRUE"
guestOS.detailed.data = ""
vhv.enable = "TRUE"
hypervisor.cpuid.v0 = "FALSE"
mce.enable = "TRUE"
usb:0.present = "TRUE"
usb:0.deviceType = "hid"
usb:0.port = "0"
usb:0.parent = "-1"
unity.wasCapable = "TRUE"

bitdefender / napoca

Please add support for nested virtualization #1