search

nanovms / nanos

A kernel designed to run one and only one application in a virtualized environment
https://nanos.org
Apache License 2.0
2.6k stars 137 forks source link

feat(vmm): MPS support when ACPI is not supported/available #1840

Closed rinor closed 1 year ago

rinor commented 1 year ago

Well probably on Firecracker it works (kind of) because afaik Firecracker does not implement/support ACPI, which is used by nanos for power handling and CPU(s) discovery.

Hence nanos, from what I understand, being unable to process MP detection without ACPI support, (couldn't find anything related to MP Floating Pointer Structure/MP Configuration Table parsing), falls back to single CPU by default (no matter the VM config you provide to it)

Firecracker nanos we pass `4 vcpus`, but still nanos starts just `1` ```json "machine-config": { "vcpu_count": 4, "cpu_template": "None", "smt": false, "mem_size_mib": 2000, "track_dirty_pages": true } ``` **nanos** output: ```sh INIT: init_service INIT: physical memory: INIT: [0000000000400000, 000000007ceff000) INIT: parsing cmdline INIT: in init_service_new_stack INIT: init_hwrand INIT: init cpu features INIT: calling kernel_runtime_init ACPI: find_rsdp: could not find valid RSDP ACPI: AcpiInitializeTables returned 5 APIC: walking MADT table... APIC: MADT not found, detecting apic interface... x2APIC: x2APIC detected APIC: using x2APIC interface x2APIC: per cpu init, writing 0xc00 x2APIC: read from reg 0x803 x2APIC: -> read 0x50014 x2APIC: read from reg 0x802 x2APIC: -> read 0x0 x2APIC: apic id 0, apic ver 50014 x2APIC: write to reg 0x80f, val 0x120 x2APIC: write to reg 0x835, val 0x10000 x2APIC: write to reg 0x836, val 0x10000 x2APIC: write to reg 0x837, val 0x21 x2APIC: write to reg 0x832, val 0x40024 INIT: KVM detected warning: ACPI MADT not found, default to 1 processor INIT: init_mxcsr INIT: starting APs INIT: started 1 total processors INIT: hypervisor undetected or HVM platform; registering all PCI drivers... x2APIC: read from reg 0x802 x2APIC: -> read 0x0 x2APIC: read from reg 0x802 x2APIC: -> read 0x0 ACPI: AcpiEnableSubsystem returned 2 x2APIC: write to reg 0x80b, val 0x0 x2APIC: write to reg 0x80b, val 0x0 x2APIC: write to reg 0x80b, val 0x0 x2APIC: write to reg 0x80b, val 0x0 x2APIC: write to reg 0x80b, val 0x0 x2APIC: write to reg 0x80b, val 0x0 ```

Originally posted by @rinor in https://github.com/nanovms/nanos/issues/1838#issuecomment-1510477226

rinor commented 1 year ago

Just to be clear, atm I don't care much about Firecracker support, at least not for production and/or long running services. Have yet to test, but I suspect that also qemu microvm requires this. While I understand that this may not be the target for nanos, I still think it has a valid use case. I.e: we could use this in dev environments onprem or in clouds that support nested virtualization (do,gcp,...) where booting the whole dev infrastructure in milliseconds is a nice dev experience.

francescolavra commented 1 year ago

MP tables are a legacy, x86-only mechanism for reporting CPU information, and you are right, Nanos does not implement MP table detection. We could add MP table support, but even Firecracker is likely going to drop MP table support and transition to ACPI: there is a feature request for this, and a PR has already been opened.

rinor commented 5 months ago

Just for information, some work was done/completed at https://github.com/firecracker-microvm/firecracker/pull/3155 although not yet available on main/release branches. It needs https://github.com/firecracker-microvm/firecracker/pull/4073 or similar before moving to main branch.

edit: this is the correct pr https://github.com/firecracker-microvm/firecracker/pull/4428 and nanos works as expected

  "machine-config": {
    "vcpu_count": 4,
  }
2024-04-24T12:23:48.509652483 [anonymous-instance:main] Running Firecracker v1.8.0-dev
2024-04-24T12:23:48.540906510 [anonymous-instance:main] Artificially kick devices.
2024-04-24T12:23:48.541003839 [anonymous-instance:fc_vcpu 0] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541028852 [anonymous-instance:fc_vcpu 1] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541054060 [anonymous-instance:fc_vcpu 3] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541126165 [anonymous-instance:fc_vcpu 2] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541162264 [anonymous-instance:main] Successfully started microvm that was configured from one single json
en1: assigned 10.0.2.15
en1: assigned FE80::A8FC:FF:FE00:1
francescolavra commented 5 months ago

It seems to me that the PRs you mentioned are unrelated to ACPI. Anyway, I just saw another PR (https://github.com/firecracker-microvm/firecracker/pull/4428) which finally adds ACPI support to Firecracker, and has been merged into the main branch last month.

rinor commented 5 months ago

thanks, somehow I mixed the links (was investigating some other unrelated stuff around pvh), but yeah https://github.com/firecracker-microvm/firecracker/pull/4428 is the one

rinor commented 2 months ago

for reference https://github.com/firecracker-microvm/firecracker/pull/4428 was merged and released at https://github.com/firecracker-microvm/firecracker/releases/tag/v1.8.0