Only 1 CPU booted in bhyve

[alikhanz]

Hello! I'm installed ARPL in TrueNAS VM (bhyve hypervisor), all works except for expanding the number of cores. When i increase number of cores bootloader booting only 1 CPU. That i see in serial logs:

[    0.130036] smpboot: CPU0: Intel(R) Xeon(R) E-2224G CPU @ 3.50GHz (family: 0x6, model: 0x9e, stepping: 0xa)
[    0.131331] Performance Events: unsupported p6 CPU model 158 no PMU driver, software events only.
[    0.132389] NMI watchdog: disabled (cpu0): hardware events not enabled
[    0.133082] NMI watchdog: Shutting down hard lockup detector on all cpus
[    0.133824] x86: Booting SMP configuration:
[    0.134271] .... node  #0, CPUs:      #1
[   10.134576] smpboot: do_boot_cpu failed(-1) to wakeup CPU#1
[   10.135561]  #2
[   20.136013] smpboot: do_boot_cpu failed(-1) to wakeup CPU#2
[   20.136909]  #3
[   30.137459] smpboot: do_boot_cpu failed(-1) to wakeup CPU#3
[   30.138275] x86: Booted up 1 node, 1 CPUs
[   30.138710] smpboot: Total of 1 processors activated (7008.06 BogoMIPS)

Full logs: https://pastebin.com/5A1diVwr

lscpu output in Guest DSM:

Architecture:           x86_64
  CPU op-mode(s):       32-bit, 64-bit
  Address sizes:        39 bits physical, 48 bits virtual
  Byte Order:           Little Endian
CPU(s):                 4
  On-line CPU(s) list:  0
  Off-line CPU(s) list: 1-3
Vendor ID:              GenuineIntel
  Model name:           Intel(R) Xeon(R) E-2224G CPU @ 3.50GHz
    CPU family:         6
    Model:              158
    Thread(s) per core: 1
    Core(s) per socket: 1
    Socket(s):          1
    Stepping:           10
    BogoMIPS:           7008.06
    Flags:              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht pbe syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonst
                        op_tsc pni pclmulqdq dtes64 ds_cpl ssse3 sdbg fma cx16 xtpr pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single fsgsbase
                        bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed smap xsaveopt arat md_clear
Caches (sum of all):
  L1d:                  32 KiB (1 instance)
  L1i:                  32 KiB (1 instance)
  L2:                   256 KiB (1 instance)
  L3:                   8 MiB (1 instance)
Vulnerabilities:
  L1tf:                 Mitigation; PTE Inversion
  Mds:                  Vulnerable; SMT disabled
  Meltdown:             Vulnerable
  Spec store bypass:    Vulnerable
  Spectre v1:           Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
  Spectre v2:           Vulnerable, STIBP: disabled

As you can see here only 1 online and 3 offline CPU's. Can you help me with this issue?

System details:

CPU: Intel(R) Xeon(R) E-2224G CPU @ 3.50GHz (4 cores, 4 threads)
ARPL version: v1.0-beta13
Host OS: TrueNAS-13.0-U3.1 CORE
Hypervisor: BHYVE
DSM Model: DS918+
DSM Version: DSM 7.1.1-42962 Update 3

[susadmin]

I have a similar problem. See: https://github.com/fbelavenuto/arpl/issues/408

Still waiting on a fix but it's good to know it's not just me.

[alikhanz]

I have a similar problem. See: https://github.com/fbelavenuto/arpl/issues/408

Still waiting on a fix but it's good to know it's not just me.

Ty for your reply. Yea, seems like your prob, except that my bootloader starting fine, and system started, but with one CPU. And i giving cores instead vcpu to my vm.

Also we have different hypervisors

[fbelavenuto]

Please check if your increase number of cores and not slots. Try add the cmdline entry "disable_mtrr_trim" with no value.