search

yunionio / cloudpods

A cloud-native open-source unified multi-cloud and hybrid-cloud platform. 开源、云原生的多云管理及混合云融合平台
https://www.cloudpods.org
Apache License 2.0
2.55k stars 520 forks source link

[BUG] 宝德服务器创建虚拟机失败 #21003

Open fangpsh opened 1 month ago

fangpsh commented 1 month ago

问题描述/What happened: 新装机器，宝德PR2012A，宿主机系统openEuler 22.03 (LTS-SP4)，创建任何虚拟机，均在启动中，长时间后，手工同步状态，显示已关机。

尝试多个发行版，现象一致。

宿主机host 容器日志：

[info 2024-08-12 10:16:13 appsrv.(*Application).ServeHTTP(appsrv.go:289)] _qdQW-L4L5DVMgjUKdKcSVN1HeI= 200 f278c9-d7c0f6-033d97 POST /servers/0f3987aa-8c4d-455b-8d0a-31dd1fd07f4c/start (192.168.8.2:12125:compute_v2) 2.90ms
[error 2024-08-12 10:16:13 appsrv.execCallback.func1(workers.go:268)] WorkerManager exec callback error: runtime error: index out of range [0] with length 0
goroutine 860888 [running]:
runtime/debug.Stack()
        /usr/lib/go/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
        /usr/lib/go/src/runtime/debug/stack.go:16 +0x19
yunion.io/x/onecloud/pkg/appsrv.execCallback.func1()
        /root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:272 +0xdd
panic({0x2f05540, 0xc0015ea9c0})
        /usr/lib/go/src/runtime/panic.go:838 +0x207
yunion.io/x/onecloud/pkg/hostman/guestman.(*CpuSetCounter).AllocCpuset(0xc001cd7da0, 0x1, 0x30?, 0xc0?)
        /root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/guesthelper.go:314 +0x305
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).allocGuestNumaCpuset(0xc0015a7880)
        /root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:2885 +0xb7
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).initCpuDesc(0xc0015a7880, 0x0)
        /root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvmhelper.go:886 +0xb3
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).initGuestDesc(0xc0015a7880)
        /root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/pci.go:53 +0x25
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).updateGuestDesc(0xc0015a7880)
        /root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:155 +0x1f3
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).asyncScriptStart(0xc0015a7880, {0x37cd8a8, 0xc000efce40}, {0x3079880?, 0xc000eceac0})
        /root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:804 +0x1a5
yunion.io/x/onecloud/pkg/hostman/guestman.(*guestStartTask).Run(0xc000ecef40)
        /root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:2039 +0x3b
yunion.io/x/onecloud/pkg/appsrv.execCallback(0xc00006d678?)
        /root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:275 +0x58
yunion.io/x/onecloud/pkg/appsrv.(*SWorker).run(0xc00161a030)
        /root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:110 +0x17e
created by yunion.io/x/onecloud/pkg/appsrv.(*SWorkerManager).scheduleWithLock
        /root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:294 +0x165

环境/Environment:

fangpsh commented 1 month ago

触发重装，host-deployer 服务日志片段：

[info 2024-08-12 10:31:42 qemu_kvm.(*QemuKvmDriver).sshRun(driver.go:381)] QemuKvmDriver start command /opt/yunion/bin/host-deployer --config /opt/yunion/host.conf --deploy-action deploy_guest_fs --deploy-params-file /deploy_params
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuKvmDriver).DeployGuestfs(driver.go:414)] DeployGuestfs log:
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuKvmDriver).sshRun(driver.go:381)] QemuKvmDriver start command test -f /error && cat /error || true
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuKvmDriver).DeployGuestfs(driver.go:420)] deploy error str []
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuKvmDriver).sshRun(driver.go:381)] QemuKvmDriver start command test -f /response && cat /response || true
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuKvmDriver).DeployGuestfs(driver.go:430)] deploy response str [{"distro":"Ubuntu","version":"22.04","arch":"x86_64","os":"Linux","account":"root","key":"LzAxEmn6twTJsRAumJD/v3NDejDQroSDA8XWvYCylY4RvZAlfDuDdRHACNqKkD8neuAIjVKHvy48sP3Qv/whkchdbkQD69H+aRwRZjcqYA36n0CXplR93Tg8uv6wawX+IcrGpl4P9n7+5D6AQ54PJZu3iKSizJSY54a5dKqngULLguVemhrlFUVFGrEh0p2Fcw8OoKfI2p5WqM46sKk7DWwsP0JKtpB5gHeNS43bxMz9qvLUI8thNm+79TO3+cBBxpnXL1FC9g3XQ0MHvmiyHmPyImqgUcddQciIPv6iXl/fWkAOcr3ezOXVqyXrSxutTB+EcPWrHfvmXtDiSFwyKw10/1ORQ+SRDy+ZNaPGhN3TzYZ0FgQcAQ0NY5e6d2ZR+Wrkw0KeERNaEUGfgjORDrmgwaRO6MsCPBZh2KjCJciPYYUdGpMyH+bGpoaF5DjWIavLCDJQZwB1HH0XnzbcolxpQDZWWs+OR/Le+9kmDNSxzKGoN8Up0vebG03bDUs6XI/mK9ECZIz3ytIfryY1GOWzxahTPHKTPpqxoH0vA6AAfwoMsMFmqpFLcvCEHxJPSC08KfVI/hSMQvy5WIgFHHBrB2jCd5WqUpyBVN9+prul6BLHhToEQQJ5S8w63RJ9iG+2DQAGAmBEEcyCxm3XEXhFiOdSEsoiFYXv3BmvJa4=","telegraf_deployed":true}]
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuKvmDriver).sshRun(driver.go:381)] QemuKvmDriver start command test -f /log && cat /log
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuKvmDriver).DeployGuestfs.func1(driver.go:401)] DeployGuestfs log: [info 2024-08-12 10:31:42 qemu_kvm.(*LocalDiskDriver).Connect(local_driver.go:62)] found part dev /dev/sda
[info 2024-08-12 10:31:42 qemu_kvm.(*LocalDiskDriver).Connect(local_driver.go:62)] found part dev /dev/sda1
[info 2024-08-12 10:31:42 qemu_kvm.(*LocalDiskDriver).Connect(local_driver.go:62)] found part dev /dev/sda14
[info 2024-08-12 10:31:42 qemu_kvm.(*LocalDiskDriver).Connect(local_driver.go:62)] found part dev /dev/sda15
[info 2024-08-12 10:31:42 fsutils.MountRootfs(fsutils.go:468)] detect partition /dev/sda
[error 2024-08-12 10:31:42 kvmpart.(*SKVMGuestDiskPartition).Mount(kvmpart.go:119)] Mount fs failed: unsupport fs  on /dev/sda
[info 2024-08-12 10:31:42 fsutils.MountRootfs(fsutils.go:468)] detect partition /dev/sda1
[info 2024-08-12 10:31:42 guestfs.IsPartitionReadonly(core.go:219)] File system /tmp/_dev_sda1 is not readonly
[info 2024-08-12 10:31:42 kvmpart.(*SKVMGuestDiskPartition).Mount(kvmpart.go:149)] mount fs ext4 on /dev/sda1 successfully
[info 2024-08-12 10:31:42 fsutils.MountRootfs(fsutils.go:476)] Use rootfs UbuntuRootFs, partition /dev/sda1
[error 2024-08-12 10:31:42 fsdriver.(*sLinuxRootFs).GetArch(linux.go:502)] stat file /tmp/_dev_sda1/usr/lib64/ld-linux-x86-64.so.2: stat /tmp/_dev_sda1/usr/lib64/ld-linux-x86-64.so.2: no such file or directory
[error 2024-08-12 10:31:42 fsdriver.(*sLinuxRootFs).GetArch(linux.go:502)] stat file /tmp/_dev_sda1/lib64/ld-linux-x86-64.so.2: stat /tmp/_dev_sda1/lib64/ld-linux-x86-64.so.2: no such file or directory
[info 2024-08-12 10:31:42 guestfs.IsPartitionReadonly(core.go:219)] File system /tmp/_dev_sda1 is not readonly
[info 2024-08-12 10:31:43 fsdriver.(*sLinuxRootFs).DeployNetworkingScripts(linux.go:388)] netdev prefix: eth
[info 2024-08-12 10:31:44 kvmpart.(*SLocalGuestFS).Passwd(localfs.go:174)] Passwd  New password: Retype new password: passwd: password updated successfully

[info 2024-08-12 10:31:44 kvmpart.(*SLocalGuestFS).userAdd(localfs.go:331)] Useradd:
[info 2024-08-12 10:31:44 fsdriver.(*sLinuxRootFs).DeployYunionroot(linux.go:278)] DeployYunionroot cloudroot home /opt/cloudroot
[info 2024-08-12 10:31:44 guestfs.DoDeployGuestFs(core.go:199)] Deploy finished, return: distro:"Ubuntu"  version:"22.04"  arch:"x86_64"  os:"Linux"  account:"root"  key:"LzAxEmn6twTJsRAumJD/v3NDejDQroSDA8XWvYCylY4RvZAlfDuDdRHACNqKkD8neuAIjVKHvy48sP3Qv/whkchdbkQD69H+aRwRZjcqYA36n0CXplR93Tg8uv6wawX+IcrGpl4P9n7+5D6AQ54PJZu3iKSizJSY54a5dKqngULLguVemhrlFUVFGrEh0p2Fcw8OoKfI2p5WqM46sKk7DWwsP0JKtpB5gHeNS43bxMz9qvLUI8thNm+79TO3+cBBxpnXL1FC9g3XQ0MHvmiyHmPyImqgUcddQciIPv6iXl/fWkAOcr3ezOXVqyXrSxutTB+EcPWrHfvmXtDiSFwyKw10/1ORQ+SRDy+ZNaPGhN3TzYZ0FgQcAQ0NY5e6d2ZR+Wrkw0KeERNaEUGfgjORDrmgwaRO6MsCPBZh2KjCJciPYYUdGpMyH+bGpoaF5DjWIavLCDJQZwB1HH0XnzbcolxpQDZWWs+OR/Le+9kmDNSxzKGoN8Up0vebG03bDUs6XI/mK9ECZIz3ytIfryY1GOWzxahTPHKTPpqxoH0vA6AAfwoMsMFmqpFLcvCEHxJPSC08KfVI/hSMQvy5WIgFHHBrB2jCd5WqUpyBVN9+prul6BLHhToEQQJ5S8w63RJ9iG+2DQAGAmBEEcyCxm3XEXhFiOdSEsoiFYXv3BmvJa4="  telegraf_deployed:true
[info 2024-08-12 10:31:44 kvmpart.(*SKVMGuestDiskPartition).Umount(kvmpart.go:321)] umount /dev/sda1: /tmp/_dev_sda1
[info 2024-08-12 10:31:44 kvmpart.(*SKVMGuestDiskPartition).Umount(kvmpart.go:327)] umount /dev/sda1 successfully

[info 2024-08-12 10:31:45 monitor.(*HmpMonitor).write(hmp.go:125)] HMP Write : quit

[info 2024-08-12 10:31:45 monitor.(*HmpMonitor).read(hmp.go:79)] HMP Read : quit

[info 2024-08-12 10:31:45 monitor.(*HmpMonitor).read(hmp.go:91)] Scan over  ...
[error 2024-08-12 10:31:45 qemu_kvm.(*QemuX86Driver).StartGuest.func2(driver.go:695)] monitor disconnect %!s(<nil>)
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuBaseDriver).CleanGuest(driver.go:587)] kill  process kill: cannot find process "1438020
"
 exit status 1
[info 2024-08-12 10:31:45 qemu_kvm.(*QemuDeployManager).Release(driver.go:126)] release QemuDeployManager
fangpsh commented 1 month ago

https://github.com/yunionio/cloudpods/blob/v3.11.5/pkg/hostman/guestman/guesthelper.go#L308C8-L308C19

看起来是因为!NumaEnabled ，但是是因为只安装了一块 CPU 在 槽 2 上，导致数组越界？

补充下 CPU 信息： AMD EPYC 7551 32-Core Processor

 numastat
                           node0           node1           node2           node3
numa_hit                       0       352137638               0       237420104
numa_miss                      0               0               0               0
numa_foreign                   0               0               0               0
interleave_hit                 0            4099               0            4259
local_node                     0       138573607               0       129856737
other_node                     0       213564031               0       107563367

                           node4           node5           node6           node7
numa_hit                       0       404871280               0       475572715
numa_miss                      0               0               0               0
numa_foreign                   0               0               0               0
interleave_hit                 0            4090               0            4266
local_node                     0       265089169               0       217734416
other_node                     0       139782111               0       257838299
fangpsh commented 1 month ago 
 numactl -H
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3 4 5 6 7 64 65 66 67 68 69 70 71
node 0 size: 0 MB
node 0 free: 0 MB
node 1 cpus: 8 9 10 11 12 13 14 15 72 73 74 75 76 77 78 79
node 1 size: 31572 MB
node 1 free: 25994 MB
node 2 cpus: 16 17 18 19 20 21 22 23 80 81 82 83 84 85 86 87
node 2 size: 0 MB
node 2 free: 0 MB
node 3 cpus: 24 25 26 27 28 29 30 31 88 89 90 91 92 93 94 95
node 3 size: 32251 MB
node 3 free: 30763 MB
node 4 cpus: 32 33 34 35 36 37 38 39 96 97 98 99 100 101 102 103
node 4 size: 0 MB
node 4 free: 0 MB
node 5 cpus: 40 41 42 43 44 45 46 47 104 105 106 107 108 109 110 111
node 5 size: 32251 MB
node 5 free: 30283 MB
node 6 cpus: 48 49 50 51 52 53 54 55 112 113 114 115 116 117 118 119
node 6 size: 0 MB
node 6 free: 0 MB
node 7 cpus: 56 57 58 59 60 61 62 63 120 121 122 123 124 125 126 127
node 7 size: 32178 MB
node 7 free: 29502 MB
node distances:
node   0   1   2   3   4   5   6   7
  0:  10  16  16  16  32  32  32  32
  1:  16  10  16  16  32  32  32  32
  2:  16  16  10  16  32  32  32  32
  3:  16  16  16  10  32  32  32  32
  4:  32  32  32  32  10  16  16  16
  5:  32  32  32  32  16  10  16  16
  6:  32  32  32  32  16  16  10  16
  7:  32  32  32  32  16  16  16  10
fangpsh commented 1 month ago

https://github.com/yunionio/cloudpods/commit/8d05cf2067b59fe9769df0043c85a0a3c4ced2d0

这个 commit 看起来改动了这块代码，是已知问题？ @wanyaoqi

wanyaoqi commented 1 month ago

https://github.com/yunionio/cloudpods/blob/v3.11.5/pkg/hostman/guestman/guesthelper.go#L308C8-L308C19

看起来是因为!NumaEnabled ，但是是因为只安装了一块 CPU 在 槽 2 上，导致数组越界？

补充下 CPU 信息： AMD EPYC 7551 32-Core Processor

 numastat
                           node0           node1           node2           node3
numa_hit                       0       352137638               0       237420104
numa_miss                      0               0               0               0
numa_foreign                   0               0               0               0
interleave_hit                 0            4099               0            4259
local_node                     0       138573607               0       129856737
other_node                     0       213564031               0       107563367

                           node4           node5           node6           node7
numa_hit                       0       404871280               0       475572715
numa_miss                      0               0               0               0
numa_foreign                   0               0               0               0
interleave_hit                 0            4090               0            4266
local_node                     0       265089169               0       217734416
other_node                     0       139782111               0       257838299

@fangpsh 看起来是因为这个原因，能贴一下完整的 host日志文件吗

fangpsh commented 1 month ago

7551 32-Core


[info 240813 02:39:15 procutils.WaitZombieLoop(zombie_others.go:36)] My pid is not 1 and no need to wait zombies
[info 240813 02:39:15 options.parseOptions(options.go:336)] Use configuration file: /etc/yunion/host.conf
[warning 240813 02:39:15 structarg.(*ArgumentParser).parseJSONKeyValue(structarg.go:1215)] Cannot find argument start-host-ignore-sys-error
[warning 240813 02:39:15 structarg.(*ArgumentParser).parseJSONKeyValue(structarg.go:1215)] Cannot find argument enable-rbac
[warning 240813 02:39:15 structarg.(*ArgumentParser).parseJSONKeyValue(structarg.go:1215)] Cannot find argument health-driver
[warning 240813 02:39:15 structarg.(*ArgumentParser).parseJSONKeyValue(structarg.go:1215)] Cannot find argument disk-is-ssd
[warning 240813 02:39:15 structarg.(*ArgumentParser).parseJSONKeyValue(structarg.go:1215)] Cannot find argument enable-health-checker
[warning 240813 02:39:15 structarg.(*ArgumentParser).parseJSONKeyValue(structarg.go:1215)] Cannot find argument enable-qmp-monitor
[info 240813 02:39:15 options.parseOptions(options.go:359)] Set log level to "info"
[info 2024-08-13 02:39:15 options.parseOptions(options.go:336)] Use configuration file: /etc/yunion/common/common.conf
[info 2024-08-13 02:39:15 options.parseOptions(options.go:359)] Set log level to "info"
[info 2024-08-13 02:39:15 hostman.(*SHostService).InitService(host_services.go:64)] exec socket path: /var/run/onecloud/exec.sock
[info 2024-08-13 02:39:15 app.InitApp(app.go:32)] RequestWorkerCount: 8
[info 2024-08-13 02:39:15 appsrv.NewApplication(appsrv.go:121)] App hostId: _qdQW-L4L5DVMgjUKdKcSVN1HeI= (host,sz-node-8-20,192.168.8.20)
2024/08/13 02:39:15 Allow hosts []
[info 2024-08-13 02:39:15 appsrv.(*Application).SetDefaultTimeout(appsrv.go:137)] adjust application default timeout to 60.000000 seconds
[info 2024-08-13 02:39:15 hostinfo.DetectCpuInfo(hostinfohelper.go:78)] cpuinfo freq 2539
[info 2024-08-13 02:39:15 hostinfo.NewHostInfo(hostinfo.go:2445)] CPU Model AMD EPYC 7551 32-Core Processor Microcode 0x800126e
[info 2024-08-13 02:39:15 hostinfo.NewHostInfo(hostinfo.go:2465)] Get kubelet container image Fs: /opt/docker, eviction config: {"evictionHard":{"imagefs.available":{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05}},"memory.available":{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0}},"nodefs.available":{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05}},"nodefs.inodesFree":{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05}}}}
[error 2024-08-13 02:39:17 fileutils2.GetAllBlkdevsIoSchedulers(fileutils.go:171)] no block device avaiable
[info 2024-08-13 02:39:17 hostinfo.(*SHostInfo).prepareEnv(hostinfo.go:411)] I/O Scheduler switch to none
[info 2024-08-13 02:39:17 hostinfo.(*SHostInfo).getKubeReservedMemMb(hostinfo.go:1572)] Kubelet memory threshold subtracted: 100MB
[info 2024-08-13 02:39:17 hostinfo.(*SHostInfo).Init(hostinfo.go:196)] Start detectHostInfo
[info 2024-08-13 02:39:17 hostinfo.(*SHostInfo).detectKVMMaxCpus(hostinfo.go:885)] KVM API VERSION 12
[info 2024-08-13 02:39:17 hostinfo.(*SHostInfo).detectKVMMaxCpus(hostinfo.go:890)] KVM CAP MAX VCPUS: 288
[info 2024-08-13 02:39:17 hostinfo.(*SHostInfo).detectKVMMaxCpus(hostinfo.go:898)] KVM CAP NR VCPUS: 240
[info 2024-08-13 02:39:17 sysutils.detectNestSupport(kvm.go:146)] Host is support kvm nest ...
[info 2024-08-13 02:39:18 sysutils.detectNestSupport(kvm.go:151)] Host kvm nest is enabled ...
[info 2024-08-13 02:39:18 hostinfo.(*SHostInfo).detectOsDist(hostinfo.go:778)] DetectOsDist openEuler 22.03
[info 2024-08-13 02:39:18 hostinfo.(*SHostInfo).detectQemuVersion(hostinfo.go:852)] Detect qemu version is 4.2.0
[info 2024-08-13 02:39:18 hostinfo.(*SHostInfo).detectOvsVersion(hostinfo.go:993)] Detect OVS version is 2.12.4
[info 2024-08-13 02:39:18 hostinfo.(*SHostInfo).detectOvsKOVersion(hostinfo.go:1010)] kernel module openvswitch vermagic:       5.10.0-221.0.0.120.oe2203sp4.x86_64 SMP mod_unload modversions 
WARNING: failed to determine memory area for node: open /sys/devices/system/node/node0/hugepages: no such file or directory
[info 2024-08-13 02:39:18 hostinfo.(*SHostInfo).Init(hostinfo.go:205)] Start parseConfig
[info 2024-08-13 02:39:18 hostinfo.NewNIC(hostinfohelper.go:241)] IP 192.168.8.20/br0/eno1
[info 2024-08-13 02:39:18 hostbridge.(*SBaseBridgeDriver).ConfirmToConfig(hostbridge.go:180)] bridge br0 already has ip 192.168.8.20
[info 2024-08-13 02:39:18 hostinfo.NewNIC(hostinfohelper.go:291)] Confirm to configuration!!
[info 2024-08-13 02:39:18 hostinfo.(*SNIC).SetupDhcpRelay(hostinfohelper.go:203)] Not enable dhcp relay on nic: &hostinfo.SNIC{Inter:"eno1", Bridge:"br0", Ip:"192.168.8.20", Wire:"", WireId:"", Mask:24, Bandwidth:1000, BridgeDev:(*hostbridge.SOVSBridgeDriver)(0xc0017cf260), dhcpServer:(*hostdhcp.SGuestDHCPServer)(0xc0017cfe60)}
[info 2024-08-13 02:39:18 hostinfo.(*SHostInfo).setupOvnChassis(hostinfo.go:223)] Start setting up ovn chassis
[error 2024-08-13 02:39:18 auth.(*authManager).startRefreshRevokeTokens(auth.go:193)] refreshRevokeTokens: No valid admin token credential
[info 2024-08-13 02:39:19 hostman.(*SHostService).RunService.func1(host_services.go:85)] Auth complete!!
[info 2024-08-13 02:39:19 policy.(*SPolicyManager).init(policy.go:160)] policy fetch worker count 1
[info 2024-08-13 02:39:19 consts.SetNonDefaultDomainProjects(consts.go:109)] set non_default_domain_projects to false
[info 2024-08-13 02:39:19 options.StartOptionManagerWithSessionDriver(manager.go:68)] OptionManager start to fetch service configs with interval 30m0s ...
[info 2024-08-13 02:39:19 watcher.(*SInformerSyncManager).startWatcher(watcher.go:83)]EndpointChangeManager: Start resource informer watcher for endpoint
[info 2024-08-13 02:39:19 options.optionsEquals(manager.go:120)] Options added: {"api_server":"https://it-one.zego.cloud"}
[info 2024-08-13 02:39:19 watcher.(*SInformerSyncManager).startWatcher(watcher.go:83)]ServiceConfigManager: Start resource informer watcher for service
[info 2024-08-13 02:39:19 guestman.(*SGuestManager).InitQemuMaxCpus(guestman.go:147)] KVM max cpus count: 240
[info 2024-08-13 02:39:19 guestman.(*SGuestManager).InitQemuMaxCpus(guestman.go:165)] Machine type pc max cpus: 240
[info 2024-08-13 02:39:19 guestman.(*SGuestManager).InitQemuMaxCpus(guestman.go:165)] Machine type q35 max cpus: 240
[info 2024-08-13 02:39:19 informer.(*EtcdBackendForClient).StartClientWatch(etcd_client.go:84)] /onecloud/informer watched
[info 2024-08-13 02:39:19 informer.NewWatchManagerBySessionBg.func1(watcher.go:51)] callback with watchMan success.
[info 2024-08-13 02:39:19 guestman.(*SGuestManager).InitPythonPath.func1(guestman.go:180)] Python path /usr/bin/python
[info 2024-08-13 02:39:19 informer.(*EtcdBackendForClient).StartClientWatch(etcd_client.go:84)] /onecloud/informer watched
[info 2024-08-13 02:39:19 informer.NewWatchManagerBySessionBg.func1(watcher.go:51)] callback with watchMan success.
[info 2024-08-13 02:39:21 hostinfo.(*SHostInfo).ensureMasterNetworks(hostinfo.go:1208)] Master ip 192.168.8.20 to fetch wire
[info 2024-08-13 02:39:21 hostinfo.(*SHostInfo).initZoneInfo(hostinfo.go:1252)] Start GetZoneInfo c1c48acb-53f4-4ddd-8fb2-758c024da888
[info 2024-08-13 02:39:21 hostinfo.(*SHostInfo).ensureHostRecord(hostinfo.go:1294)] Master MAC: ac:1f:6b:e8:7c:8a
[info 2024-08-13 02:39:21 hostinfo.(*SHostInfo).initHostRecord(hostinfo.go:1153)] host health manager on host down 
[warning 2024-08-13 02:39:21 hostinfo.(*SHostInfo).isVirtualFunction(hostinfo.go:1650)] failed get nic eno1 phys_port_name: read /sys/class/net/eno1/phys_port_name: operation not supported
[warning 2024-08-13 02:39:21 hostinfo.(*SHostInfo).isVirtualFunction(hostinfo.go:1650)] failed get nic eno2 phys_port_name: read /sys/class/net/eno2/phys_port_name: operation not supported
[info 2024-08-13 02:39:21 hostinfo.(*SHostInfo).doSendPhysicalNicInfo(hostinfo.go:1730)] upload physical nic: eno2(ac:1f:6b:e8:7c:8b)
[info 2024-08-13 02:39:21 hostinfo.(*SHostInfo).doUploadNicInfoInternal(hostinfo.go:1747)] Upload NIC br: if:eno2
[info 2024-08-13 02:39:21 hostinfo.(*SHostInfo).doUploadNicInfoInternal(hostinfo.go:1747)] Upload NIC br:br0 if:eno1
[info 2024-08-13 02:39:21 isolated_device.getPassthroughGPUs(gpu.go:86)] filter address [], enableWhiteList: false
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:18.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:19.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1a.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1b.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1c.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1d.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1e.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.0 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1460]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.1 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1461]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.2 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1462]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.3 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1463]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.4 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1464]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.5 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1465]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.6 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1466]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[warning 2024-08-13 02:39:22 isolated_device.NewPCIDevice2(gpu.go:229)] fillPCIEInfo for line: "00:1f.7 \"Host bridge [0600]\" \"Advanced Micro Devices, Inc. [AMD] [1022]\" \"Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1467]\" -p00 \"\" \"\"", device: {}, error: device address is empty: {}
[info 2024-08-13 02:39:23 isolated_device.(*PCIDevice).IsBootVGA(gpu.go:385)] PCI address 12:00.0 is boot_vga: /sys/devices/pci0000:10/0000:10:01.1/0000:11:00.0/0000:12:00.0/boot_vga
[info 2024-08-13 02:39:23 isolated_device.getPassthroughGPUs(gpu.go:118)] skip boot vga device 12:00.0
[info 2024-08-13 02:39:24 hostinfo.(*SHostInfo).initIsolatedDevices(hostinfo.go:2016)] probeSyncIsolatedDevices []
[info 2024-08-13 02:39:24 hostinfo.(*SHostInfo).initStoragesInternal(hostinfo.go:1883)] Storage host_192.168.8.20_local_storage_0(local) mountpoint /opt/cloud/workspace/disks
[info 2024-08-13 02:39:24 storageman.(*SLocalStorage).GetAvailSizeMb(storage_local.go:218)] Storage /opt/cloud/workspace/disks and kubelet nodeFs share same device /dev/mapper/openeuler-root
[info 2024-08-13 02:39:24 storageman.(*SLocalStorage).GetAvailSizeMb(storage_local.go:223)] Storage /opt/cloud/workspace/disks sizeMb 3562119, usablePercent 0.950000
[warning 2024-08-13 02:39:24 storageman.(*SLocalStorage).SyncStorageInfo(storage_local.go:296)] get hardware info: storage: host_192.168.8.20_local_storage_0, [read model file: /sys/class/block/device/model: open /sys/class/block/device/model: no such file or directory, read vendor file: /sys/class/block/device/vendor: open /sys/class/block/device/vendor: no such file or directory]
[info 2024-08-13 02:39:24 storageman.(*SLocalStorage).SyncStorageInfo(storage_local.go:306)] Sync storage info 05487ad1-4f24-4d64-8984-c4dbfe27b5d7/host_192.168.8.20_local_storage_0
[info 2024-08-13 02:39:24 hostinfo.(*SHostInfo).onSyncStorageInfoSucc(hostinfo.go:1970)] storage id 05487ad1-4f24-4d64-8984-c4dbfe27b5d7
[info 2024-08-13 02:39:25 hostinfo.(*SHostInfo).onSucc(hostinfo.go:2140)] Host registration process success....
[info 2024-08-13 02:39:25 hostinfo.(*SHostPingTask).Start(hostpinger.go:76)] Start host pinger ...
[info 2024-08-13 02:39:25 guestman.NewGuestCpuSetCounter(guesthelper.go:253)] cpusetcounter {"numa_enabled":false}
[info 2024-08-13 02:39:25 guestman.(*SGuestManager).LoadExistingGuests(guestman.go:419)] Find existing guest 0f3987aa-8c4d-455b-8d0a-31dd1fd07f4c
[info 2024-08-13 02:39:25 hostdhcp.(*SGuestDHCPServer).Start(dhcpserver.go:72)] SGuestDHCPServer starting ...
[info 2024-08-13 02:39:25 guestman.(*SGuestManager).Bootstrap(guestman.go:255)] Loading existing guests ...
[info 2024-08-13 02:39:25 guestman.(*SGuestManager).OnLoadExistingGuestsComplete(guestman.go:326)] Load existing guests complete...
[info 2024-08-13 02:39:25 guestman.(*SKVMGuestInstance).ImportServer(qemu-kvm.go:969)] atalas-probe(0f3987aa-8c4d-455b-8d0a-31dd1fd07f4c) is stopped, pending_delete=false
[info 2024-08-13 02:39:25 app.InitApp(app.go:32)] RequestWorkerCount: 8
[info 2024-08-13 02:39:25 appsrv.NewApplication(appsrv.go:121)] App hostId: _qdQW-L4L5DVMgjUKdKcSVN1HeI= (host,sz-node-8-20,192.168.8.20)
2024/08/13 02:39:25 Allow hosts []
[info 2024-08-13 02:39:25 appsrv.(*Application).SetDefaultTimeout(appsrv.go:137)] adjust application default timeout to 60.000000 seconds
[info 2024-08-13 02:39:25 app.ServeForeverExtended(app.go:60)] Start listen on https://0.0.0.0:8885, isMaster: true
[info 2024-08-13 02:39:25 metadata.Start(metadatahandler.go:46)] Start metadata service on http://0.0.0.0:9885
[info 2024-08-13 02:39:35 appsrv.(*Application).ServeHTTP(appsrv.go:289)] _qdQW-L4L5DVMgjUKdKcSVN1HeI= 200 4af4db-79cff4-809135 GET /servers/0f3987aa-8c4d-455b-8d0a-31dd1fd07f4c/status (192.168.8.2:31529:compute_v2) 42.16ms
[info 2024-08-13 02:39:35 modules.TaskComplete(task.go:34)] Sync task 9af05982-18bd-4e8e-839f-fad58c8fa394 complete succ
[info 2024-08-13 02:40:18 ovnutils.configBridgeMtu.func1(ovnutils.go:42)] set brvpc MTU to 1500 success!
[info 2024-08-13 02:40:30 appsrv.(*Application).ServeHTTP(appsrv.go:289)] _qdQW-L4L5DVMgjUKdKcSVN1HeI= 200 cc6d25-cd60b8-87c27b POST /disks/image_cache (192.168.8.2:24545:compute_v2) 0.28ms
[info 2024-08-13 02:40:35 workmanager.(*workerTask).Run(manager.go:99)] DelayTask complete: {"image_id":"83e41416-b80e-46f9-87c2-013a5d210629","name":"83e41416-b80e-46f9-87c2-013a5d210629","path":"/opt/cloud/workspace/disks/image_cache/83e41416-b80e-46f9-87c2-013a5d210629","size":671088640}
[info 2024-08-13 02:40:36 modules.TaskComplete(task.go:34)] Sync task afddce5d-f870-414c-83c9-ee487e236cfb complete succ
[info 2024-08-13 02:40:36 appsrv.(*Application).ServeHTTP(appsrv.go:289)] _qdQW-L4L5DVMgjUKdKcSVN1HeI= 200 cc6d25-cd60b8-9a495b POST /disks/05487ad1-4f24-4d64-8984-c4dbfe27b5d7/create/dc943e5d-da57-472f-88f1-c07ab977a6e5 (192.168.8.2:29397:compute_v2) 0.41ms
[info 2024-08-13 02:40:36 storageman.(*SLocalDisk).Delete(disk_local.go:129)] Delete guest disk /opt/cloud/workspace/disks/dc943e5d-da57-472f-88f1-c07ab977a6e5
[info 2024-08-13 02:40:36 storageman.(*SLocalStorage).DeleteDiskfile(storage_local.go:437)] Start Delete /opt/cloud/workspace/disks/dc943e5d-da57-472f-88f1-c07ab977a6e5
[info 2024-08-13 02:40:36 storageman.(*SLocalStorage).DeleteDiskfile(storage_local.go:453)] Move deleted disk file /opt/cloud/workspace/disks/dc943e5d-da57-472f-88f1-c07ab977a6e5 to recycle /opt/cloud/workspace/disks/recycle_bin/20240813024036
[info 2024-08-13 02:40:36 storageman.(*SBaseStorage).CreateDiskByDiskinfo(storage_base.go:368)] storage local start create disk
[info 2024-08-13 02:40:36 storageman.(*SBaseStorage).CreateDiskByDiskinfo(storage_base.go:374)] CreateDiskFromTemplate disk_id: dc943e5d-da57-472f-88f1-c07ab977a6e5, disk_info: {"datastore":{"port":0},"disk_size_mb":30720,"encryption":false,"format":"qcow2","image_id":"83e41416-b80e-46f9-87c2-013a5d210629","rebuild":true,"snapshot_out_of_chain":false}
[info 2024-08-13 02:40:36 storageman.(*SLocalDisk).createFromTemplateAndResize(disk_local.go:259)] REQSIZE: 30720, RETSIZE: 2252
[info 2024-08-13 02:40:47 workmanager.(*workerTask).Run(manager.go:99)] DelayTask complete: {"disk_id":"dc943e5d-da57-472f-88f1-c07ab977a6e5","disk_path":"/opt/cloud/workspace/disks/dc943e5d-da57-472f-88f1-c07ab977a6e5","disk_size":30720,"format":"qcow2"}
[info 2024-08-13 02:40:47 modules.TaskComplete(task.go:34)] Sync task 0af24151-14bb-4525-804b-f41654f1533d complete succ
[info 2024-08-13 02:40:47 appsrv.(*Application).ServeHTTP(appsrv.go:289)] _qdQW-L4L5DVMgjUKdKcSVN1HeI= 200 cc6d25-cd60b8-3dcffb POST /servers/0f3987aa-8c4d-455b-8d0a-31dd1fd07f4c/rebuild (192.168.8.2:52011:compute_v2) 1.04ms
[info 2024-08-13 02:40:57 workmanager.(*workerTask).Run(manager.go:99)] DelayTask complete: {"account":"root","arch":"x86_64","distro":"Ubuntu","key":"nELqzosscTVl3X1pFfEwh04nwiIDFnJ+EzqEW0tX/nl6vP8kJw7Gw+iMN1OvMIVbFf7j5N226dzMAIw+F7RY1v89x6RmWzRlw9BGWEmitBJeWfVHp+qLggWe4OhYoSr+cl2AyoltDZH9YFcWP9bK6SiytvREzTSrNfVuiY5XTlsKyd6XHtCco1vYJF+TvaSMNuAKsf3H1SM22RKtDNYe0/x+2ZemfciSWDm1inUi5L1HrAbWgdy3siFVWAeFLh+HYmIlkrQ2W4Lykb0hRpBYe9v56hJlvpLzxl9h6U6upqiz9hIpw540ETCygBL1b20AlofHWNlJUZvLytUQ9TZeuFq3Pzn+EYZHl9IbUaINFBTZygnt2/RnWezc6jV0YcCJKPyCMTHPuDgCRqOL+8Jt5qNa6Fj56sYg3pjpe1SwKVR/aQbOqPV3HbfPLuKRDhtcLnZ/DK2CUmm9XZ32BJovmBKvc4KidoyNbaN/UuSEbTCE1wDsk1zjiiP/w/rUkup8cuqU1/GjK5nKs6X2OvcYPcXwqYf7A3kXv7RKyLQK0qF6dSf8w/7V7f2BlrdqjcU6n4PgrJmCmV8bLSkh2Lak4S4+Ou6XEWajbHmocS2ub81hqwA7N64p1hdGEi1pT0g5tGseBET0zaey3ygN7k7V2IC4KjrwZhNnPFa6X8DuuZw=","os":"Linux","telegraf_deployed":true,"version":"22.04"}
[info 2024-08-13 02:40:57 modules.TaskComplete(task.go:34)] Sync task 396099d0-3994-47c0-8903-6c039553b145 complete succ
[info 2024-08-13 02:40:57 appsrv.(*Application).ServeHTTP(appsrv.go:289)] _qdQW-L4L5DVMgjUKdKcSVN1HeI= 200 cc6d25-cd60b8-0995da GET /servers/0f3987aa-8c4d-455b-8d0a-31dd1fd07f4c/status (192.168.8.2:47476:compute_v2) 0.22ms
[info 2024-08-13 02:40:57 modules.TaskComplete(task.go:34)] Sync task 678e560f-6423-4438-80fb-615a33e3ea0f complete succ
[info 2024-08-13 02:40:58 appsrv.(*Application).ServeHTTP(appsrv.go:289)] _qdQW-L4L5DVMgjUKdKcSVN1HeI= 200 cc6d25-cd60b8-8ef163 POST /servers/0f3987aa-8c4d-455b-8d0a-31dd1fd07f4c/start (192.168.8.2:13923:compute_v2) 2.60ms
[error 2024-08-13 02:40:58 appsrv.execCallback.func1(workers.go:268)] WorkerManager exec callback error: runtime error: index out of range [0] with length 0
goroutine 2823 [running]:
runtime/debug.Stack()
/usr/lib/go/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
/usr/lib/go/src/runtime/debug/stack.go:16 +0x19
yunion.io/x/onecloud/pkg/appsrv.execCallback.func1()
/root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:272 +0xdd
panic({0x2f05540, 0xc0008968b8})
/usr/lib/go/src/runtime/panic.go:838 +0x207
yunion.io/x/onecloud/pkg/hostman/guestman.(*CpuSetCounter).AllocCpuset(0xc00291be00, 0x1, 0x30?, 0xc0?)
/root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/guesthelper.go:314 +0x305
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).allocGuestNumaCpuset(0xc000440a80)
/root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:2885 +0xb7
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).initCpuDesc(0xc000440a80, 0x0)
/root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvmhelper.go:886 +0xb3
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).initGuestDesc(0xc000440a80)
/root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/pci.go:53 +0x25
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).updateGuestDesc(0xc000440a80)
/root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:155 +0x1f3
yunion.io/x/onecloud/pkg/hostman/guestman.(*SKVMGuestInstance).asyncScriptStart(0xc000440a80, {0x37cd8a8, 0xc001df2990}, {0x3079880?, 0xc001221440})
/root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:804 +0x1a5
yunion.io/x/onecloud/pkg/hostman/guestman.(*guestStartTask).Run(0xc0012218c0)
/root/go/src/yunion.io/x/onecloud/pkg/hostman/guestman/qemu-kvm.go:2039 +0x3b
yunion.io/x/onecloud/pkg/appsrv.execCallback(0x0?)
/root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:275 +0x58
yunion.io/x/onecloud/pkg/appsrv.(*SWorker).run(0xc001aa88d0)
/root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:110 +0x17e
created by yunion.io/x/onecloud/pkg/appsrv.(*SWorkerManager).scheduleWithLock
/root/go/src/yunion.io/x/onecloud/pkg/appsrv/workers.go:294 +0x165
wanyaoqi commented 1 month ago

@fangpsh 麻烦贴一下lscpu 的输出信息

fangpsh commented 1 month ago

lscpu

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  128
  On-line CPU(s) list:   0-127
Vendor ID:               AuthenticAMD
  BIOS Vendor ID:        Advanced Micro Devices, Inc.
  Model name:            AMD EPYC 7551 32-Core Processor
    BIOS Model name:     AMD EPYC 7551 32-Core Processor
    CPU family:          23
    Model:               1
    Thread(s) per core:  2
    Core(s) per socket:  32
    Socket(s):           2
    Stepping:            2
    Frequency boost:     enabled
    CPU max MHz:         2000.0000
    CPU min MHz:         1200.0000
    BogoMIPS:            4000.04
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr
                          sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl non
                         stop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_
                         1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legac
                         y abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_n
                         b bpext perfctr_llc mwaitx cpb hw_pstate ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2
                         rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 clzero irperf xsaveerptr arat
                         npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter p
                         fthreshold avic v_vmsave_vmload vgif overflow_recov succor smca ibpb_brtype
Virtualization features:
  Virtualization:        AMD-V
Caches (sum of all):
  L1d:                   2 MiB (64 instances)
  L1i:                   4 MiB (64 instances)
  L2:                    32 MiB (64 instances)
  L3:                    128 MiB (16 instances)
NUMA:
  NUMA node(s):          8
  NUMA node0 CPU(s):     0-7,64-71
  NUMA node1 CPU(s):     8-15,72-79
  NUMA node2 CPU(s):     16-23,80-87
  NUMA node3 CPU(s):     24-31,88-95
  NUMA node4 CPU(s):     32-39,96-103
  NUMA node5 CPU(s):     40-47,104-111
  NUMA node6 CPU(s):     48-55,112-119
  NUMA node7 CPU(s):     56-63,120-127
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Mitigation; untrained return thunk; SMT vulnerable
  Spec rstack overflow:  Mitigation; Safe RET
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, STIBP disabled, RSB filling, PBRSB-eIBRS Not a
                         ffected
  Srbds:                 Not affected
  Tsx async abort:       Not affected
fangpsh commented 1 month ago

image image

sys_info

{
    "cpu_info": {
        "processors": [
            {
                "capabilities": [
                    "fpu",
                    "vme",
                    "de",
                    "pse",
                    "tsc",
                    "msr",
                    "pae",
                    "mce",
                    "cx8",
                    "apic",
                    "sep",
                    "mtrr",
                    "pge",
                    "mca",
                    "cmov",
                    "pat",
                    "pse36",
                    "clflush",
                    "mmx",
                    "fxsr",
                    "sse",
                    "sse2",
                    "ht",
                    "syscall",
                    "nx",
                    "mmxext",
                    "fxsr_opt",
                    "pdpe1gb",
                    "rdtscp",
                    "lm",
                    "constant_tsc",
                    "rep_good",
                    "nopl",
                    "nonstop_tsc",
                    "cpuid",
                    "extd_apicid",
                    "amd_dcm",
                    "aperfmperf",
                    "pni",
                    "pclmulqdq",
                    "monitor",
                    "ssse3",
                    "fma",
                    "cx16",
                    "sse4_1",
                    "sse4_2",
                    "movbe",
                    "popcnt",
                    "aes",
                    "xsave",
                    "avx",
                    "f16c",
                    "rdrand",
                    "lahf_lm",
                    "cmp_legacy",
                    "svm",
                    "extapic",
                    "cr8_legacy",
                    "abm",
                    "sse4a",
                    "misalignsse",
                    "3dnowprefetch",
                    "osvw",
                    "skinit",
                    "wdt",
                    "tce",
                    "topoext",
                    "perfctr_core",
                    "perfctr_nb",
                    "bpext",
                    "perfctr_llc",
                    "mwaitx",
                    "cpb",
                    "hw_pstate",
                    "ssbd",
                    "ibpb",
                    "vmmcall",
                    "fsgsbase",
                    "bmi1",
                    "avx2",
                    "smep",
                    "bmi2",
                    "rdseed",
                    "adx",
                    "smap",
                    "clflushopt",
                    "sha_ni",
                    "xsaveopt",
                    "xsavec",
                    "xgetbv1",
                    "clzero",
                    "irperf",
                    "xsaveerptr",
                    "arat",
                    "npt",
                    "lbrv",
                    "svm_lock",
                    "nrip_save",
                    "tsc_scale",
                    "vmcb_clean",
                    "flushbyasid",
                    "decodeassists",
                    "pausefilter",
                    "pfthreshold",
                    "avic",
                    "v_vmsave_vmload",
                    "vgif",
                    "overflow_recov",
                    "succor",
                    "smca",
                    "ibpb_brtype"
                ],
                "cores": [
                    {
                        "id": 0,
                        "index": 0,
                        "logical_processors": [
                            0,
                            64
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 1,
                        "logical_processors": [
                            1,
                            65
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 10,
                        "logical_processors": [
                            10,
                            74
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 11,
                        "logical_processors": [
                            11,
                            75
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 12,
                        "logical_processors": [
                            12,
                            76
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 13,
                        "logical_processors": [
                            13,
                            77
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 14,
                        "logical_processors": [
                            14,
                            78
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 15,
                        "logical_processors": [
                            15,
                            79
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 16,
                        "logical_processors": [
                            16,
                            80
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 17,
                        "logical_processors": [
                            17,
                            81
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 18,
                        "logical_processors": [
                            18,
                            82
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 19,
                        "logical_processors": [
                            19,
                            83
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 2,
                        "logical_processors": [
                            2,
                            66
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 20,
                        "logical_processors": [
                            20,
                            84
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 21,
                        "logical_processors": [
                            21,
                            85
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 22,
                        "logical_processors": [
                            22,
                            86
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 23,
                        "logical_processors": [
                            23,
                            87
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 24,
                        "logical_processors": [
                            24,
                            88
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 25,
                        "logical_processors": [
                            25,
                            89
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 26,
                        "logical_processors": [
                            26,
                            90
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 27,
                        "logical_processors": [
                            27,
                            91
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 28,
                        "logical_processors": [
                            28,
                            92
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 29,
                        "logical_processors": [
                            29,
                            93
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 3,
                        "logical_processors": [
                            3,
                            67
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 30,
                        "logical_processors": [
                            30,
                            94
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 31,
                        "logical_processors": [
                            31,
                            95
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 4,
                        "logical_processors": [
                            4,
                            68
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 5,
                        "logical_processors": [
                            5,
                            69
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 6,
                        "logical_processors": [
                            6,
                            70
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 7,
                        "logical_processors": [
                            7,
                            71
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 8,
                        "logical_processors": [
                            72,
                            8
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 9,
                        "logical_processors": [
                            73,
                            9
                        ],
                        "total_threads": 2
                    }
                ],
                "id": 0,
                "model": "AMD EPYC 7551 32-Core Processor",
                "total_cores": 32,
                "total_threads": 64,
                "vendor": "AuthenticAMD"
            },
            {
                "capabilities": [
                    "fpu",
                    "vme",
                    "de",
                    "pse",
                    "tsc",
                    "msr",
                    "pae",
                    "mce",
                    "cx8",
                    "apic",
                    "sep",
                    "mtrr",
                    "pge",
                    "mca",
                    "cmov",
                    "pat",
                    "pse36",
                    "clflush",
                    "mmx",
                    "fxsr",
                    "sse",
                    "sse2",
                    "ht",
                    "syscall",
                    "nx",
                    "mmxext",
                    "fxsr_opt",
                    "pdpe1gb",
                    "rdtscp",
                    "lm",
                    "constant_tsc",
                    "rep_good",
                    "nopl",
                    "nonstop_tsc",
                    "cpuid",
                    "extd_apicid",
                    "amd_dcm",
                    "aperfmperf",
                    "pni",
                    "pclmulqdq",
                    "monitor",
                    "ssse3",
                    "fma",
                    "cx16",
                    "sse4_1",
                    "sse4_2",
                    "movbe",
                    "popcnt",
                    "aes",
                    "xsave",
                    "avx",
                    "f16c",
                    "rdrand",
                    "lahf_lm",
                    "cmp_legacy",
                    "svm",
                    "extapic",
                    "cr8_legacy",
                    "abm",
                    "sse4a",
                    "misalignsse",
                    "3dnowprefetch",
                    "osvw",
                    "skinit",
                    "wdt",
                    "tce",
                    "topoext",
                    "perfctr_core",
                    "perfctr_nb",
                    "bpext",
                    "perfctr_llc",
                    "mwaitx",
                    "cpb",
                    "hw_pstate",
                    "ssbd",
                    "ibpb",
                    "vmmcall",
                    "fsgsbase",
                    "bmi1",
                    "avx2",
                    "smep",
                    "bmi2",
                    "rdseed",
                    "adx",
                    "smap",
                    "clflushopt",
                    "sha_ni",
                    "xsaveopt",
                    "xsavec",
                    "xgetbv1",
                    "clzero",
                    "irperf",
                    "xsaveerptr",
                    "arat",
                    "npt",
                    "lbrv",
                    "svm_lock",
                    "nrip_save",
                    "tsc_scale",
                    "vmcb_clean",
                    "flushbyasid",
                    "decodeassists",
                    "pausefilter",
                    "pfthreshold",
                    "avic",
                    "v_vmsave_vmload",
                    "vgif",
                    "overflow_recov",
                    "succor",
                    "smca",
                    "ibpb_brtype"
                ],
                "cores": [
                    {
                        "id": 0,
                        "index": 4,
                        "logical_processors": [
                            100,
                            36
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 5,
                        "logical_processors": [
                            101,
                            37
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 6,
                        "logical_processors": [
                            102,
                            38
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 7,
                        "logical_processors": [
                            103,
                            39
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 8,
                        "logical_processors": [
                            104,
                            40
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 9,
                        "logical_processors": [
                            105,
                            41
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 10,
                        "logical_processors": [
                            106,
                            42
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 11,
                        "logical_processors": [
                            107,
                            43
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 12,
                        "logical_processors": [
                            108,
                            44
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 13,
                        "logical_processors": [
                            109,
                            45
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 14,
                        "logical_processors": [
                            110,
                            46
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 15,
                        "logical_processors": [
                            111,
                            47
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 16,
                        "logical_processors": [
                            112,
                            48
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 17,
                        "logical_processors": [
                            113,
                            49
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 18,
                        "logical_processors": [
                            114,
                            50
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 19,
                        "logical_processors": [
                            115,
                            51
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 20,
                        "logical_processors": [
                            116,
                            52
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 21,
                        "logical_processors": [
                            117,
                            53
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 22,
                        "logical_processors": [
                            118,
                            54
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 23,
                        "logical_processors": [
                            119,
                            55
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 24,
                        "logical_processors": [
                            120,
                            56
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 25,
                        "logical_processors": [
                            121,
                            57
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 26,
                        "logical_processors": [
                            122,
                            58
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 27,
                        "logical_processors": [
                            123,
                            59
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 28,
                        "logical_processors": [
                            124,
                            60
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 29,
                        "logical_processors": [
                            125,
                            61
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 30,
                        "logical_processors": [
                            126,
                            62
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 31,
                        "logical_processors": [
                            127,
                            63
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 0,
                        "logical_processors": [
                            32,
                            96
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 1,
                        "logical_processors": [
                            33,
                            97
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 2,
                        "logical_processors": [
                            34,
                            98
                        ],
                        "total_threads": 2
                    },
                    {
                        "id": 0,
                        "index": 3,
                        "logical_processors": [
                            35,
                            99
                        ],
                        "total_threads": 2
                    }
                ],
                "id": 1,
                "model": "AMD EPYC 7551 32-Core Processor",
                "total_cores": 32,
                "total_threads": 64,
                "vendor": "AuthenticAMD"
            }
        ],
        "total_cores": 64,
        "total_threads": 128
    },
    "hugepage_size_kb": 0,
    "hugepages_option": "transparent",
    "kernel_version": "5.10.0-221.0.0.120.oe2203sp4.x86_64",
    "kvm_module": "kvm-amd",
    "manufacture": "OEM",
    "model": "PR2012A",
    "motherboard_info": {
        "manufacture": "OEM",
        "model": "H11DSi",
        "oem_name": "oem",
        "sn": "ZM..",
        "version": "2.00"
    },
    "nest": "enabled",
    "oem_name": "oem",
    "os_distribution": "openEuler",
    "os_version": "22.03",
    "ovs_version": "2.12.4",
    "qemu_version": "4.2.0",
    "sn": "PR...",
    "storage_type": "rotate",
    "topology": {
        "architecture": "numa"
    },
    "version": "2.1"
}
wanyaoqi commented 1 month ago

这个问题的主要原因还是获取不到 cpu 缓存和 cpu id的绑定关系。这个commit 加了判断如果没有获取到则不分配 cpuset https://github.com/yunionio/cloudpods/pull/21039/commits/0a77ef6630834bc940bb5127e00661fcd9cbe724