search

yunionio / cloudpods

A cloud-native open-source unified multi-cloud and hybrid-cloud platform. 开源、云原生的多云管理及混合云融合平台
https://www.cloudpods.org
Apache License 2.0
2.55k stars 520 forks source link

[求助/Help]Ceph rbd 作为后端存储，创建虚拟机报：no enough backend "rbd" #18380

Closed chenjacken closed 10 months ago

chenjacken commented 10 months ago

一、版本 v3.10.6

二、root+ceph

[root@rook-ceph-tools-55dcd4bf7b-z457m /]# ceph osd status
+----+-------+-------+-------+--------+---------+--------+---------+-----------+
| id |  host |  used | avail | wr ops | wr data | rd ops | rd data |   state   |
+----+-------+-------+-------+--------+---------+--------+---------+-----------+
| 0  | node6 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 1  | node5 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 2  | node7 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 3  | node4 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 4  | node4 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 5  | node9 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
| 6  | node8 | 1031M |  464G |    0   |     0   |    0   |     0   | exists,up |
+----+-------+-------+-------+--------+---------+--------+---------+-----------+
[root@rook-ceph-tools-55dcd4bf7b-z457m /]# ceph osd pool ls detail
pool 2 'hwcloudpool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 274 pgp_num 262 pg_num_target 32 pgp_num_target 32 autoscale_mode on last_change 937 lfor 0/937/935 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd
        removed_snaps [1~3]

[root@rook-ceph-tools-55dcd4bf7b-z457m /]# ceph osd tree
ID  CLASS WEIGHT  TYPE NAME      STATUS REWEIGHT PRI-AFF 
 -1       3.18352 root default                           
 -9       0.90958     host node4                         
  3  nvme 0.45479         osd.3      up  1.00000 1.00000 
  4  nvme 0.45479         osd.4      up  1.00000 1.00000 
 -5       0.45479     host node5                         
  1  nvme 0.45479         osd.1      up  1.00000 1.00000 
 -3       0.45479     host node6                         
  0  nvme 0.45479         osd.0      up  1.00000 1.00000 
 -7       0.45479     host node7                         
  2  nvme 0.45479         osd.2      up  1.00000 1.00000 
-13       0.45479     host node8                         
  6  nvme 0.45479         osd.6      up  1.00000 1.00000 
-11       0.45479     host node9                         
  5  nvme 0.45479         osd.5      up  1.00000 1.00000

三、平台存储关联rbd存储：

image

四、创建虚拟机时候报的信息：

image

五、命令创建镜像，是可以的

[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd ls hwcloudpool
[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd create --size 10240 hwcloudpool/test-image01
[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd ls hwcloudpool
test-image01
[root@rook-ceph-tools-55dcd4bf7b-z457m /]# rbd info  hwcloudpool/test-image01
rbd image 'test-image01':
        size 10 GiB in 2560 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: e79d896e56fd
        block_name_prefix: rbd_data.e79d896e56fd
        format: 2
        features: layering
        op_features: 
        flags: 
        create_timestamp: Sun Oct 22 00:59:09 2023
        access_timestamp: Sun Oct 22 00:59:09 2023
        modify_timestamp: Sun Oct 22 00:59:09 2023

六、在web端创建硬盘，出现：调度失败的错误，日志：

[
    "No matching resources",
    "{\"error\":{\"code\":400,\"details\":\"Error #01: GetCandidatesByIds [b877f88b-5d8c-48bd-8d03-efd269104188]: object b877f88b-5d8c-48bd-8d03-efd269104188 not found\\n\",\"request\":{\"body\":\"{\\\"backup\\\":false,\\\"best_effort\\\":false,\\\"change_config\\\":false,\\\"count\\\":1,\\\"details\\\":false,\\\"disks\\\":[{\\\"backe...u_count\\\":0,\\\"vmem_size\\\":0}\",\"headers\":{\"Content-Length\":\"1156\",\"Content-Type\":\"application/json\",\"User-Agent\":\"yunioncloud-go/201708\",\"X-Auth-Token\":\"*\",\"X-Request-Id\":\"607ab5-52f075\",\"X-Yunion-Lang\":\"zh-CN\",\"X-Yunion-Parent-Id\":\"0.0\",\"X-Yunion-Peer-Service-Name\":\"compute_v2\",\"X-Yunion-Remote-Addr\":\"default-scheduler:30887\",\"X-Yunion-Span-Id\":\"0.0.0\",\"X-Yunion-Span-Name\":\"\",\"X-Yunion-Strace-Debug\":\"true\",\"X-Yunion-Strace-Id\":\"ae8e5d5a\"},\"method\":\"POST\",\"url\":\"https://default-scheduler:30887/scheduler\"}}}"
]
chenjacken commented 10 months ago

六、default-scheduler的日志[root@master1 ~]# kubectl -n onecloud logs default-scheduler-84bc8c87f4-p2fjs --since 10m -f

[info 2023-10-22 02:50:58 candidate.generalGetUpdateFunc.func1(default.go:93)] FullUpdateCounter: 1, update all baremetals.
[info 2023-10-22 02:50:58 cache.(*schedulerCache).loadObjects(cache.go:155)] Start load CandidateCache - Baremetals, period: 30s, ttl: 0s
[info 2023-10-22 02:50:58 cache.(*schedulerCache).loadObjects.func1(cache.go:159)] End load CandidateCache - Baremetals, elapsed 1.387069ms
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:239)] original schedtag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:263)] updated sched tag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:239)] original schedtag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:263)] updated sched tag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:239)] original schedtag map[string]*compute.SchedtagConfig{}
[info 2023-10-22 02:51:00 models.applyResourceSchedPolicy(schedpolicies.go:263)] updated sched tag map[string]*compute.SchedtagConfig{}
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "fe071d8d-46d3-4266-835d-93365960ec18", filter: "host_status", is_ok: false, reason: "enable_status is 'false', expected 'true'", error: <nil>
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "9eb94b52-f91c-4082-8014-3709c1920fd7", filter: "host_status", is_ok: false, reason: "enable_status is 'false', expected 'true'", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.7","access_mac":"6c:0b:84:d7:1c:df","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2802,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T14:29:25.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node6-172-16-1-7","id":"91018292-660b-498f-8d8e-984afcd66625","imported_at":"2023-10-19T14:29:25.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:29:13.000000Z","manager_uri":"https://172.16.1.7:8885","mem_cmtbound":1,"mem_reserved":13155,"mem_size":64355,"name":"node6-172-16-1-7","node_count":2,"ovn_mapped_ip_addr":"100.64.0.131","ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC0DWDVW","source":"local","status":"running","storage_driver":"Linux","storage_size":443695,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":50,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20221208.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC0DWDVW","storage_type":"ssd","topology":{"architecture":"numa","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":1,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":11,"index":2,"logical_processors":[10,38],"total_threads":2},{"id":12,"index":3,"logical_processors":[11,39],"total_threads":2},{"id":13,"index":4,"logical_processors":[12,40],"total_threads":2},{"id":14,"index":5,"logical_processors":[13,41],"total_threads":2},{"id":2,"index":6,"logical_processors":[2,30],"total_threads":2},{"id":3,"index":7,"logical_processors":[3,31],"total_threads":2},{"id":4,"index":8,"logical_processors":[32,4],"total_threads":2},{"id":5,"index":9,"logical_processors":[33,5],"total_threads":2},{"id":6,"index":10,"logical_processors":[34,6],"total_threads":2},{"id":8,"index":11,"logical_processors":[35,7],"total_threads":2},{"id":9,"index":12,"logical_processors":[36,8],"total_threads":2},{"id":10,"index":13,"logical_processors":[37,9],"total_threads":2}],"distances":[10,21],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":34359738368,"total_usable_bytes":33665085440}},{"caches":[{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":1,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":2,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":3,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":4,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":5,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":6,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":8,"index":7,"logical_processors":[21,49],"total_threads":2},{"id":9,"index":8,"logical_processors":[22,50],"total_threads":2},{"id":10,"index":9,"logical_processors":[23,51],"total_threads":2},{"id":11,"index":10,"logical_processors":[24,52],"total_threads":2},{"id":12,"index":11,"logical_processors":[25,53],"total_threads":2},{"id":13,"index":12,"logical_processors":[26,54],"total_threads":2},{"id":14,"index":13,"logical_processors":[27,55],"total_threads":2}],"distances":[21,10],"id":1,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":34359738368,"total_usable_bytes":33816535040}}]},"version":"70FR003JCN"},"update_version":656,"updated_at":"2023-10-20T01:29:13.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "91018292-660b-498f-8d8e-984afcd66625", filter: "host_memory", is_ok: false, reason: "no enough resource: node6-172-16-1-7, requested: 8192, total: 64355, free: 6144", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.11","access_mac":"6c:0b:84:11:01:4c","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":1959,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T21:25:14.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node7-172-16-1-11","id":"d2805334-011a-4ff8-876b-e4026d352a00","imported_at":"2023-10-19T21:25:14.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:35.000000Z","manager_uri":"https://172.16.1.11:8885","mem_cmtbound":1,"mem_reserved":18692,"mem_size":96516,"name":"node7-172-16-1-11","node_count":2,"ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC07HBGC","source":"local","status":"running","storage_driver":"Linux","storage_size":471791,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":76,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20230805.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC07HBGC","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":103079215104,"total_usable_bytes":101204516864}}]},"version":"70FR0017CN"},"update_version":247,"updated_at":"2023-10-20T01:28:35.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.5","access_mac":"6c:0b:84:81:ca:7d","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2389,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T21:01:04.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node4-172-16-1-5","id":"9e4c3c93-9605-4cc9-84ba-01c52157e78b","imported_at":"2023-10-19T21:01:04.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:28.000000Z","manager_uri":"https://172.16.1.5:8885","mem_cmtbound":1,"mem_reserved":13060,"mem_size":64260,"name":"node4-172-16-1-5","node_count":2,"ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC07L7XZ","source":"local","status":"running","storage_driver":"Linux","storage_size":443695,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":50,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20230805.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC07L7XZ","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":68719476736,"total_usable_bytes":67381719040}}]},"version":"70FR0017CN"},"update_version":271,"updated_at":"2023-10-20T01:28:28.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "9e4c3c93-9605-4cc9-84ba-01c52157e78b", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "d2805334-011a-4ff8-876b-e4026d352a00", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.10","access_mac":"6c:0b:84:bd:ec:cb","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2898,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T07:19:12.000000Z","deleted":false,"description":"主节点3","domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"master3-172-16-1-10","id":"9e438931-4ce8-4ed2-83e0-3a67f8b755d1","imported_at":"2023-10-19T07:19:12.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:50.000000Z","manager_uri":"https://172.16.1.10:8885","mem_cmtbound":1,"mem_reserved":10340,"mem_size":128772,"name":"master3-172-16-1-10","node_count":2,"ovn_mapped_ip_addr":"100.64.0.130","ovn_version":"2.12.4-2","page_size_kb":4,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC0AA8CL","source":"local","status":"running","storage_driver":"Linux","storage_size":471791,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","ida","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_size_kb":0,"hugepages_option":"transparent","kernel_version":"5.4.130-1.yn20221208.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC0AA8CL","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":137438953472,"total_usable_bytes":135027351552}}]},"version":"70FR001GCN"},"update_version":1083,"updated_at":"2023-10-20T01:28:50.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "9e438931-4ce8-4ed2-83e0-3a67f8b755d1", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[error 2023-10-22 02:51:00 predicates.(*SchedtagChecker).getDynamicSchedtags(schedtag_helper.go:234)] Condition parse eval: tag: "same_cloudprovider_project", condition: "host.cloudprovider.tenant_id == server.owner_tenant_id", desc: {"host":{"access_ip":"172.16.1.6","access_mac":"6c:0b:84:81:d3:08","auto_renew":false,"billing_type":"postpaid","cpu_architecture":"x86_64","cpu_cache":35840,"cpu_cmtbound":8,"cpu_count":56,"cpu_desc":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","cpu_mhz":2394,"cpu_microcode":"0xb00001e","cpu_reserved":0,"created_at":"2023-10-19T21:16:03.000000Z","deleted":false,"domain_id":"default","domain_src":"local","enable_health_check":true,"enable_pxe_boot":true,"enabled":true,"host_status":"online","host_type":"hypervisor","hostname":"node5-172-16-1-6","id":"0ec8e63a-10c1-4028-897c-1942435bcd65","imported_at":"2023-10-19T21:16:03.000000Z","ipmi_info":{"present":false},"is_baremetal":false,"is_emulated":false,"is_import":false,"is_maintenance":false,"is_public":true,"last_ping_at":"2023-10-20T01:28:28.000000Z","manager_uri":"https://172.16.1.6:8885","mem_cmtbound":1,"mem_reserved":18692,"mem_size":96516,"name":"node5-172-16-1-6","node_count":2,"ovn_version":"2.12.4-2","page_size_kb":1048576,"progress":100,"public_scope":"system","rack":"rack0","resource_type":"shared","slots":"slot0","sn":"PC0849AN","source":"local","status":"running","storage_driver":"Linux","storage_size":471791,"storage_type":"ssd","sys_info":{"cpu_info":{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56},"hugepage_nr":76,"hugepage_size_kb":1048576,"hugepages_option":"native","kernel_version":"5.4.130-1.yn20230805.el7.x86_64","kvm_module":"kvm-intel","manufacture":"Lenovo","model":"RD450X","nest":"enabled","oem_name":"lenovo","os_distribution":"CentOS Linux","os_version":"7.9.2009","ovs_version":"2.12.4","qemu_version":"4.2.0","sn":"PC0849AN","storage_type":"ssd","topology":{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":103079215104,"total_usable_bytes":101204717568}}]},"version":"70FRR00DCN"},"update_version":256,"updated_at":"2023-10-20T01:28:28.000000Z","version":"release/3.10(978c65dbc323101517)","zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},"server":{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","disk_type":"sys","image_id":"b6966dc3-c493-498a-84b9-3d2d903c3363","index":0,"medium":"hybrid","size":30720,"storage_id":"77008084-6a7a-42c0-8687-7c8b9222160a"}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","instance_type":"ecs.g1.c4m8","live_migrate":false,"nets":[{"bw_limit":0,"domain_id":"default","exit":false,"index":0,"num_queues":0,"private":false,"require_designated_ip":false,"require_teaming":false,"reserved":false,"rx_traffic_limit":0,"standby_addr_count":0,"standby_port_count":0,"try_teaming":false,"tx_traffic_limit":0,"vip":false}],"os_arch":"x86","owner_tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"session_id":"1697943060836","sku":"ecs.g1.c4m8","suggestion":false,"suggestion_all":true,"suggestion_limit":100,"vcpu_count":4,"vmem_size":8192}}, error: eval: evalBinary: eval binary left: evalSelector: unsupported type *conditionparser.funcCaller: invalid operation
[warning 2023-10-22 02:51:00 predicates.(*PredicateHelper).GetResult(predicates.go:89)]Filter Result: candidate: "0ec8e63a-10c1-4028-897c-1942435bcd65", filter: "host_storage", is_ok: false, reason: "no enough backend \"rbd\", mediumType \"hybrid\" storage, req=30720(30720), free=0()", error: <nil>
[info 2023-10-22 02:51:00 handler.timer.func1(handler.go:55)] Handler "/scheduler/forecast" cost: 56.267904ms
[info 2023-10-22 02:51:06 schedtag.(*dataManager).syncOnce(sync.go:118)] Schedtag data start sync
[info 2023-10-22 02:51:06 schedtag.(*dataManager).syncOnce(sync.go:135)] Schedtag finish sync, elapsed 5.949985ms
[info 2023-10-22 02:51:36 schedtag.(*dataManager).syncOnce(sync.go:118)] Schedtag data start sync
[info 2023-10-22 02:51:36 schedtag.(*dataManager).syncOnce(sync.go:135)] Schedtag finish sync, elapsed 6.369307ms
[info 2023-10-22 02:52:06 schedtag.(*dataManager).syncOnce(sync.go:118)] Schedtag data start sync
[info 2023-10-22 02:52:06 schedtag.(*dataManager).syncOnce(sync.go:135)] Schedtag finish sync, elapsed 5.699228ms
[info 2023-10-22 02:52:10 candidate.generalGetUpdateFunc.func1(default.go:99)] FullUpdateCounter: 1, update all hosts.
[info 2023-10-22 02:52:10 cache.(*schedulerCache).loadObjects(cache.go:155)] Start load CandidateCache - Hosts, period: 30s, ttl: 0s
[info 2023-10-22 02:52:10 cache.(*schedulerCache).loadObjects.func1(cache.go:159)] End load CandidateCache - Hosts, elapsed 36.242898ms
zexi commented 10 months ago

@chenjacken 这个存储有没有绑定到宿主机上？

chenjacken commented 10 months ago

@chenjacken 这个存储有没有绑定到宿主机上？

都绑定到了所有的计算节点和三个控制节点的宿主机。

chenjacken commented 10 months ago

发现一个问题： 1，创建虚拟机就报那个错误 2，直接创建一个硬盘，不指定块存储就报如上的那个错误调度失败，如果选择了指定的块存储，硬盘会创建成功。

image image image

调度失败日志：

[
    "No matching resources",
    "{\"error\":{\"code\":400,\"details\":\"Error #01: genericScheduler.Schedule: No resource are avaliable that match all of the following predicates: filter by host_status(-2), host_storage(-5), session_id=\\\"1697945758843\\\"\\n\",\"request\":{\"body\":\"{\\\"backup\\\":false,\\\"best_effort\\\":false,\\\"change_config\\\":false,\\\"count\\\":1,\\\"details\\\":false,\\\"disks\\\":[{\\\"backe...u_count\\\":0,\\\"vmem_size\\\":0}\",\"headers\":{\"Content-Length\":\"1100\",\"Content-Type\":\"application/json\",\"User-Agent\":\"yunioncloud-go/201708\",\"X-Auth-Token\":\"*\",\"X-Request-Id\":\"d0b8b5-1094bc\",\"X-Yunion-Lang\":\"zh-CN\",\"X-Yunion-Parent-Id\":\"0.0\",\"X-Yunion-Peer-Service-Name\":\"compute_v2\",\"X-Yunion-Remote-Addr\":\"default-scheduler:30887\",\"X-Yunion-Span-Id\":\"0.0.0\",\"X-Yunion-Span-Name\":\"\",\"X-Yunion-Strace-Debug\":\"true\",\"X-Yunion-Strace-Id\":\"4160b767\"},\"method\":\"POST\",\"url\":\"https://default-scheduler:30887/scheduler\"}}}"
]
zexi commented 10 months ago

[ "No matching resources", "{\"error\":{\"code\":400,\"details\":\"Error #01: genericScheduler.Schedule: No resource are avaliable that match all of the following predicates: filter by host_status(-2), host_storage(-5), session_id=\\"1697945758843\\"\n\",\"request\": ]

@chenjacken 使用下面的命令看下调度失败详细日志：

# 1697945758843 是上面日志里面的 session_id
climc scheduler-history-show --log --raw 1697945758843
chenjacken commented 10 months ago

[ "No matching resources", "{"error":{"code":400,"details":"Error #1: genericScheduler.Schedule: No resource are avaliable that match all of the following predicates: filter by host_status(-2), host_storage(-5), session_id=\"1697945758843\"\n","request": ]

@chenjacken 使用下面的命令看下调度失败详细日志：

# 1697945758843 是上面日志里面的 session_id
climc scheduler-history-show --log --raw 1697945758843
[root@master1 ~]# climc scheduler-history-show --log --raw 1697945758843
consuming: 0s
error: 'genericScheduler.Schedule: No resource are avaliable that match all of the
  following predicates: filter by host_status(-2), host_storage(-5), session_id="1697945758843"'
input: '{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","index":0,"medium":"hybrid","size":10240}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","live_migrate":false,"name":"test-image","pending_usages":[{"brand":"OneCloud","cloud_env":"onpremise","count":0,"cpu":0,"deleted":false,"domain_id":"default","group":0,"hypervisor":"kvm","isolated_device":0,"memory":0,"provider":"OneCloud","region_id":"default","storage":10240,"tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","update_version":0,"zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},{"bucket":0,"cache":0,"deleted":false,"eip":0,"eport":0,"instance_snapshot":0,"loadbalancer":0,"mongodb":0,"object_cnt":0,"object_gb":0,"port":0,"rds":0,"snapshot":0,"update_version":0}],"prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"suggestion":false,"suggestion_all":false,"suggestion_limit":100,"vcpu_count":0,"vmem_size":0}'
output: ""
session_id: "1697945758843"
tasks:
- capacity_map:
    0ec8e63a-10c1-4028-897c-1942435bcd65:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    91018292-660b-498f-8d8e-984afcd66625:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    9e438931-4ce8-4ed2-83e0-3a67f8b755d1:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    9e4c3c93-9605-4cc9-84ba-01c52157e78b:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    9eb94b52-f91c-4082-8014-3709c1920fd7:
      MinValue: -1
      Values:
        host_status:
          Value: 0
    d2805334-011a-4ff8-876b-e4026d352a00:
      MinValue: -1
      Values:
        host_storage:
          Value: 0
    fe071d8d-46d3-4266-835d-93365960ec18:
      MinValue: -1
      Values:
        host_status:
          Value: 0
  consuming: 2.606056ms
  data:
    __meta__: null
    ChangeConfig: false
    HasIsolatedDevice: false
    InstanceGroupsDetail: null
    PendingUsages:
    - {}
    - {}
    Raw: '{"backup":false,"baremetal_disk_configs":[{"conf":"none","count":0,"type":"hybrid"}],"best_effort":false,"change_config":false,"count":1,"details":false,"disks":[{"backend":"rbd","index":0,"medium":"hybrid","size":10240}],"domain_id":"default","has_isolated_device":false,"host_mem_page_size":0,"hypervisor":"hypervisor","live_migrate":false,"name":"test-image","pending_usages":[{"brand":"OneCloud","cloud_env":"onpremise","count":0,"cpu":0,"deleted":false,"domain_id":"default","group":0,"hypervisor":"kvm","isolated_device":0,"memory":0,"provider":"OneCloud","region_id":"default","storage":10240,"tenant_id":"43d375dfbfc1426d829bf8e28b8c3889","update_version":0,"zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852"},{"bucket":0,"cache":0,"deleted":false,"eip":0,"eport":0,"instance_snapshot":0,"loadbalancer":0,"mongodb":0,"object_cnt":0,"object_gb":0,"port":0,"rds":0,"snapshot":0,"update_version":0}],"prefer_region_id":"default","prefer_zone_id":"d7754248-9f87-4e31-81a9-517c7b0ef852","project_id":"43d375dfbfc1426d829bf8e28b8c3889","record_to_history":false,"resource_type":"shared","reuse_network":false,"suggestion":false,"suggestion_all":false,"suggestion_limit":100,"vcpu_count":0,"vmem_size":0}'
    UserCred:
      TokenCredential:
        Context:
          ip: 112.94.186.250
          source: web
        Domain: Default
        DomainId: default
        Expires: "2023-10-22T17:01:51Z"
        ProjectDomain: Default
        ProjectDomainId: default
        RoleIds: 849fb962d16a4dc78e8c93458aa57a1a
        Roles: admin
        Token: 4e1fc4931efe9490deba0a4d07137d8079291bec25820a819f7da985dddc73ce
        User: admin
        UserId: 4876b8260fd9493e858f9efa6b3c45a6
        tenant: system
        tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
    backup: false
    baremetal_disk_configs:
    - conf: none
      count: 0
      driver: ""
      range: null
      size: null
      splits: ""
      type: hybrid
    best_effort: false
    candidates: []
    cdrom: ""
    count: 1
    cpu_desc: ""
    cpu_microcode: ""
    cpu_mode: ""
    details: false
    disks:
    - backend: rbd
      backup_id: ""
      boot_index: null
      cache: ""
      disk_id: ""
      disk_type: ""
      driver: ""
      existing_path: ""
      format: ""
      fs: ""
      image_encrypt_key_id: ""
      image_id: ""
      image_properties: null
      index: 0
      medium: hybrid
      mountpoint: ""
      nvme_device: null
      os_arch: ""
      schedtags: []
      size: 10240
      snapshot_id: ""
      storage_id: ""
    domain_id: default
    for_guests: null
    group_releations: null
    groups: null
    guest_status: ""
    host_id: ""
    host_mem_page_size: 0
    hypervisor: hypervisor
    id: ""
    ignore_filters: {}
    instance_backup_id: ""
    instance_snapshot_id: ""
    instance_type: ""
    is_container: false
    is_daemon: null
    isolated_devices: null
    live_migrate: false
    name: test-image
    nets: null
    os_arch: ""
    prefer_backup_host: ""
    prefer_host_id: ""
    prefer_manager_id: ""
    prefer_region_id: default
    prefer_wire_id: ""
    prefer_zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
    project_id: 43d375dfbfc1426d829bf8e28b8c3889
    record_to_history: false
    required_candidates: 1
    resource_type: shared
    reuse_network: false
    schedtags: []
    session_id: "1697945758843"
    skip_cpu_check: null
    skip_kernel_check: null
    sku: ""
    suggestion: false
    suggestion_all: false
    suggestion_details: false
    suggestion_limit: 100
    tag: ""
    target_host_kernel: ""
    type: ""
    vcpu_count: 0
    vmem_size: 0
  error: 'genericScheduler.Schedule: No resource are avaliable that match all of the
    following predicates: filter by host_status(-2), host_storage(-5), session_id="1697945758843"'
  logs:
  - 'Failed: master1-172-16-1-8:fe071d8d-46d3-4266-835d-93365960ec18 [host_status]
    host_status: enable_status is ''false'', expected ''true'''
  - 'Failed: master2-172-16-1-9:9eb94b52-f91c-4082-8014-3709c1920fd7 [host_status]
    host_status: enable_status is ''false'', expected ''true'''
  - 'Success: master3-172-16-1-10:9e438931-4ce8-4ed2-83e0-3a67f8b755d1 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: master3-172-16-1-10:9e438931-4ce8-4ed2-83e0-3a67f8b755d1 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node4-172-16-1-5:9e4c3c93-9605-4cc9-84ba-01c52157e78b [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node4-172-16-1-5:9e4c3c93-9605-4cc9-84ba-01c52157e78b [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node5-172-16-1-6:0ec8e63a-10c1-4028-897c-1942435bcd65 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node5-172-16-1-6:0ec8e63a-10c1-4028-897c-1942435bcd65 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node6-172-16-1-7:91018292-660b-498f-8d8e-984afcd66625 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node6-172-16-1-7:91018292-660b-498f-8d8e-984afcd66625 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  - 'Success: node7-172-16-1-11:d2805334-011a-4ff8-876b-e4026d352a00 [class_metadata,host_domain,host_hypervisor_runtime,host_schedtag,host_status] '
  - 'Failed: node7-172-16-1-11:d2805334-011a-4ff8-876b-e4026d352a00 [host_storage]
    host_storage: no enough backend "rbd", mediumType "hybrid" storage, req=10240(10240),
    free=0()'
  result: null
  status: failed
  time: "2023-10-22 03:35:58"
  type: hypervisor
time: "2023-10-22 03:35:58"
zexi commented 10 months ago

@chenjacken 看下存储那里的介质类型，看报错是没有找到 backend 为 rbd ，mediumType 为 hybrid 的存储

host_storage: no enough backend "rbd", mediumType "hybrid" storage

另外用 climc storage-list 看下每个存储的属性:

climc storage-list
chenjacken commented 10 months ago

@chenjacken 看下存储那里的介质类型，看报错是没有找到 backend 为 rbd ，mediumType 为 hybrid 的存储

host_storage: no enough backend "rbd", mediumType "hybrid" storage

另外用 climc storage-list 看下每个存储的属性:

climc storage-list

是这个问题，所以创建硬盘时候指定存储块是成功的。

[root@master1 ~]# climc storage-list
+--------------------------------------+----------------------------------+----------+----------------------+--------+--------------+-------------+---------+--------------+
|                  ID                  |               Name               | Capacity | Actual_capacity_used | Status | Storage_type | Medium_type | Enabled | public_scope |
+--------------------------------------+----------------------------------+----------+----------------------+--------+--------------+-------------+---------+--------------+
| 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | 1054767  | 0                    | online | rbd          | hybrid      | true    | system       |
| 28a37de9-b491-4a16-8628-a282bccee8a8 | host_172.16.1.16_local_storage_0 | 455143   | 10128                | online | local        | ssd         | true    | system       |
| 1debb826-1d79-442e-8cf2-1dbd0cafdd60 | host_172.16.1.15_local_storage_0 | 459823   | 10120                | online | local        | ssd         | true    | system       |
| af307604-7179-4d51-8a9b-3dd68874e049 | host_172.16.1.11_local_storage_0 | 471791   | 24961                | online | local        | ssd         | true    | system       |
| 2d2cabfa-70d1-4229-8499-5fb1687e9f08 | host_172.16.1.6_local_storage_0  | 471791   | 10812                | online | local        | ssd         | true    | system       |
| c76bea60-b48b-4d49-89fa-72399efe1b06 | host_172.16.1.5_local_storage_0  | 443695   | 10650                | online | local        | ssd         | true    | system       |
| 7a36954f-44aa-47eb-88a3-4bb62cc398a9 | host_172.16.1.7_local_storage_0  | 443695   | 126198               | online | local        | ssd         | true    | system       |
| b11390f9-59db-4d9c-8608-d3f9fdb04886 | host_172.16.1.8_local_storage_0  | 443695   | 85723                | online | local        | ssd         | true    | system       |
| 6153222e-d59b-46ff-8f3a-cca7a804dc6b | host_172.16.1.9_local_storage_0  | 443695   | 228956               | online | local        | ssd         | true    | system       |
| 293ab9ee-952a-41cf-8a8c-a6d9098d33b9 | host_172.16.1.10_local_storage_0 | 471791   | 239840               | online | local        | ssd         | true    | system       |
+--------------------------------------+----------------------------------+----------+----------------------+--------+--------------+-------------+---------+--------------+
***  Total: 10 Pages: 1 Limit: 20 Offset: 0 Page: 1  ***
chenjacken commented 10 months ago

@zexi 帮忙看看，会是哪里出问题了？

zexi commented 10 months ago

@chenjacken 执行 climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b ，把结果发下

chenjacken commented 10 months ago 
[root@master1 ~]# climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b
access_ip: 172.16.1.5
access_mac: 6c:0b:84:81:ca:7d
auto_renew: false
billing_type: postpaid
cpu_architecture: x86_64
cpu_bound_count: 0
cpu_cache: 35840
cpu_cmtbound: 8
cpu_count: 56
cpu_desc: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
cpu_mhz: 2389
cpu_microcode: "0xb00001e"
cpu_reserved: 0
created_at: "2023-10-19T21:01:04.000000Z"
creating_cpu_count: 0
creating_guest_count: 0
creating_mem_size: 0
deleted: false
domain_id: default
domain_src: local
enable_health_check: true
enable_pxe_boot: true
enabled: true
fake_deleted_cpu_count: 0
fake_deleted_mem_size: 0
free_cpu_count: 440
free_mem_size: 43008
guest_count: 0
guest_reserved_resource:
  cpu_count: 8
  memory_size: 8192
  storage_size: 102400
guest_reserved_used:
  cpu_count: 0
  memory_size: 0
  storage_size: 0
host_status: online
host_type: hypervisor
hostname: node4-172-16-1-5
id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
imported_at: "2023-10-19T21:01:04.000000Z"
io_bound_count: 0
ipmi_info:
  present: false
is_baremetal: false
is_emulated: false
is_import: false
is_maintenance: false
is_public: true
isolated_devices:
- addr: "04:00.0"
  dev_type: GPU-HPC
  host_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
  id: fe5cf8b8-c42b-4d2d-8e06-b4d60d6f4e59
  model: GeForce RTX 3090
  vendor_device_id: 10de:2204
last_ping_at: "2023-10-20T01:28:28.000000Z"
manager_uri: https://172.16.1.5:8885
mem_cmtbound: 1
mem_reserved: 13060
mem_size: 64260
metadata:
  __auto_migrate_on_host_down: enable
  __auto_migrate_on_host_shutdown: enable
  __host_errors: '[{"content":"GPU 07:00.0 use kernel driver ast, skip it","name":"
    ","time":"2023-10-19T21:01:05.612871Z","type":"isolated_devices"}]'
  cpu_info: '{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56}'
  generate_name: node4-172-16-1-5
  hostname: node4-172-16-1-5
  hugepage_nr: "50"
  hugepage_size_kb: "1048576"
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  memory_used_mb: "54058"
  model: RD450X
  nest: enabled
  oem_name: lenovo
  on_kubernetes: "true"
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  root_partition_total_capacity_mb: "443695"
  root_partition_used_capacity_mb: "6957"
  sn: PC07L7XZ
  storage_type: ssd
  topology: '{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":68719476736,"total_usable_bytes":67381719040}}]}'
  version: 70FR0017CN
name: node4-172-16-1-5
net_interfaces:
  62d395c7-8a5c-4ea1-8741-1ccef5f521ca:
  - baremetal_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
    index: 0
    link_up: true
    mac: 6c:0b:84:81:ca:7d
    mtu: 0
    nic_type: admin
    rate: 0
    wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
networks:
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:16:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.6
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.6
  guest_ip6_mask: 0
  id: dab40ba9-01c7-48ad-8c16-a7ff708a36e8
  ifname_hint: adm023
  imported_at: "2023-10-19T21:16:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm023
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:12.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.10
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.10
  guest_ip6_mask: 0
  id: 3a632875-3c85-4056-8b10-8d2b9d521537
  ifname_hint: adm01
  imported_at: "2023-10-19T07:19:12.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm01
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:33:10.000000Z"
  deleted: false
  domain_id: default
  free_port: 51
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.200
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.150
  guest_ip6_mask: 0
  id: 51b8fbd8-9b90-48e6-8369-33fc86e58717
  ifname_hint: DHCP1
  imported_at: "2023-10-19T08:33:10.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: DHCP1
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T14:29:25.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.7
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.7
  guest_ip6_mask: 0
  id: ff1242fc-7b3f-42e8-878d-a4dfc562d6c0
  ifname_hint: adm021
  imported_at: "2023-10-19T14:29:25.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm021
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:00.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.8
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.8
  guest_ip6_mask: 0
  id: 0b27e637-cefa-4735-86fe-201116d34fa8
  ifname_hint: adm0
  imported_at: "2023-10-19T07:19:00.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm0
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: none
  public_src: local
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:01:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.5
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.5
  guest_ip6_mask: 0
  id: 0c973695-9738-403d-88ec-8620151137ce
  ifname_hint: adm022
  imported_at: "2023-10-19T21:01:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm022
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:30:40.000000Z"
  deleted: false
  description: 手动指定静态IP,自动分配请勿使用该子网。
  domain_id: default
  free_port: 15
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.90
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.70
  guest_ip6_mask: 0
  id: 7e49c4fd-0c08-49b3-8e2a-983bb8f8a0e2
  ifname_hint: HOST
  imported_at: "2023-10-19T08:30:40.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: Static
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:19.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.9
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.9
  guest_ip6_mask: 0
  id: 0a596e84-89ef-4f94-8c06-20b1f71449ed
  ifname_hint: adm02
  imported_at: "2023-10-19T07:19:19.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm02
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:25:14.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.11
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.11
  guest_ip6_mask: 0
  id: 8e82abdc-7e5f-45a2-8627-0b164ed3ed77
  ifname_hint: adm024
  imported_at: "2023-10-19T21:25:14.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm024
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
nics:
- link_up: true
  mac: 00:1b:21:bc:79:07
  masklen: 0
  mtu: 0
  rate: 0
- dns: 223.5.5.5,223.6.6.6
  gateway: 172.16.1.1
  ip_addr: 172.16.1.5
  link_up: true
  mac: 6c:0b:84:81:ca:7d
  masklen: 24
  mtu: 0
  net: adm022
  net_id: 0c973695-9738-403d-88ec-8620151137ce
  nic_type: admin
  rate: 0
  wire: bcast0
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- link_up: true
  mac: 6c:0b:84:81:ca:7e
  masklen: 0
  mtu: 0
  rate: 0
node_count: 2
ovn_version: 2.12.4-2
page_size_kb: 1048576
pending_usage:
  cpu: 0
  disk: {}
  instance_groups: {}
  isolated_device: 0
  memory: 0
  net: {}
progress: 100
public_scope: system
rack: rack0
region:
  created_at: "2023-10-19T06:12:20.000000Z"
  deleted: false
  description: Default Region
  enabled: true
  id: default
  imported_at: "2023-10-19T06:12:20.000000Z"
  is_emulated: false
  latitude: 0
  longitude: 0
  name: Default
  progress: 100
  provider: OneCloud
  source: local
  status: inservice
  update_version: 0
  updated_at: "2023-10-19T06:12:20.000000Z"
required_cpu_count: 0
required_mem_size: 0
resource_type: shared
running_cpu_count: 0
running_guest_count: 0
running_mem_size: 0
slots: slot0
sn: PC07L7XZ
source: local
status: running
storage_driver: Linux
storage_size: 443695
storage_type: ssd
storages:
- actual_capacity_used: 6957
  actual_free_capacity: 436738
  capacity: 443695
  cmtbound: 1
  created_at: "2023-10-19T21:01:06.000000Z"
  deleted: false
  domain_id: default
  domain_src: local
  enabled: true
  free_capacity: 443695
  id: c76bea60-b48b-4d49-89fa-72399efe1b06
  imported_at: "2023-10-19T21:01:06.000000Z"
  is_emulated: false
  is_public: true
  is_sys_disk_store: true
  medium_type: ssd
  name: host_172.16.1.5_local_storage_0
  progress: 100
  public_scope: system
  public_src: local
  reserved: 0
  source: local
  status: online
  storage_type: local
  storagecache_id: 42baacbf-e4d5-44ba-887c-b6bfdd579d36
  update_version: 61
  updated_at: "2023-10-20T01:27:28.000000Z"
  zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
sys_info:
  cpu_info:
...//more
      id: 0
      memory:
        supported_page_sizes:
        - 1073741824
        total_physical_bytes: 68719476736
        total_usable_bytes: 67381719040
  version: 70FR0017CN
total_cpu_count: 448
total_mem_size: 64260
update_version: 271
updated_at: "2023-10-20T01:28:28.000000Z"
version: release/3.10(978c65dbc323101517)
zone:
  cloudregion_id: default
  created_at: "2023-10-19T06:12:29.000000Z"
  deleted: false
  id: d7754248-9f87-4e31-81a9-517c7b0ef852
  imported_at: "2023-10-19T06:12:29.000000Z"
  is_emulated: false
  name: zone0
  progress: 100
  source: local
  status: enable
  update_version: 0
  updated_at: "2023-10-19T06:12:29.000000Z"
zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
zexi commented 10 months ago

@chenjacken 我看这台宿主机没有关联上这个存储，只有一个 local 的

storages:
- actual_capacity_used: 6957
  actual_free_capacity: 436738
  capacity: 443695
  cmtbound: 1
  created_at: "2023-10-19T21:01:06.000000Z"
  deleted: false
  domain_id: default
  domain_src: local
  enabled: true
  free_capacity: 443695
  id: c76bea60-b48b-4d49-89fa-72399efe1b06
  imported_at: "2023-10-19T21:01:06.000000Z"
  is_emulated: false
  is_public: true
  is_sys_disk_store: true
  medium_type: ssd
  name: host_172.16.1.5_local_storage_0
  progress: 100
  public_scope: system
  public_src: local
  reserved: 0
  source: local
  status: online
  storage_type: local
  storagecache_id: 42baacbf-e4d5-44ba-887c-b6bfdd579d36
  update_version: 61
  updated_at: "2023-10-20T01:27:28.000000Z"
  zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852

执行下：climc host-storage-list --details 再看下这个 ceph 存储有没有和相关的宿主机挂载上

chenjacken commented 10 months ago
image 
[root@master1 ~]# climc host-storage-list --details
+--------------------------------------+--------------------------------------+----------------------------------+----------------------------+----------+---------------+----------------+---------------+----------+
|               Host_ID                |              Storage_ID              |             Storage              |        Mount_point         | Capacity | Used_capacity | Waste_capacity | Free_capacity | cmtbound |
+--------------------------------------+--------------------------------------+----------------------------------+----------------------------+----------+---------------+----------------+---------------+----------+
| d2805334-011a-4ff8-876b-e4026d352a00 | 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | rbd:hwcloudpool            | 1052194  | 102400        | 0              | 949794        | 1        |
| 91018292-660b-498f-8d8e-984afcd66625 | 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | rbd:hwcloudpool            | 1052194  | 102400        | 0              | 949794        | 1        |
| 0ec8e63a-10c1-4028-897c-1942435bcd65 | 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | rbd:hwcloudpool            | 1052194  | 102400        | 0              | 949794        | 1        |
| 9e4c3c93-9605-4cc9-84ba-01c52157e78b | 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | rbd:hwcloudpool            | 1052194  | 102400        | 0              | 949794        | 1        |
| 94044e06-9464-4dab-803d-7cf000b3ccc1 | 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | rbd:hwcloudpool            | 1052194  | 102400        | 0              | 949794        | 1        |
| b877f88b-5d8c-48bd-8d03-efd269104188 | 77008084-6a7a-42c0-8687-7c8b9222160a | HWCloud-Ceph                     | rbd:hwcloudpool            | 1052194  | 102400        | 0              | 949794        | 1        |
| b877f88b-5d8c-48bd-8d03-efd269104188 | 28a37de9-b491-4a16-8628-a282bccee8a8 | host_172.16.1.16_local_storage_0 | /opt/cloud/workspace/disks | 455143   | 0             | 0              | 455143        | 1        |
| 94044e06-9464-4dab-803d-7cf000b3ccc1 | 1debb826-1d79-442e-8cf2-1dbd0cafdd60 | host_172.16.1.15_local_storage_0 | /opt/cloud/workspace/disks | 459823   | 0             | 0              | 459823        | 1        |
| d2805334-011a-4ff8-876b-e4026d352a00 | af307604-7179-4d51-8a9b-3dd68874e049 | host_172.16.1.11_local_storage_0 | /opt/cloud/workspace/disks | 471791   | 358400        | 0              | 113391        | 1        |
| 0ec8e63a-10c1-4028-897c-1942435bcd65 | 2d2cabfa-70d1-4229-8499-5fb1687e9f08 | host_172.16.1.6_local_storage_0  | /opt/cloud/workspace/disks | 471791   | 0             | 0              | 471791        | 1        |
| 9e4c3c93-9605-4cc9-84ba-01c52157e78b | c76bea60-b48b-4d49-89fa-72399efe1b06 | host_172.16.1.5_local_storage_0  | /opt/cloud/workspace/disks | 443695   | 0             | 0              | 443695        | 1        |
| 91018292-660b-498f-8d8e-984afcd66625 | 7a36954f-44aa-47eb-88a3-4bb62cc398a9 | host_172.16.1.7_local_storage_0  | /opt/cloud/workspace/disks | 443695   | 409600        | 0              | 34095         | 1        |
| fe071d8d-46d3-4266-835d-93365960ec18 | b11390f9-59db-4d9c-8608-d3f9fdb04886 | host_172.16.1.8_local_storage_0  | /opt/cloud/workspace/disks | 443695   | 0             | 0              | 443695        | 1        |
| 9eb94b52-f91c-4082-8014-3709c1920fd7 | 6153222e-d59b-46ff-8f3a-cca7a804dc6b | host_172.16.1.9_local_storage_0  | /opt/cloud/workspace/disks | 443695   | 0             | 0              | 443695        | 1        |
| 9e438931-4ce8-4ed2-83e0-3a67f8b755d1 | 293ab9ee-952a-41cf-8a8c-a6d9098d33b9 | host_172.16.1.10_local_storage_0 | /opt/cloud/workspace/disks | 471791   | 307200        | 0              | 164591        | 1        |
+--------------------------------------+--------------------------------------+----------------------------------+----------------------------+----------+---------------+----------------+---------------+----------+
***  Total: 15 Pages: 1 Limit: 20 Offset: 0 Page: 1  ***
[root@master1 ~]# 

[root@master1 ~]# climc host-list --details
+--------------------------------------+---------------------+-------------------+-------------+--------------------+--------------------------+---------+---------+-------------+--------+----------------+---------+--------------+-----------------+--------------+---------------------+----------+------------+-----------+------------+---------------------+---------------------+------------------+------------------+------------+----------+--------------+------------+----------------------------------+--------------+-----------+----------------+--------------+
|                  ID                  |        Name         |    Access_mac     |  Access_ip  | Ovn_Mapped_Ip_Addr |       Manager_URI        | Status  | enabled | host_status | Guests | Running_guests | storage | storage_used | storage_virtual | storage_free | storage_commit_rate | mem_size | mem_commit | cpu_count | cpu_commit |   cpu_commit_rate   |   mem_commit_rate   | cpu_commit_bound | mem_commit_bound | node_count |    sn    | storage_type | host_type  |             version              | storage_size | domain_id | project_domain | public_scope |
+--------------------------------------+---------------------+-------------------+-------------+--------------------+--------------------------+---------+---------+-------------+--------+----------------+---------+--------------+-----------------+--------------+---------------------+----------+------------+-----------+------------+---------------------+---------------------+------------------+------------------+------------+----------+--------------+------------+----------------------------------+--------------+-----------+----------------+--------------+
| b877f88b-5d8c-48bd-8d03-efd269104188 | node9-172-16-1-16   | 6c:0b:84:bd:e9:ed | 172.16.1.16 |                    | https://172.16.1.16:8885 | running | true    | online      | 0      | 0              | 455143  | 0            | 455143          | 455143       | 0                   | 64260    | 0          | 56        | 0          | 0                   | 0                   | 8                | 1                | 2          | PC0B9SA9 | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 455143       | default   | Default        | system       |
| 94044e06-9464-4dab-803d-7cf000b3ccc1 | node8-172-16-1-15   | 6c:0b:84:ff:d8:ff | 172.16.1.15 |                    | https://172.16.1.15:8885 | running | true    | online      | 0      | 0              | 459823  | 0            | 459823          | 459823       | 0                   | 32100    | 0          | 56        | 0          | 0                   | 0                   | 8                | 1                | 2          | PC0LADFL | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 459823       | default   | Default        | system       |
| d2805334-011a-4ff8-876b-e4026d352a00 | node7-172-16-1-11   | 6c:0b:84:11:01:4c | 172.16.1.11 |                    | https://172.16.1.11:8885 | running | true    | online      | 5      | 4              | 471791  | 358400       | 471791          | 113391       | 0.76                | 96516    | 57344      | 56        | 28         | 0.5                 | 0.7368421052631579  | 8                | 1                | 2          | PC07HBGC | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 471791       | default   | Default        | system       |
| 0ec8e63a-10c1-4028-897c-1942435bcd65 | node5-172-16-1-6    | 6c:0b:84:81:d3:08 | 172.16.1.6  | 100.64.0.132       | https://172.16.1.6:8885  | running | true    | online      | 0      | 0              | 471791  | 0            | 471791          | 471791       | 0                   | 96516    | 0          | 56        | 0          | 0                   | 0                   | 8                | 1                | 2          | PC0849AN | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 471791       | default   | Default        | system       |
| 9e4c3c93-9605-4cc9-84ba-01c52157e78b | node4-172-16-1-5    | 6c:0b:84:81:ca:7d | 172.16.1.5  |                    | https://172.16.1.5:8885  | running | true    | online      | 0      | 0              | 443695  | 0            | 443695          | 443695       | 0                   | 64260    | 0          | 56        | 0          | 0                   | 0                   | 8                | 1                | 2          | PC07L7XZ | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 443695       | default   | Default        | system       |
| 91018292-660b-498f-8d8e-984afcd66625 | node6-172-16-1-7    | 6c:0b:84:d7:1c:df | 172.16.1.7  | 100.64.0.131       | https://172.16.1.7:8885  | running | true    | online      | 3      | 3              | 443695  | 409600       | 443695          | 34095        | 0.92                | 64355    | 45056      | 56        | 14         | 0.25                | 0.88                | 8                | 1                | 2          | PC0DWDVW | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 443695       | default   | Default        | system       |
| fe071d8d-46d3-4266-835d-93365960ec18 | master1-172-16-1-8  | 6c:0b:84:81:d2:ec | 172.16.1.8  | 100.64.0.128       | https://172.16.1.8:8885  | running | false   | online      | 0      | 0              | 443695  | 0            | 443695          | 443695       | 0                   | 32052    | 0          | 56        | 0          | 0                   | 0                   | 8                | 1                | 2          | PC080MUU | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 443695       | default   | Default        | system       |
| 9eb94b52-f91c-4082-8014-3709c1920fd7 | master2-172-16-1-9  | 6c:0b:84:10:ff:08 | 172.16.1.9  | 100.64.0.129       | https://172.16.1.9:8885  | running | false   | online      | 0      | 0              | 443695  | 0            | 443695          | 443695       | 0                   | 64260    | 0          | 56        | 0          | 0                   | 0                   | 8                | 1                | 2          | PC07JGFU | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 443695       | default   | Default        | system       |
| 9e438931-4ce8-4ed2-83e0-3a67f8b755d1 | master3-172-16-1-10 | 6c:0b:84:bd:ec:cb | 172.16.1.10 | 100.64.0.130       | https://172.16.1.10:8885 | running | true    | online      | 2      | 2              | 471791  | 307200       | 471791          | 164591       | 0.65                | 128772   | 20480      | 56        | 10         | 0.17857142857142858 | 0.17292623615239125 | 8                | 1                | 2          | PC0AA8CL | ssd          | hypervisor | release/3.10(978c65dbc323101517) | 471791       | default   | Default        | system       |
+--------------------------------------+---------------------+-------------------+-------------+--------------------+--------------------------+---------+---------+-------------+--------+----------------+---------+--------------+-----------------+--------------+---------------------+----------+------------+-----------+------------+---------------------+---------------------+------------------+------------------+------------+----------+--------------+------------+----------------------------------+--------------+-----------+----------------+--------------+
***  Total: 9 Pages: 1 Limit: 20 Offset: 0 Page: 1  ***
[root@master1 ~]#
zexi commented 10 months ago

@chenjacken 麻烦把 default-scheduler-xxx 这个 pod 的日志导出来放这里看下，现在看起来是调度器没有加载关联的存储。

chenjacken commented 10 months ago

default-scheduler.log

谢谢！

zexi commented 10 months ago

default-scheduler.log

谢谢！

@chenjacken 看日志没有看到相关报错，麻烦把 default-scheduler 这个 deployment 换成镜像：registry.cn-beijing.aliyuncs.com/yunionio/scheduler:v3.10-lzx-debug-storages 后，再用之前的 climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b 看下有没有 rbd 的 storages，如果还是没有，麻烦再把新的 scheduler 日志导出来看看。

chenjacken commented 10 months ago

kubectl set image -n onecloud deployment/default-scheduler *=registry.cn-beijing.aliyuncs.com/yunionio/scheduler:v3.10-lzx-debug-storages

更换完镜像后，在web端创建虚拟机，是一样的错误。

default-schedulerr-2.log

[root@master1 ~]# climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b
access_ip: 172.16.1.5
access_mac: 6c:0b:84:81:ca:7d
auto_renew: false
billing_type: postpaid
cpu_architecture: x86_64
cpu_bound_count: 0
cpu_cache: 35840
cpu_cmtbound: 8
cpu_count: 56
cpu_desc: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
cpu_mhz: 2389
cpu_microcode: "0xb00001e"
cpu_reserved: 0
created_at: "2023-10-19T21:01:04.000000Z"
creating_cpu_count: 0
creating_guest_count: 0
creating_mem_size: 0
deleted: false
domain_id: default
domain_src: local
enable_health_check: true
enable_pxe_boot: true
enabled: true
fake_deleted_cpu_count: 0
fake_deleted_mem_size: 0
free_cpu_count: 440
free_mem_size: 43008
guest_count: 0
guest_reserved_resource:
  cpu_count: 8
  memory_size: 8192
  storage_size: 102400
guest_reserved_used:
  cpu_count: 0
  memory_size: 0
  storage_size: 0
host_status: online
host_type: hypervisor
hostname: node4-172-16-1-5
id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
imported_at: "2023-10-19T21:01:04.000000Z"
io_bound_count: 0
ipmi_info:
  present: false
is_baremetal: false
is_emulated: false
is_import: false
is_maintenance: false
is_public: true
isolated_devices:
- addr: "04:00.0"
  dev_type: GPU-HPC
  host_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
  id: fe5cf8b8-c42b-4d2d-8e06-b4d60d6f4e59
  model: GeForce RTX 3090
  vendor_device_id: 10de:2204
last_ping_at: "2023-10-20T01:28:28.000000Z"
manager_uri: https://172.16.1.5:8885
mem_cmtbound: 1
mem_reserved: 13060
mem_size: 64260
metadata:
  __auto_migrate_on_host_down: enable
  __auto_migrate_on_host_shutdown: enable
  __host_errors: '[{"content":"GPU 07:00.0 use kernel driver ast, skip it","name":"
    ","time":"2023-10-19T21:01:05.612871Z","type":"isolated_devices"}]'
  cpu_info: '{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56}'
  generate_name: node4-172-16-1-5
  hostname: node4-172-16-1-5
  hugepage_nr: "50"
  hugepage_size_kb: "1048576"
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  memory_used_mb: "54058"
  model: RD450X
  nest: enabled
  oem_name: lenovo
  on_kubernetes: "true"
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  root_partition_total_capacity_mb: "443695"
  root_partition_used_capacity_mb: "6957"
  sn: PC07L7XZ
  storage_type: ssd
  topology: '{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":68719476736,"total_usable_bytes":67381719040}}]}'
  version: 70FR0017CN
name: node4-172-16-1-5
net_interfaces:
  62d395c7-8a5c-4ea1-8741-1ccef5f521ca:
  - baremetal_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
    index: 0
    link_up: true
    mac: 6c:0b:84:81:ca:7d
    mtu: 0
    nic_type: admin
    rate: 0
    wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
networks:
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:01:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.5
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.5
  guest_ip6_mask: 0
  id: 0c973695-9738-403d-88ec-8620151137ce
  ifname_hint: adm022
  imported_at: "2023-10-19T21:01:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm022
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:33:10.000000Z"
  deleted: false
  domain_id: default
  free_port: 51
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.200
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.150
  guest_ip6_mask: 0
  id: 51b8fbd8-9b90-48e6-8369-33fc86e58717
  ifname_hint: DHCP1
  imported_at: "2023-10-19T08:33:10.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: DHCP1
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:19.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.9
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.9
  guest_ip6_mask: 0
  id: 0a596e84-89ef-4f94-8c06-20b1f71449ed
  ifname_hint: adm02
  imported_at: "2023-10-19T07:19:19.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm02
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:12.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.10
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.10
  guest_ip6_mask: 0
  id: 3a632875-3c85-4056-8b10-8d2b9d521537
  ifname_hint: adm01
  imported_at: "2023-10-19T07:19:12.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm01
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:30:40.000000Z"
  deleted: false
  description: 手动指定静态IP,自动分配请勿使用该子网。
  domain_id: default
  free_port: 15
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.90
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.70
  guest_ip6_mask: 0
  id: 7e49c4fd-0c08-49b3-8e2a-983bb8f8a0e2
  ifname_hint: HOST
  imported_at: "2023-10-19T08:30:40.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: Static
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:25:14.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.11
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.11
  guest_ip6_mask: 0
  id: 8e82abdc-7e5f-45a2-8627-0b164ed3ed77
  ifname_hint: adm024
  imported_at: "2023-10-19T21:25:14.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm024
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:16:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.6
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.6
  guest_ip6_mask: 0
  id: dab40ba9-01c7-48ad-8c16-a7ff708a36e8
  ifname_hint: adm023
  imported_at: "2023-10-19T21:16:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm023
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T14:29:25.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.7
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.7
  guest_ip6_mask: 0
  id: ff1242fc-7b3f-42e8-878d-a4dfc562d6c0
  ifname_hint: adm021
  imported_at: "2023-10-19T14:29:25.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm021
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:00.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.8
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.8
  guest_ip6_mask: 0
  id: 0b27e637-cefa-4735-86fe-201116d34fa8
  ifname_hint: adm0
  imported_at: "2023-10-19T07:19:00.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm0
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: none
  public_src: local
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
nics:
- link_up: true
  mac: 00:1b:21:bc:79:07
  masklen: 0
  mtu: 0
  rate: 0
- dns: 223.5.5.5,223.6.6.6
  gateway: 172.16.1.1
  ip_addr: 172.16.1.5
  link_up: true
  mac: 6c:0b:84:81:ca:7d
  masklen: 24
  mtu: 0
  net: adm022
  net_id: 0c973695-9738-403d-88ec-8620151137ce
  nic_type: admin
  rate: 0
  wire: bcast0
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- link_up: true
  mac: 6c:0b:84:81:ca:7e
  masklen: 0
  mtu: 0
  rate: 0
node_count: 2
ovn_version: 2.12.4-2
page_size_kb: 1048576
pending_usage:
  cpu: 0
  disk: {}
  instance_groups: {}
  isolated_device: 0
  memory: 0
  net: {}
progress: 100
public_scope: system
rack: rack0
region:
  created_at: "2023-10-19T06:12:20.000000Z"
  deleted: false
  description: Default Region
  enabled: true
  id: default
  imported_at: "2023-10-19T06:12:20.000000Z"
  is_emulated: false
  latitude: 0
  longitude: 0
  name: Default
  progress: 100
  provider: OneCloud
  source: local
  status: inservice
  update_version: 0
  updated_at: "2023-10-19T06:12:20.000000Z"
required_cpu_count: 0
required_mem_size: 0
resource_type: shared
running_cpu_count: 0
running_guest_count: 0
running_mem_size: 0
slots: slot0
sn: PC07L7XZ
source: local
status: running
storage_driver: Linux
storage_size: 443695
storage_type: ssd
storages:
- actual_capacity_used: 6957
  actual_free_capacity: 436738
  capacity: 443695
  cmtbound: 1
  created_at: "2023-10-19T21:01:06.000000Z"
  deleted: false
  domain_id: default
  domain_src: local
  enabled: true
  free_capacity: 443695
  id: c76bea60-b48b-4d49-89fa-72399efe1b06
  imported_at: "2023-10-19T21:01:06.000000Z"
  is_emulated: false
  is_public: true
  is_sys_disk_store: true
  medium_type: ssd
  name: host_172.16.1.5_local_storage_0
  progress: 100
  public_scope: system
  public_src: local
  reserved: 0
  source: local
  status: online
  storage_type: local
  storagecache_id: 42baacbf-e4d5-44ba-887c-b6bfdd579d36
  update_version: 61
  updated_at: "2023-10-20T01:27:28.000000Z"
  zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
sys_info:
  cpu_info:

...//more

        total_threads: 2
      id: 1
      model: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
      total_cores: 14
      total_threads: 28
      vendor: GenuineIntel
    total_cores: 28
    total_threads: 56
  hugepage_nr: 50
  hugepage_size_kb: 1048576
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  model: RD450X
  nest: enabled
  oem_name: lenovo
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  sn: PC07L7XZ
  storage_type: ssd
  topology:
    architecture: smp
    nodes:

...//more

      memory:
        supported_page_sizes:
        - 1073741824
        total_physical_bytes: 68719476736
        total_usable_bytes: 67381719040
  version: 70FR0017CN
total_cpu_count: 448
total_mem_size: 64260
update_version: 271
updated_at: "2023-10-20T01:28:28.000000Z"
version: release/3.10(978c65dbc323101517)
zone:
  cloudregion_id: default
  created_at: "2023-10-19T06:12:29.000000Z"
  deleted: false
  id: d7754248-9f87-4e31-81a9-517c7b0ef852
  imported_at: "2023-10-19T06:12:29.000000Z"
  is_emulated: false
  name: zone0
  progress: 100
  source: local
  status: enable
  update_version: 0
  updated_at: "2023-10-19T06:12:29.000000Z"
zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852

[root@master1 ~]#
zexi commented 10 months ago

@chenjacken 我看了日志，发现调度服务没有查询到关联的 rbd 存储，用下面的命令能查询出来吗？

climc host-storage-list --host 9e4c3c93-9605-4cc9-84ba-01c52157e78b
chenjacken commented 10 months ago

@chenjacken 我看了日志，发现调度服务没有查询到关联的 rbd 存储，用下面的命令能查询出来吗？

climc host-storage-list --host 9e4c3c93-9605-4cc9-84ba-01c52157e78b

命令是能查询到。

[root@master1 ~]# climc host-storage-list --host 9e4c3c93-9605-4cc9-84ba-01c52157e78b
+--------------------------------------+--------------------------------------+----------------------------+
|               Host_ID                |              Storage_ID              |        Mount_point         |
+--------------------------------------+--------------------------------------+----------------------------+
| 9e4c3c93-9605-4cc9-84ba-01c52157e78b | 77008084-6a7a-42c0-8687-7c8b9222160a | rbd:hwcloudpool            |
| 9e4c3c93-9605-4cc9-84ba-01c52157e78b | c76bea60-b48b-4d49-89fa-72399efe1b06 | /opt/cloud/workspace/disks |
+--------------------------------------+--------------------------------------+----------------------------+
***  Total: 2 Pages: 1 Limit: 20 Offset: 0 Page: 1  ***
[root@master1 ~]#
chenjacken commented 10 months ago

是否可以重置或者替换POD镜像来解决问题？ 或者是否数据库存在混乱的数据导致？

zexi commented 10 months ago

是否可以重置或者替换POD镜像来解决问题？ 或者是否数据库存在混乱的数据导致？

@chenjacken 这个问题看起来有点奇怪，有点像代码层面的问题，我又改了一个版本，用这个版本的调度器试试：registry.cn-beijing.aliyuncs.com/yunionio/scheduler:v3.10-lzx-debug-storages.0，然后再把日志发一份出来

chenjacken commented 10 months ago

default-scheduler-3.log

谢谢~~ climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b的日志：

[root@master1 ~]# climc scheduler-candidate-show 9e4c3c93-9605-4cc9-84ba-01c52157e78b
access_ip: 172.16.1.5
access_mac: 6c:0b:84:81:ca:7d
auto_renew: false
billing_type: postpaid
cpu_architecture: x86_64
cpu_bound_count: 0
cpu_cache: 35840
cpu_cmtbound: 8
cpu_count: 56
cpu_desc: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
cpu_mhz: 2389
cpu_microcode: "0xb00001e"
cpu_reserved: 0
created_at: "2023-10-19T21:01:04.000000Z"
creating_cpu_count: 0
creating_guest_count: 0
creating_mem_size: 0
deleted: false
domain_id: default
domain_src: local
enable_health_check: true
enable_pxe_boot: true
enabled: true
fake_deleted_cpu_count: 0
fake_deleted_mem_size: 0
free_cpu_count: 440
free_mem_size: 43008
guest_count: 0
guest_reserved_resource:
  cpu_count: 8
  memory_size: 8192
  storage_size: 102400
guest_reserved_used:
  cpu_count: 0
  memory_size: 0
  storage_size: 0
host_status: online
host_type: hypervisor
hostname: node4-172-16-1-5
id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
imported_at: "2023-10-19T21:01:04.000000Z"
io_bound_count: 0
ipmi_info:
  present: false
is_baremetal: false
is_emulated: false
is_import: false
is_maintenance: false
is_public: true
isolated_devices:
- addr: "04:00.0"
  dev_type: GPU-HPC
  host_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
  id: fe5cf8b8-c42b-4d2d-8e06-b4d60d6f4e59
  model: GeForce RTX 3090
  vendor_device_id: 10de:2204
last_ping_at: "2023-10-20T01:28:28.000000Z"
manager_uri: https://172.16.1.5:8885
mem_cmtbound: 1
mem_reserved: 13060
mem_size: 64260
metadata:
  __auto_migrate_on_host_down: enable
  __auto_migrate_on_host_shutdown: enable
  __host_errors: '[{"content":"GPU 07:00.0 use kernel driver ast, skip it","name":"
    ","time":"2023-10-19T21:01:05.612871Z","type":"isolated_devices"}]'
  cpu_info: '{"processors":[{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[0,28],"total_threads":2},{"id":0,"index":1,"logical_processors":[1,29],"total_threads":2},{"id":0,"index":11,"logical_processors":[10,38],"total_threads":2},{"id":0,"index":12,"logical_processors":[11,39],"total_threads":2},{"id":0,"index":13,"logical_processors":[12,40],"total_threads":2},{"id":0,"index":14,"logical_processors":[13,41],"total_threads":2},{"id":0,"index":2,"logical_processors":[2,30],"total_threads":2},{"id":0,"index":3,"logical_processors":[3,31],"total_threads":2},{"id":0,"index":4,"logical_processors":[32,4],"total_threads":2},{"id":0,"index":5,"logical_processors":[33,5],"total_threads":2},{"id":0,"index":6,"logical_processors":[34,6],"total_threads":2},{"id":0,"index":8,"logical_processors":[35,7],"total_threads":2},{"id":0,"index":9,"logical_processors":[36,8],"total_threads":2},{"id":0,"index":10,"logical_processors":[37,9],"total_threads":2}],"id":0,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"},{"capabilities":["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","dts","acpi","mmx","fxsr","sse","sse2","ss","ht","tm","pbe","syscall","nx","pdpe1gb","rdtscp","lm","constant_tsc","arch_perfmon","pebs","bts","rep_good","nopl","xtopology","nonstop_tsc","cpuid","aperfmperf","pni","pclmulqdq","dtes64","monitor","ds_cpl","vmx","smx","est","tm2","ssse3","sdbg","fma","cx16","xtpr","pdcm","pcid","dca","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","abm","3dnowprefetch","cpuid_fault","epb","cat_l3","cdp_l3","invpcid_single","pti","intel_ppin","tpr_shadow","vnmi","flexpriority","ept","vpid","ept_ad","fsgsbase","tsc_adjust","bmi1","hle","avx2","smep","bmi2","erms","invpcid","rtm","cqm","rdt_a","rdseed","adx","smap","intel_pt","xsaveopt","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","dtherm","arat","pln","pts"],"cores":[{"id":0,"index":0,"logical_processors":[14,42],"total_threads":2},{"id":0,"index":1,"logical_processors":[15,43],"total_threads":2},{"id":0,"index":2,"logical_processors":[16,44],"total_threads":2},{"id":0,"index":3,"logical_processors":[17,45],"total_threads":2},{"id":0,"index":4,"logical_processors":[18,46],"total_threads":2},{"id":0,"index":5,"logical_processors":[19,47],"total_threads":2},{"id":0,"index":6,"logical_processors":[20,48],"total_threads":2},{"id":0,"index":8,"logical_processors":[21,49],"total_threads":2},{"id":0,"index":9,"logical_processors":[22,50],"total_threads":2},{"id":0,"index":10,"logical_processors":[23,51],"total_threads":2},{"id":0,"index":11,"logical_processors":[24,52],"total_threads":2},{"id":0,"index":12,"logical_processors":[25,53],"total_threads":2},{"id":0,"index":13,"logical_processors":[26,54],"total_threads":2},{"id":0,"index":14,"logical_processors":[27,55],"total_threads":2}],"id":1,"model":"Intel(R)
    Xeon(R) CPU E5-2680 v4 @ 2.40GHz","total_cores":14,"total_threads":28,"vendor":"GenuineIntel"}],"total_cores":28,"total_threads":56}'
  generate_name: node4-172-16-1-5
  hostname: node4-172-16-1-5
  hugepage_nr: "50"
  hugepage_size_kb: "1048576"
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  memory_used_mb: "54058"
  model: RD450X
  nest: enabled
  oem_name: lenovo
  on_kubernetes: "true"
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  root_partition_total_capacity_mb: "443695"
  root_partition_used_capacity_mb: "6957"
  sn: PC07L7XZ
  storage_type: ssd
  topology: '{"architecture":"smp","nodes":[{"caches":[{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"instruction"},{"level":1,"logical_processors":[0,28],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[1,29],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[2,30],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[3,31],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[4,32],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[5,33],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[6,34],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[7,35],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[8,36],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[9,37],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[10,38],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[11,39],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[12,40],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[13,41],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[14,42],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[15,43],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[16,44],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[17,45],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[18,46],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[19,47],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[20,48],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[21,49],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[22,50],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[23,51],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[24,52],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[25,53],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[26,54],"size_bytes":32768,"type":"data"},{"level":1,"logical_processors":[27,55],"size_bytes":32768,"type":"data"},{"level":2,"logical_processors":[0,28],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[1,29],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[2,30],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[3,31],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[4,32],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[5,33],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[6,34],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[7,35],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[8,36],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[9,37],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[10,38],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[11,39],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[12,40],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[13,41],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[14,42],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[15,43],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[16,44],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[17,45],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[18,46],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[19,47],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[20,48],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[21,49],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[22,50],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[23,51],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[24,52],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[25,53],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[26,54],"size_bytes":262144,"type":"unified"},{"level":2,"logical_processors":[27,55],"size_bytes":262144,"type":"unified"},{"level":3,"logical_processors":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"size_bytes":36700160,"type":"unified"},{"level":3,"logical_processors":[14,15,16,17,18,19,20,21,22,23,24,25,26,27,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"size_bytes":36700160,"type":"unified"}],"cores":[{"id":0,"index":0,"logical_processors":[0,14,28,42],"total_threads":4},{"id":1,"index":1,"logical_processors":[1,15,29,43],"total_threads":4},{"id":11,"index":2,"logical_processors":[10,24,38,52],"total_threads":4},{"id":12,"index":3,"logical_processors":[11,25,39,53],"total_threads":4},{"id":13,"index":4,"logical_processors":[12,26,40,54],"total_threads":4},{"id":14,"index":5,"logical_processors":[13,27,41,55],"total_threads":4},{"id":2,"index":6,"logical_processors":[16,2,30,44],"total_threads":4},{"id":3,"index":7,"logical_processors":[17,3,31,45],"total_threads":4},{"id":4,"index":8,"logical_processors":[18,32,4,46],"total_threads":4},{"id":5,"index":9,"logical_processors":[19,33,47,5],"total_threads":4},{"id":6,"index":10,"logical_processors":[20,34,48,6],"total_threads":4},{"id":8,"index":11,"logical_processors":[21,35,49,7],"total_threads":4},{"id":9,"index":12,"logical_processors":[22,36,50,8],"total_threads":4},{"id":10,"index":13,"logical_processors":[23,37,51,9],"total_threads":4}],"distances":[10],"id":0,"memory":{"supported_page_sizes":[1073741824],"total_physical_bytes":68719476736,"total_usable_bytes":67381719040}}]}'
  version: 70FR0017CN
name: node4-172-16-1-5
net_interfaces:
  62d395c7-8a5c-4ea1-8741-1ccef5f521ca:
  - baremetal_id: 9e4c3c93-9605-4cc9-84ba-01c52157e78b
    index: 0
    link_up: true
    mac: 6c:0b:84:81:ca:7d
    mtu: 0
    nic_type: admin
    rate: 0
    wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
networks:
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:00.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.8
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.8
  guest_ip6_mask: 0
  id: 0b27e637-cefa-4735-86fe-201116d34fa8
  ifname_hint: adm0
  imported_at: "2023-10-19T07:19:00.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm0
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: none
  public_src: local
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:12.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.10
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.10
  guest_ip6_mask: 0
  id: 3a632875-3c85-4056-8b10-8d2b9d521537
  ifname_hint: adm01
  imported_at: "2023-10-19T07:19:12.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm01
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:25:14.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.11
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.11
  guest_ip6_mask: 0
  id: 8e82abdc-7e5f-45a2-8627-0b164ed3ed77
  ifname_hint: adm024
  imported_at: "2023-10-19T21:25:14.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm024
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:30:40.000000Z"
  deleted: false
  description: 手动指定静态IP,自动分配请勿使用该子网。
  domain_id: default
  free_port: 15
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.90
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.70
  guest_ip6_mask: 0
  id: 7e49c4fd-0c08-49b3-8e2a-983bb8f8a0e2
  ifname_hint: HOST
  imported_at: "2023-10-19T08:30:40.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: Static
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:16:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.6
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.6
  guest_ip6_mask: 0
  id: dab40ba9-01c7-48ad-8c16-a7ff708a36e8
  ifname_hint: adm023
  imported_at: "2023-10-19T21:16:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm023
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T14:29:25.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.7
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.7
  guest_ip6_mask: 0
  id: ff1242fc-7b3f-42e8-878d-a4dfc562d6c0
  ifname_hint: adm021
  imported_at: "2023-10-19T14:29:25.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm021
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:35.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T07:19:19.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.9
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.9
  guest_ip6_mask: 0
  id: 0a596e84-89ef-4f94-8c06-20b1f71449ed
  ifname_hint: adm02
  imported_at: "2023-10-19T07:19:19.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm02
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T07:50:41.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_timout_seconds: 0
  created_at: "2023-10-19T21:01:03.000000Z"
  deleted: false
  domain_id: default
  free_port: 0
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.5
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.5
  guest_ip6_mask: 0
  id: 0c973695-9738-403d-88ec-8620151137ce
  ifname_hint: adm022
  imported_at: "2023-10-19T21:01:03.000000Z"
  is_auto_alloc: false
  is_emulated: false
  is_public: false
  is_system: false
  name: adm022
  pending_deleted: false
  progress: 0
  public_scope: system
  server_type: baremetal
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 3
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- alloc_policy: none
  alloc_timout_seconds: 0
  created_at: "2023-10-19T08:33:10.000000Z"
  deleted: false
  domain_id: default
  free_port: 51
  freezed: false
  guest_gateway: 172.16.1.1
  guest_ip_end: 172.16.1.200
  guest_ip_mask: 24
  guest_ip_start: 172.16.1.150
  guest_ip6_mask: 0
  id: 51b8fbd8-9b90-48e6-8369-33fc86e58717
  ifname_hint: DHCP1
  imported_at: "2023-10-19T08:33:10.000000Z"
  is_auto_alloc: true
  is_emulated: false
  is_public: true
  is_system: false
  name: DHCP1
  pending_deleted: false
  progress: 0
  project_src: local
  public_scope: system
  public_src: local
  server_type: guest
  source: local
  status: available
  tenant_id: 43d375dfbfc1426d829bf8e28b8c3889
  update_version: 2
  updated_at: "2023-10-19T21:36:34.000000Z"
  vlan_id: 1
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
nics:
- link_up: true
  mac: 00:1b:21:bc:79:07
  masklen: 0
  mtu: 0
  rate: 0
- dns: 223.5.5.5,223.6.6.6
  gateway: 172.16.1.1
  ip_addr: 172.16.1.5
  link_up: true
  mac: 6c:0b:84:81:ca:7d
  masklen: 24
  mtu: 0
  net: adm022
  net_id: 0c973695-9738-403d-88ec-8620151137ce
  nic_type: admin
  rate: 0
  wire: bcast0
  wire_id: 62d395c7-8a5c-4ea1-8741-1ccef5f521ca
- link_up: true
  mac: 6c:0b:84:81:ca:7e
  masklen: 0
  mtu: 0
  rate: 0
node_count: 2
ovn_version: 2.12.4-2
page_size_kb: 1048576
pending_usage:
  cpu: 0
  disk: {}
  instance_groups: {}
  isolated_device: 0
  memory: 0
  net: {}
progress: 100
public_scope: system
rack: rack0
region:
  created_at: "2023-10-19T06:12:20.000000Z"
  deleted: false
  description: Default Region
  enabled: true
  id: default
  imported_at: "2023-10-19T06:12:20.000000Z"
  is_emulated: false
  latitude: 0
  longitude: 0
  name: Default
  progress: 100
  provider: OneCloud
  source: local
  status: inservice
  update_version: 0
  updated_at: "2023-10-19T06:12:20.000000Z"
required_cpu_count: 0
required_mem_size: 0
resource_type: shared
running_cpu_count: 0
running_guest_count: 0
running_mem_size: 0
slots: slot0
sn: PC07L7XZ
source: local
status: running
storage_driver: Linux
storage_size: 443695
storage_type: ssd
storages:
- actual_capacity_used: 6957
  actual_free_capacity: 436738
  capacity: 443695
  cmtbound: 1
  created_at: "2023-10-19T21:01:06.000000Z"
  deleted: false
  domain_id: default
  domain_src: local
  enabled: true
  free_capacity: 443695
  id: c76bea60-b48b-4d49-89fa-72399efe1b06
  imported_at: "2023-10-19T21:01:06.000000Z"
  is_emulated: false
  is_public: true
  is_sys_disk_store: true
  medium_type: ssd
  name: host_172.16.1.5_local_storage_0
  progress: 100
  public_scope: system
  public_src: local
  reserved: 0
  source: local
  status: online
  storage_type: local
  storagecache_id: 42baacbf-e4d5-44ba-887c-b6bfdd579d36
  update_version: 61
  updated_at: "2023-10-20T01:27:28.000000Z"
  zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852
sys_info:

...//more

      id: 1
      model: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
      total_cores: 14
      total_threads: 28
      vendor: GenuineIntel
    total_cores: 28
    total_threads: 56
  hugepage_nr: 50
  hugepage_size_kb: 1048576
  hugepages_option: native
  kernel_version: 5.4.130-1.yn20230805.el7.x86_64
  kvm_module: kvm-intel
  manufacture: Lenovo
  model: RD450X
  nest: enabled
  oem_name: lenovo
  os_distribution: CentOS Linux
  os_version: 7.9.2009
  ovs_version: 2.12.4
  qemu_version: 4.2.0
  sn: PC07L7XZ
  storage_type: ssd
  topology:
    architecture: smp
    nodes:

...//more

      memory:
        supported_page_sizes:
        - 1073741824
        total_physical_bytes: 68719476736
        total_usable_bytes: 67381719040
  version: 70FR0017CN
total_cpu_count: 448
total_mem_size: 64260
update_version: 271
updated_at: "2023-10-20T01:28:28.000000Z"
version: release/3.10(978c65dbc323101517)
zone:
  cloudregion_id: default
  created_at: "2023-10-19T06:12:29.000000Z"
  deleted: false
  id: d7754248-9f87-4e31-81a9-517c7b0ef852
  imported_at: "2023-10-19T06:12:29.000000Z"
  is_emulated: false
  name: zone0
  progress: 100
  source: local
  status: enable
  update_version: 0
  updated_at: "2023-10-19T06:12:29.000000Z"
zone_id: d7754248-9f87-4e31-81a9-517c7b0ef852

[root@master1 ~]#
chenjacken commented 10 months ago
image
zexi commented 10 months ago

@chenjacken 这个问题看起来是调度器scheduler查询到的数据和控制服务region不一致导致的，应该不是代码问题了。 你环境的数据库有没有做高可用同步？会不会是数据库两边数据不一致了？

chenjacken commented 10 months ago

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

zexi commented 10 months ago

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

@chenjacken 那应该是数据库的问题了，之前也遇到过数据不一致的问题，按照这个文档排查下：https://www.cloudpods.org/zh/docs/ops/databases/mysql_ha_recovery/

chenjacken commented 10 months ago

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

@chenjacken 那应该是数据库的问题了，之前也遇到过数据不一致的问题，按照这个文档排查下：https://www.cloudpods.org/zh/docs/ops/databases/mysql_ha_recovery/

谢谢。我排查下数据库的问题。

Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; 
Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; 
the event's master log mysql-bin.000003, end_log_pos 1031126226

1，数据库1节点：

MariaDB [(none)]> SHOW MASTER STATUS\G
*************************** 1. row ***************************
            File: mysql-bin.000008
        Position: 1004954600
    Binlog_Do_DB: 
Binlog_Ignore_DB: 
1 row in set (0.00 sec)

MariaDB [(none)]> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 172.16.1.10
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 2
              Master_Log_File: mysql-bin.000004
          Read_Master_Log_Pos: 84173703
               Relay_Log_File: mariadb-relay-bin.000008
                Relay_Log_Pos: 114629023
        Relay_Master_Log_File: mysql-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: information_schema,performance_schema
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 1062
                   Last_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1031126226
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1031124771
              Relay_Log_Space: 241383704
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 1062
               Last_SQL_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1031126226
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
1 row in set (0.00 sec)

MariaDB [(none)]>

1，数据库2节点：

MariaDB [(none)]> SHOW MASTER STATUS\G
*************************** 1. row ***************************
            File: mysql-bin.000004
        Position: 84162184
    Binlog_Do_DB: 
Binlog_Ignore_DB: 
1 row in set (0.00 sec)

MariaDB [(none)]> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 172.16.1.9
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 2
              Master_Log_File: mysql-bin.000008
          Read_Master_Log_Pos: 1004862639
               Relay_Log_File: mariadb-relay-bin.000008
                Relay_Log_Pos: 917050651
        Relay_Master_Log_File: mysql-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: information_schema,performance_schema
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 1062
                   Last_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1030599783
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1030598328
              Relay_Log_Space: 6260191426
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 1062
               Last_SQL_Error: Could not execute Write_rows event on table yunioncloud.cachedimages_tbl; Duplicate entry '9f47726b-aff1-4ba9-86c3-61940e7a2461' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000003, end_log_pos 1030599783
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
1 row in set (0.00 sec)

MariaDB [(none)]>
chenjacken commented 10 months ago

数据库是做了高可用，2个主数据库。根据 https://www.cloudpods.org/zh/docs/setup/db-ha/ 指引来部署的。

@chenjacken 那应该是数据库的问题了，之前也遇到过数据不一致的问题，按照这个文档排查下：https://www.cloudpods.org/zh/docs/ops/databases/mysql_ha_recovery/

!!!!问题是数据库双主高可用时候，主主数据同步出现问题，目前已经解决了。

非常感谢！！！！！！ @zexi