Closed hpcpony closed 1 month ago
@hpcpony OpenVINO is the library of one application process.
May I know what's the use case and platform information in your requirement?
I think pretty much any use case and any (linux) platform that has more than 1024 cores/threads will suffice, but if you need to see an example:
$ cat prop.py
import openvino as ov
core = ov.Core()
print(core.get_property("CPU","SUPPORTED_PROPERTIES"))
$ python prop.py
Traceback (most recent call last):
File "/home/hpcpony/openvino/prop.py", line 4, in <module>
print(core.get_property("CPU","SUPPORTED_PROPERTIES"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Exception from src/inference/src/cpp/core.cpp:214:
Exception from src/inference/src/dev/core_impl.cpp:708:
Failed to create plugin /opt/openvino-2024.2.0/lib/openvino-2024.2.0/libopenvino_intel_cpu_plugin.so for device CPU
Please, check your environment
Exception from src/plugins/intel_cpu/src/plugin.cpp:633:
Exception from src/inference/src/os/lin/lin_system_conf.cpp:276:
CPU affinity check failed. No CPU is eligible to run inference.
Platform was an HPE Superdome Flex that I had a chance to try out.
If it matters, this was OpenVINO 2024.2 but I just looked at the source on GitHub and at least one of the places (src/os/lin/lin_system_conf.cpp (check_valid_cpu)) is still non-dynamic.
@hpcpony Could you share the logs of lscpu
and lscpu -e
on your platform?
@hpcpony Could you try this PR26303 on your machine, thank you!
@hpcpony PR26303 has been updated, please take a look, thank you!
I'm not sure exactly what copy of your fixes I got (my git skills are pretty limited), but I was able to build a CPU only version of OpenVino
$ git log
commit 51e39eea83b997b2293a3f43b6f5a9e5c826edc1 (HEAD -> xiaoxia/cpu_set_1024, origin/xiaoxia/cpu_set_1024)
Author: sunxiaoxia2022 <xiaoxia.sun@intel.com>
Date: Tue Sep 3 00:06:52 2024 +0800
change name
commit e3834af754bc672342c58c77da0fac2bf2c10e20
Author: sunxiaoxia2022 <xiaoxia.sun@intel.com>
Date: Tue Sep 3 00:04:56 2024 +0800
remove get_affinity
commit 89d9e51f7414bb9609209ea50e1243ac70b41489
Author: sunxiaoxia2022 <xiaoxia.sun@intel.com>
Date: Fri Aug 30 03:37:44 2024 +0800
support machines with more than 1024 cores
.
.
.
Then running the little program that failed before now seems to work.
$ python prop.py
{'SUPPORTED_PROPERTIES': 'RO', 'AVAILABLE_DEVICES': 'RO', 'RANGE_FOR_ASYNC_INFER_REQUESTS': 'RO', 'RANGE_FOR_STREAMS': 'RO', 'EXECUTION_DEVICES': 'RO', 'FULL_DEVICE_NAME': 'RO', 'OPTIMIZATION_CAPABILITIES': 'RO', 'DEVICE_TYPE': 'RO', 'DEVICE_ARCHITECTURE': 'RO', 'NUM_STREAMS': 'RW', 'INFERENCE_NUM_THREADS': 'RW', 'PERF_COUNT': 'RW', 'INFERENCE_PRECISION_HINT': 'RW', 'PERFORMANCE_HINT': 'RW', 'EXECUTION_MODE_HINT': 'RW', 'PERFORMANCE_HINT_NUM_REQUESTS': 'RW', 'ENABLE_CPU_PINNING': 'RW', 'SCHEDULING_CORE_TYPE': 'RW', 'MODEL_DISTRIBUTION_POLICY': 'RW', 'ENABLE_HYPER_THREADING': 'RW', 'DEVICE_ID': 'RW', 'CPU_DENORMALS_OPTIMIZATION': 'RW', 'LOG_LEVEL': 'RW', 'CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE': 'RW', 'DYNAMIC_QUANTIZATION_GROUP_SIZE': 'RW', 'KV_CACHE_PRECISION': 'RW', 'AFFINITY': 'RW'}
Thanks.
Request Description
2024.3 does not consistently handle large numbers of core w.r.t. cpu_set. thread_affinity.cpp does use dynamically sized cpu sets, but other source files (lin_system_conf.cpp, system_conf.cpp) do not.
Feature Use Case
Would support use on systems that have large (>1024) cores.
Issue submission checklist