pyopencl does not respect OCL_ICD_FILENAMES

inducer / pyopencl

OpenCL integration for Python, plus shiny features

http://mathema.tician.de/software/pyopencl

Other

1.07k stars 242 forks source link

pyopencl does not respect OCL_ICD_FILENAMES #797

Open bjourne opened 2 weeks ago

bjourne commented 2 weeks ago

MWE:

$ python -c "import pyopencl; print(pyopencl.get_platforms())"                                                                                                                                                               
[<pyopencl.Platform 'NVIDIA CUDA' at 0x5b7c78be31c0>]
$ clinfo -l                                                                                                                                                                                                                  
Platform #0: Intel(R) FPGA Emulation Platform for OpenCL(TM)
 `-- Device #0: Intel(R) FPGA Emulation Device
Platform #1: Intel(R) OpenCL
 `-- Device #0: Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz
Platform #2: NVIDIA CUDA
 `-- Device #0: Quadro P400

So pyopencl does not find platforms whose shared objects are found in the OCL_ICD_FILENAMES environment variable. I'm not sure why since get_platforms simply wraps clGetPlatformIDs, doesn't it?

bjourne commented 2 weeks ago

Apparently this is because of the ICD loader pyopencl uses. Khronos own ICD loader supports environment variables but ocl-icd does not: https://github.com/KhronosGroup/OpenCL-ICD-Loader Looking at repo activity it seems ocl-icd doesn't get many updates.

inducer commented 1 week ago

I'm confused here.

PyOpenCL is not specific to any ICD loader, it works with Khronos and other loaders. The wheel also does not ship a specific loader.
For user convenience, the wheels do include an ICD loader: that's ocl-icd on Windows and Mac, and Khronos on Windows. You can always build from source and avoid this.

If you need a Linux/Mac wheel with Khronos, I'm happy to consider PRs that either replace ocl-icd or offer Khronos as an alternative.

bjourne commented 1 week ago

Ah, ok. I think I understand what is going on. The pip package puts a libOpenCL.so file in a ".libs" directory which shadows the system-installed libOpenCL.so. This libOpenCL.so comes from ocl-icd and not from Khronos official ICD loader. Shadowing system libraries is always dubious. Honestly, on Linux not including any ICD loader would be better since the user likely already has one installed. And if they don't it's just one "apt-get install ocl-icd" away. If not that then swapping ocl-icd with Kronos' ICD loader would be good. It appears to be more feature-complete and better maintained.