import gputools problem

[KoenVisscher]

python 2.7, Anaconda. There appears a problem when import gputools (pyopencl appears to work fine, at least upon running the example benchmark.py code). It appears some image3d_t image2d_t issue. Any insights very much appreciated.

Error below

WARNING:gputools.core.ocldevice | prefered platform/device (1/1) not available (device type = 4) ...choosing the best from the rest <pyopencl.Device 'Quadro 600' on 'NVIDIA CUDA' at 0x4b8400> Traceback (most recent call last): File "C:\Users\kvisscher\My Documents\LiClipse Workspace\MyFirstPythonProject\benchmark.py", line 10, in import gputools File "C:\Users\kvisscher\AppData\Local\Continuum\Anaconda2\lib\site-packages\gputools__init.py", line 16, in from gputools.core.ocltypes import OCLArray, OCLImage File "C:\Users\kvisscher\AppData\Local\Continuum\Anaconda2\lib\site-packages\gputools\core\ocltypes.py", line 330, in OCLArray = _wrap_OCLArray(cl_array.Array) File "C:\Users\kvisscher\AppData\Local\Continuum\Anaconda2\lib\site-packages\gputools\core\ocltypes.py", line 121, in _wrap_OCLArray cls._resample_prog = OCLProgram(abspath("kernels/copy_resampled.cl")) File "C:\Users\kvisscher\AppData\Local\Continuum\Anaconda2\lib\site-packages\gputools\core\oclprogram.py", line 40, in init self.build(options = build_options) File "C:\Users\kvisscher\AppData\Local\Continuum\Anaconda2\lib\site-packages\pyopencl__init__.py", line 440, in build options_bytes=options_bytes, source=self._source) File "C:\Users\kvisscher\AppData\Local\Continuum\Anaconda2\lib\site-packages\pyopencl\init__.py", line 475, in _build_and_catch_errors raise err pyopencl.cffi_cl.RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE -

Build on <pyopencl.Device 'Quadro 600' on 'NVIDIA CUDA' at 0x4b8400>:

:114:16: error: passing '__attribute__((address_space(16776963))) image3d_t' to parameter of incompatible type 'image2d_t' write_imagef(dest,(int4)(i,j,k,0),val); ^~~~ cl_kernel.h:18597:46: note: passing argument to parameter 'image' here void __OVERLOADABLE__ write_imagef(image2d_t image, int2 coord, float4 val); ^ (options: -I "c:\users\kvisscher\appdata\local\continuum\anaconda2\lib\site-packages\pyopencl\cl") (source saved as c:\users\kvissc~1\appdata\local\temp\tmpyyvnob.cl)

[maweigert]

Hi, You're graphics card is probably missing the opencl extension that enables writing 3d textures, which might happen for older cards (Quadro 600 is from 2010, which is ancient for GPUs ;).

Could you check the available extensions via the following?

import pyopencl as cl
ctx = cl.create_some_context()  # here choose your platform and device
print ctx.devices[0].get_info(cl.device_info.EXTENSIONS)

if cl_khr_3d_image_writes is not among the extensions, you might be out of luck :(

[VolkerH]

Hi Martin, just trying to install spimagine on a new Laptop (Windows 10 with core i7 and Intel UHD 620) . Installation through conda from the Talley Lamber channel. Unfortunaly I get build errors somewhat similar to the ones mentioned above. I will paste the error message at the end.

The issue is that the cl_khr_3d_image_writes OpenCL extension is not enabled. However, various internet sources and your code snippet above indicate that Intel 620 supports this extension. However, the error message indicates that the extension needs to be activated via a pragma command

#pragma OPENCL EXTENSION cl_khr_3d_image_writes : enable

I tried various versions of pyopencl from conda as well as pip-installing the current binary wheel from Chris Gohlke but I keep getting the same error message. I'm not sure whether this #pragma to activate the extension is missing in pyopencl or whether it should be added to the gputools .cl files? Finally here is the error message:

(spimenv) C:\Users\Volker\Dropbox\Github\gputools>ipython
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import gputools
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-b134f1e90096> in <module>()
----> 1 import gputools

~\Dropbox\Github\gputools\gputools\__init__.py in <module>()
     14
     15
---> 16 from gputools.core.ocltypes import OCLArray, OCLImage
     17 from gputools.core.oclprogram import OCLProgram
     18

~\Dropbox\Github\gputools\gputools\core\ocltypes.py in <module>()
    336
    337
--> 338 OCLArray = _wrap_OCLArray(cl_array.Array)
    339 OCLImage = _wrap_OCLImage(cl.Image)
    340

~\Dropbox\Github\gputools\gputools\core\ocltypes.py in _wrap_OCLArray(cls)
    123     cls.write_array = write_array
    124
--> 125     cls._resample_prog = OCLProgram(abspath("kernels/copy_resampled.cl"))
    126
    127     for f in ["sum","max","min","dot","vdot"]:

~\Dropbox\Github\gputools\gputools\core\oclprogram.py in __init__(self, file_name, src_str, build_options, dev)
     38         self._kernel_dict = {}
     39         super(OCLProgram,self).__init__(self._dev.context,src_str)
---> 40         self.build(options = build_options)
     41
     42     def run_kernel(self, name, global_size, local_size,*args,**kwargs):

~\Anaconda3\lib\site-packages\pyopencl\__init__.py in build(self, options, devices, cache_dir)
    460                         self._context, self._source, options_bytes, devices,
    461                         cache_dir=cache_dir, include_path=include_path),
--> 462                     options_bytes=options_bytes, source=self._source)
    463
    464             if was_cached:

~\Anaconda3\lib\site-packages\pyopencl\__init__.py in _build_and_catch_errors(self, build_func, options_bytes, source)
    504         # Python 3.2 outputs the whole list of currently active exceptions
    505         # This serves to remove one (redundant) level from that nesting.
--> 506         raise err
    507
    508     # }}}

RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE -

Build on <pyopencl.Device 'Intel(R) UHD Graphics 620' on 'Intel(R) OpenCL' at 0x27fa6bc1ca0>:

1:98:21: error: use of type '__write_only image3d_t' requires cl_khr_3d_image_writes extension to be enabled
                                                 __write_only image3d_t dest){
                                                              ^

(options: -I "c:\users\volker\anaconda3\lib\site-packages\pyopencl\cl")
(source saved as C:\Users\Volker\AppData\Local\Temp\tmpmfim7j9p.cl)

[maweigert]

Hi Volker,

Yes, it very much could be that in this case the extension needs to be enabled in the kernel. As I haven't access to a testing machine with an integrated GPU, could you try if the following little script runs without error (Its a simple kernel with the pragma enabled)?

https://gist.github.com/maweigert/6c0be17e82b2eb0b2faf047282ffb975

If it does, I would than enable that in the gputools kernel and everything should work for you afterwards. Thanks!

[VolkerH]

Hi Martin,

thanks for looking into this. Unfortunately, even this little code snippet produces the error message. It appears the pragma doesn't enable the extension. I did install what I believe are the latest drivers from intel but their website isn't so clear.

Just for completeness, here is the error message from the test snippet (although it really doesn't differ much from the one above).

(spimenv) C:\Users\Volker\Downloads>ipython
Python 3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 11:48:23) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %paste
src ="""
#pragma OPENCL EXTENSION cl_khr_3d_image_writes : enable
__kernel  void img3d_to_img(__read_only image3d_t src,
                                                 __write_only image3d_t dest){}
"""

from gputools import OCLProgram

prog = OCLProgram(src_str = src)

## -- End pasted text --
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-c0f406e697f2> in <module>()
      5 """
      6
----> 7 from gputools import OCLProgram
      8
      9 prog = OCLProgram(src_str = src)

c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\gputools\__init__.py in <module>()
     14
     15
---> 16 from gputools.core.ocltypes import OCLArray, OCLImage
     17 from gputools.core.oclprogram import OCLProgram
     18

c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\gputools\core\ocltypes.py in <module>()
    336
    337
--> 338 OCLArray = _wrap_OCLArray(cl_array.Array)
    339 OCLImage = _wrap_OCLImage(cl.Image)
    340

c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\gputools\core\ocltypes.py in _wrap_OCLArray(cls)
    123     cls.write_array = write_array
    124
--> 125     cls._resample_prog = OCLProgram(abspath("kernels/copy_resampled.cl"))
    126
    127     for f in ["sum","max","min","dot","vdot"]:

c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\gputools\core\oclprogram.py in __init__(self, file_name, src_str, build_options, dev)
     38         self._kernel_dict = {}
     39         super(OCLProgram,self).__init__(self._dev.context,src_str)
---> 40         self.build(options = build_options)
     41
     42     def run_kernel(self, name, global_size, local_size,*args,**kwargs):

c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\pyopencl\__init__.py in build(self, options, devices, cache_dir)
    460                         self._context, self._source, options_bytes, devices,
    461                         cache_dir=cache_dir, include_path=include_path),
--> 462                     options_bytes=options_bytes, source=self._source)
    463
    464             if was_cached:

c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\pyopencl\__init__.py in _build_and_catch_errors(self, build_func, options_bytes, source)
    504         # Python 3.2 outputs the whole list of currently active exceptions
    505         # This serves to remove one (redundant) level from that nesting.
--> 506         raise err
    507
    508     # }}}

RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE -

Build on <pyopencl.Device 'Intel(R) UHD Graphics 620' on 'Intel(R) OpenCL' at 0x26d18d29480>:

1:98:21: error: use of type '__write_only image3d_t' requires cl_khr_3d_image_writes extension to be enabled
                                                 __write_only image3d_t dest){
                                                              ^

(options: -I "c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\pyopencl\cl")
(source saved as C:\Users\Volker\AppData\Local\Temp\tmp7a4hoh5b.cl)

[maweigert]

Hi,

Yes, the error is the same because of my stupidity (RuntimeBrainError) - of course using gputools to compile a program will trigger the error irrespective of the kernel to be compiled. Facepalm :)

I updated the gist (using only pyopencl), could you try it again?

[VolkerH]

No error message when not importing gputools:

(spimenv) C:\Users\Volker\Downloads>ipython
Python 3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 11:48:23) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: src ="""
   ...: #pragma OPENCL EXTENSION cl_khr_3d_image_writes : enable
   ...: __kernel  void img3d_to_img(__read_only image3d_t src,
   ...:                          __write_only image3d_t dest){}
   ...: """
   ...:
   ...: import pyopencl as cl
   ...:
   ...: ctx = cl.create_some_context()
   ...:
   ...: prog = cl.Program(ctx,src).build()
   ...:
   ...:
Choose platform:
[0] <pyopencl.Platform 'Intel(R) OpenCL' at 0x2ba7af7cc30>
Choice [0]:
Choose device(s):
[0] <pyopencl.Device 'Intel(R) UHD Graphics 620' on 'Intel(R) OpenCL' at 0x2ba7c217cf0>
[1] <pyopencl.Device 'Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz' on 'Intel(R) OpenCL' at 0x2ba7bfdbbb0>
Choice, comma-separated [0]:0
Set the environment variable PYOPENCL_CTX=':0' to avoid being asked again.

it takes a little while and I'm back at the prompt.

[VolkerH]

The ipython prompt to be specific, so I assume it compiled just fine.

[VolkerH]

I had actually tried to put the pragma statement in each of the .cl files in the gputools sources and build spimagine like that. However I still got the same error. Maybe I missed one. Not knowing opencl I might have done something wrong though.

[maweigert]

Ok, I updated the gist again - could you try once more?

[VolkerH]

Seems fine. No error message:

(spimenv) C:\Users\Volker\Downloads>ipython
Python 3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 11:48:23) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %paste

src ="""
#pragma OPENCL EXTENSION cl_khr_3d_image_writes : enable
__kernel  void img3d_to_img(__read_only image3d_t src,
                                                 __write_only image3d_t dest){

  uint i = get_global_id(0);
  uint j = get_global_id(1);
  uint k = get_global_id(2);

  write_imagef(dest,(int4)(i,j,k,0),0.f);

                                                 }
"""

import pyopencl as cl

ctx = cl.create_some_context()

prog = cl.Program(ctx,src).build()

## -- End pasted text --
Choose platform:
[0] <pyopencl.Platform 'Intel(R) OpenCL' at 0x2d0a5bce580>
Choice [0]:
Choose device(s):
[0] <pyopencl.Device 'Intel(R) UHD Graphics 620' on 'Intel(R) OpenCL' at 0x2d0a5982050>
[1] <pyopencl.Device 'Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz' on 'Intel(R) OpenCL' at 0x2d0a6af9b70>
Choice, comma-separated [0]:
Set the environment variable PYOPENCL_CTX=':' to avoid being asked again.

In [2]:

[VolkerH]

What I don't understand: gputools works on some graphics cards without issue, but the opencl documentation mentions that the extension must be enabled using the pragma. Would some OpenCL GPU drivers enable the extension by default?

[maweigert]

Could you try to reinstall the current develop branch of gputools in your conda environment? e.g.

activate spimenv
pip uninstall gputools
pip install -I --no-deps git+https://github.com/maweigert/gputools@develop

[maweigert]

Would some OpenCL GPU drivers enable the extension by default?

Might be - I only encountered the missing pragma error on super old gpus so far and didn't really bothered ;)

[maweigert]

Seems like it worked that way - and thanks for the PR! :)

[VolkerH]

all good :-) Using the pragma finally everything works for me with pyopencl 2017.2 (not 2018.2)

Thanks again

[maweigert]

The issue seems to be solved, closing now (please reopen if it is still relevant).

[tlambert03]

sorry to reopen this... I'm trying to get my spimagine and gputools conda builds updated so I can update LLSpy. Everything is fine on mac and linux, but I'm (sort of) confused/struggling again with windows (Titan X Pascal, 418.81 driver). This more of a pyopencl / hardware issue, not a gputools issue. but just wondering if either of you guys have any thoughts:

Basically, with either pyopencl 2018.2 or 2017.2 (which worked for @VolkerH on his integrated GPU) , I can't import gputools or pyopencl:

C:\Users\CBMF>conda list cl
# packages in environment at C:\Users\CBMF\Anaconda3\envs\gputools:
#
# Name                    Version                   Build  Channel
khronos-opencl-icd-loader 2018.11.06        h2fa13f4_1003    conda-forge
pyopencl                  2017.2.2                 py36_0    conda-forge

C:\Users\CBMF>python -c "import gputools"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\CBMF\Anaconda3\envs\gputools\lib\site-packages\gputools\__init__.py", line 8, in <module>
    from gputools.config.config import init_device, get_device
  File "C:\Users\CBMF\Anaconda3\envs\gputools\lib\site-packages\gputools\config\config.py", line 9, in <module>
    from gputools.core.ocldevice import OCLDevice
  File "C:\Users\CBMF\Anaconda3\envs\gputools\lib\site-packages\gputools\core\ocldevice.py", line 19, in <module>
    import pyopencl
  File "C:\Users\CBMF\Anaconda3\envs\gputools\lib\site-packages\pyopencl\__init__.py", line 36, in <module>
    import pyopencl._cl as _cl
ImportError: DLL load failed: The specified procedure could not be found.

but, if I install Gohlke's binary that I packed on conda with pyopencl 2017.2 and opencl 1.2, it works fine:

C:\Users\CBMF>conda install -c talley/label/dev pyopencl=2017.2+cl12
C:\Users\CBMF>python -c "import gputools; print(gputools.get_device())"
-------- available devices -----------
platform:       NVIDIA CUDA
device type:    CPU
device type:    GPU
        TITAN X (Pascal)

-------- currently used device -------
NAME:     TITAN X (Pascal)
GLOBAL_MEM_SIZE:          12884901888
GLOBAL_MEM_SIZE:          12884901888
MAX_MEM_ALLOC_SIZE:       3221225472
LOCAL_MEM_SIZE:           49152
IMAGE2D_MAX_WIDTH:        16384
IMAGE2D_MAX_HEIGHT:       32768
IMAGE3D_MAX_WIDTH:        16384
IMAGE3D_MAX_HEIGHT:       16384
IMAGE3D_MAX_DEPTH:        16384
MAX_WORK_GROUP_SIZE:      1024
MAX_WORK_ITEM_SIZES:      [1024, 1024, 64]

it's hard to get definitive information about OpenCL 2 support on NVIDIA. over a year ago they announced beta support, but this makes it look like no? I'm curious whether anyone knows for certain whether the cl1.2 binaries are still required (maybe just on windows)? @VolkerH, you're on windows right? which pyopencl 2017.2 did you use? and have you tried this for any NVIDIA GPUs, or just your Intel GPUs?

Basically, I'd like to set up the conda package so that it will "just work" for most people (who, I think, mostly have NVIDIA), so I can certainly link it to my old pyopencl=2017.2+cl12 package, i'm just surprised this hasn't changed in the last two years...

[VolkerH]

Hi Talley,

pyopencl is a very fragile package for me. I have one environment in which it works and when trying to set up a new conda environment with the pyopencl from conda-forge it is a crap-shoot (both on Windows and Linux).

On one of our Windows workstations (also Titan card, don't know exact model) I tried many versions of pyopencl (several conda-forge and severeal Gohlke packages) until I found one that works. I currently don't have access to that workstation as I'm travelling overseas but I distinctly remember that the version that ended up working was different from the one I have on my latptop.

On Linux I experienced that different builds get pulled (for the same pyopencl versions) depending on which other packages are in the environment (I always assumed that reproducible environments where the aim of conda, but it doesn't work for me lately).

If it helps, this is what the working pyopencl on my laptop reports:

In [5]: pyopencl.version.VERSION
Out[5]: (2017, 2, 2)

and here is the relevant output line from conda list --explicit:

https://conda.anaconda.org/conda-forge/win-64/pyopencl-2017.2.2-py36_0.tar.bz2

As I'm not sure as tagging in Github works across issues I will use the opportunity to point to this: https://github.com/VolkerH/Lattice_Lightsheet_Deskew_Deconv/

[maweigert]

Hi @tlambert03, @VolkerH,

Thanks for fiddling around with this. My (limited) Windows experience with pyopencl are indeed similar - it's a 50/50 chance that it will import properly or that I see the "DLL failed" error, even with Gohlke's packages (that always tended to work better). So I basically never touch pyopencl once it runs on Windows.