search

code-iai / iai_kinect2

Tools for using the Kinect One (Kinect v2) in ROS
Apache License 2.0
872 stars 519 forks source link

Two problems for using opencl #279

Closed Chen-Xieyuanli closed 8 years ago

Chen-Xieyuanli commented 8 years ago

**_Hi everybody,

I got some troubles when I ran kinect2 with opencl

The first one, I used opencl as regmethod**

SUMMARY

PARAMETERS

NODES / kinect2 (nodelet/nodelet) kinect2_bridge (nodelet/nodelet) kinect2_points_xyzrgb_hd (nodelet/nodelet) kinect2_points_xyzrgb_qhd (nodelet/nodelet) kinect2_points_xyzrgb_sd (nodelet/nodelet)

auto-starting new master process[master]: started with pid [17634] ROS_MASTER_URI=http://localhost:11311

setting /run_id to 25db7f44-17e1-11e6-a106-5cc5d4b62664 process[rosout-1]: started with pid [17647] started core service [/rosout] process[kinect2-2]: started with pid [17659] process[kinect2_bridge-3]: started with pid [17665] process[kinect2_points_xyzrgb_sd-4]: started with pid [17666] process[kinect2_points_xyzrgb_qhd-5]: started with pid [17667] process[kinect2_points_xyzrgb_hd-6]: started with pid [17673] [ INFO] [1463016645.622453410]: Loading nodelet /kinect2_bridge of type kinect2_bridge/kinect2_bridge_nodelet to manager kinect2 with the following remappings: [ INFO] [1463016645.627384363]: waitForService: Service [/kinect2/load_nodelet] has not been advertised, waiting... [ INFO] [1463016645.659869801]: Initializing nodelet with 8 worker threads. [ INFO] [1463016645.669024472]: waitForService: Service [/kinect2/load_nodelet] is now available. [ INFO] [1463016645.741641571]: [Kinect2Bridge::initialize] parameter: base_name: kinect2 sensor: default fps_limit: -1 calib_path: /home/chenxieyuanli/catkin_ws/src/iai_kinect2/kinect2_bridge/data/ use_png: false jpeg_quality: 90 png_level: 1 depth_method: cpu depth_device: -1 reg_method: opencl reg_device: -1 max_depth: 12 min_depth: 0.1 queue_size: 5 bilateral_filter: true edge_aware_filter: true publish_tf: false base_name_tf: kinect2 worker_threads: 4 libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_39 libva info: va_openDriver() returns 0 [Info] [VaapiRgbPacketProcessorImpl] driver: Intel i965 driver for Intel(R) Haswell Mobile - 1.7.0 [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 10 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @4:2 with serial 022766543547 [Info] [Freenect2Impl] found 1 devices [ INFO] [1463016645.789276844]: [Kinect2Bridge::initDevice] Kinect2 devices found: [ INFO] [1463016645.789315617]: [Kinect2Bridge::initDevice] 0: 022766543547 (selected) [Info] [Freenect2DeviceImpl] opening... [Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20_16384 ir: 60_8_33792 [Info] [Freenect2DeviceImpl] opened [ INFO] [1463016645.924091482]: [Kinect2Bridge::initDevice] starting kinect2 [Info] [Freenect2DeviceImpl] starting... [Info] [Freenect2DeviceImpl] submitting rgb transfers... [Info] [Freenect2DeviceImpl] submitting depth transfers... [Info] [Freenect2DeviceImpl] started [ INFO] [1463016646.282296201]: [Kinect2Bridge::initDevice] device serial: 022766543547 [ INFO] [1463016646.282349328]: [Kinect2Bridge::initDevice] device firmware: 4.0.3911.0 [Info] [Freenect2DeviceImpl] stopping... [Info] [Freenect2DeviceImpl] canceling rgb transfers... [Info] [Freenect2DeviceImpl] canceling depth transfers... [Info] [Freenect2DeviceImpl] stopped [ INFO] [1463016646.549452252]: [DepthRegistration::New] Using OpenCL registration method! [ INFO] [1463016646.549547001]: [DepthRegistration::New] Using OpenCL registration method! drm_intel_gem_bo_context_exec() failed: Invalid argument drm_intel_gem_bo_context_exec() failed: Invalid argument Beignet: warning - disable atomic in L3 feature. drm_intel_gem_bo_context_exec() failed: Invalid argument drm_intel_gem_bo_context_exec() failed: Invalid argument Beignet: warning - disable atomic in L3 feature. [ INFO] [1463016646.739650993]: [DepthRegistrationOpenCL::init] devices: [ INFO] [1463016646.739710023]: [DepthRegistrationOpenCL::init] 0: Intel(R) HD Graphics Haswell GT2 Mobile (GPU)[Intel] [ INFO] [1463016646.739733820]: [DepthRegistrationOpenCL::init] selected device: Intel(R) HD Graphics Haswell GT2 Mobile (GPU)[Intel] drm_intel_gem_bo_context_exec() failed: Invalid argument drm_intel_gem_bo_context_exec() failed: Invalid argument Beignet: warning - disable atomic in L3 feature. drm_intel_gem_bo_context_exec() failed: Invalid argument drm_intel_gem_bo_context_exec() failed: Invalid argument Beignet: warning - disable atomic in L3 feature. [ INFO] [1463016647.004062707]: [DepthRegistrationOpenCL::init] devices: [ INFO] [1463016647.004138697]: [DepthRegistrationOpenCL::init] 0: Intel(R) HD Graphics Haswell GT2 Mobile (GPU)[Intel] [ INFO] [1463016647.004157282]: [DepthRegistrationOpenCL::init] selected device: Intel(R) HD Graphics Haswell GT2 Mobile (GPU)[Intel] [ INFO] [1463016647.103339757]: [Kinect2Bridge::main] waiting for clients to connect [ INFO] [1463016662.522616805]: [Kinect2Bridge::callbackStatus] client connected. starting device... [Info] [Freenect2DeviceImpl] starting... [Info] [Freenect2DeviceImpl] submitting rgb transfers... [Info] [Freenect2DeviceImpl] submitting depth transfers... [Info] [Freenect2DeviceImpl] started drm_intel_gem_bo_context_exec() failed: Invalid argument *__[ERROR] [1463016663.153344510]: [DepthRegistrationOpenCL::registerDepth] depth_registration_opencl.cpp data->queue.enqueueNDRangeKernel(data->kernelSetZero, cl::NullRange, range, cl::NullRange, NULL, &eventZero[1]) failed: -5 [Error] [RgbPacketStreamParser] packetsize or sequence doesn't match! drm_intel_gem_bo_context_exec() failed: Invalid argument [ERROR] [1463016663.394779557]: [DepthRegistrationOpenCL::registerDepth] depth_registration_opencl.cpp data->queue.enqueueNDRangeKernel(data->kernelSetZero, cl::NullRange, range, cl::NullRange, NULL, &eventZero[1]) failed: -5 drm_intel_gem_bo_context_exec() failed: Invalid argument [ERROR] [1463016663.649318157]: [DepthRegistrationOpenCL::registerDepth] depth_registration_opencl.cpp data->queue.enqueueNDRangeKernel(data->kernelSetZero, cl::NullRange, range, cl::NullRange, NULL, &eventZero[1]) failed: -5_** [ INFO] [1463016663.649806097]: [Kinect2Bridge::callbackStatus] no clients connected. stopping device... [Info] [Freenect2DeviceImpl] stopping... [Info] [Freenect2DeviceImpl] canceling rgb transfers... [Info] [DepthPacketStreamParser] 18 packets were lost [Info] [Freenect2DeviceImpl] canceling depth transfers... [Info] [Freenect2DeviceImpl] stopped

_The second one, I used opencl as depthmethod

SUMMARY

PARAMETERS

NODES / kinect2 (nodelet/nodelet) kinect2_bridge (nodelet/nodelet) kinect2_points_xyzrgb_hd (nodelet/nodelet) kinect2_points_xyzrgb_qhd (nodelet/nodelet) kinect2_points_xyzrgb_sd (nodelet/nodelet)

auto-starting new master process[master]: started with pid [18156] ROS_MASTER_URI=http://localhost:11311

setting /run_id to a64df0b6-17e2-11e6-b740-5cc5d4b62664 process[rosout-1]: started with pid [18169] started core service [/rosout] process[kinect2-2]: started with pid [18178] process[kinect2_bridge-3]: started with pid [18187] process[kinect2_points_xyzrgb_sd-4]: started with pid [18188] process[kinect2_points_xyzrgb_qhd-5]: started with pid [18190] [ INFO] [1463017290.609707057]: Loading nodelet /kinect2_bridge of type kinect2_bridge/kinect2_bridge_nodelet to manager kinect2 with the following remappings: [ INFO] [1463017290.611831556]: waitForService: Service [/kinect2/load_nodelet] has not been advertised, waiting... process[kinect2_points_xyzrgb_hd-6]: started with pid [18203] [ INFO] [1463017290.660576438]: Initializing nodelet with 8 worker threads. [ INFO] [1463017290.673854865]: waitForService: Service [/kinect2/load_nodelet] is now available. [ INFO] [1463017290.940451066]: [Kinect2Bridge::initialize] parameter: base_name: kinect2 sensor: default fps_limit: -1 calib_path: /home/chenxieyuanli/catkin_ws/src/iai_kinect2/kinect2_bridge/data/ use_png: false jpeg_quality: 90 png_level: 1 depth_method: opencl depth_device: -1 reg_method: cpu reg_device: -1 max_depth: 12 min_depth: 0.1 queue_size: 5 bilateral_filter: true edge_aware_filter: true publish_tf: false base_name_tf: kinect2 worker_threads: 4 drm_intel_gem_bo_context_exec() failed: Invalid argument drm_intel_gem_bo_context_exec() failed: Invalid argument Beignet: warning - disable atomic in L3 feature. drm_intel_gem_bo_context_exec() failed: Invalid argument drm_intel_gem_bo_context_exec() failed: Invalid argument Beignet: warning - disable atomic in L3 feature. [Info] [OpenCLDepthPacketProcessorImpl] devices: [Info] [OpenCLDepthPacketProcessorImpl] 0: Intel(R) HD Graphics Haswell GT2 Mobile (GPU)[Intel] [Info] [OpenCLDepthPacketProcessorImpl] selected device: Intel(R) HD Graphics Haswell GT2 Mobile (GPU)[Intel] [Info] [OpenCLDepthPacketProcessorImpl] building OpenCL program... [Info] [OpenCLDepthPacketProcessorImpl] OpenCL program built successfully libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function vaDriverInit_0_39 libva info: va_openDriver() returns 0 [Info] [VaapiRgbPacketProcessorImpl] driver: Intel i965 driver for Intel(R) Haswell Mobile - 1.7.0 [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 10 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @4:2 with serial 022766543547 [Info] [Freenect2Impl] found 1 devices [ INFO] [1463017291.444187917]: [Kinect2Bridge::initDevice] Kinect2 devices found: [ INFO] [1463017291.444232431]: [Kinect2Bridge::initDevice] 0: 022766543547 (selected) [Info] [Freenect2DeviceImpl] opening... [Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20_16384 ir: 60_8_33792 [Info] [Freenect2DeviceImpl] opened [ INFO] [1463017291.575917796]: [Kinect2Bridge::initDevice] starting kinect2 [Info] [Freenect2DeviceImpl] starting... [Info] [Freenect2DeviceImpl] submitting rgb transfers... [Info] [Freenect2DeviceImpl] submitting depth transfers... [Info] [Freenect2DeviceImpl] started [ INFO] [1463017291.902429855]: [Kinect2Bridge::initDevice] device serial: 022766543547 [ INFO] [1463017291.902474913]: [Kinect2Bridge::initDevice] device firmware: 4.0.3911.0 [Info] [Freenect2DeviceImpl] stopping... [Info] [Freenect2DeviceImpl] canceling rgb transfers... *drm_intel_gem_bo_contextexec() failed: Invalid argument [Error] [OpenCLDepthPacketProcessorImpl]\ queue.enqueueNDRangeKernel(kernel_processPixelStage1, cl::NullRange, cl::NDRange(IMAGESIZE), cl::NullRange, &eventWrite, &eventPPS1[0]): -5 [Info] [Freenect2DeviceImpl] canceling depth transfers... [Info] [Freenect2DeviceImpl] stopped [Info] [OpenCLDepthPacketProcessorImpl] building OpenCL program... [Info] [OpenCLDepthPacketProcessorImpl] OpenCL program built successfully [ INFO] [1463017292.467325711]: [DepthRegistration::New] Using CPU registration method! [ INFO] [1463017292.467469410]: [DepthRegistration::New] Using CPU registration method! [ INFO] [1463017292.536850169]: [Kinect2Bridge::main] waiting for clients to connect [ INFO] [1463017308.023987560]: [Kinect2Bridge::callbackStatus] client connected. starting device... [Info] [Freenect2DeviceImpl] starting... [Info] [Freenect2DeviceImpl] submitting rgb transfers... [Info] [Freenect2DeviceImpl] submitting depth transfers... [Info] [Freenect2DeviceImpl] started **[Error] [RgbPacketStreamParser] packetsize or sequence doesn't match!_** [Info] [DepthPacketStreamParser] -2 packets were lost [ INFO] [1463017311.360507492]: [Kinect2Bridge::main] depth processing: ~infms (~0Hz) publishing rate: ~0Hz [ INFO] [1463017311.360578124]: [Kinect2Bridge::main] color processing: ~15.6279ms (~63.9879Hz) publishing rate: ~26.2584Hz [Info] [VaapiRgbPacketProcessor] avg. time: 5.7987ms -> ~172.453Hz

Hope to your reply, thanks.

kohrt commented 8 years ago

Hi, there is something wrong with you OpenCL installation. Which OS/Kernel and Beignet version are you using?

Chen-Xieyuanli commented 8 years ago

Thanks Thiemo! It's so nice of you to help me so many times. I've already solved these problems.

Here is the status of my computer: Intel® Core™ i7-4710MQ CPU @ 2.50GHz × 8 Intel® Haswell Mobile & GeForce GT 730M/PCIe/SSE2

chenxieyuanli@chenxieyuanli-ThinkPad-T440p:~$ uname -r 3.16.0-70-generic

I download "intel_code_builder_for_opencl_2015_ubuntu_5.0.0.43_x64.tgz" from https://software.intel.com/en-us/articles/intel-code-builder-for-opencl-api The following steps worked for me on Ubuntu 15.04 based on Noah's instructions above:

    $ sudo apt-get install -y rpm alien libnuma1

    $ tar -xvf intel_code_builder_for_opencl_2015_ubuntu_5.0.0.43_x64.tgz
    $ cd intel_code_builder_for_opencl_2015_ubuntu_5.0.0.43_x64

    $ sudo rpm --import PUBLIC_KEY.PUB

    $ cd rpm

    $ fakeroot alien --to-deb opencl-1.2-base-5.0.0.43-1.x86_64.rpm
    $ fakeroot alien --to-deb opencl-1.2-intel-cpu-5.0.0.43-1.x86_64.rpm

    $ sudo dpkg -i opencl-1.2-base_5.0.0.43-2_amd64.deb
    $ sudo dpkg -i opencl-1.2-intel-cpu_5.0.0.43-2_amd64.deb

    The above installs the library files and installable client driver registration in /opt/intel/opencl-1.2-5.0.0.43.
    Two more steps were needed to run an OpenCL program.

    Add library to search path:
    $ sudo nano /etc/ld.so.conf.d/intelOpenCL.conf
    Add the line:
    /opt/intel/opencl-1.2-5.0.0.43/lib64

    Link to the intel icd file in the expected location:
    $ sudo ln /opt/intel/opencl-1.2-5.0.0.43/etc/intel64.icd /etc/OpenCL/vendors/intel64.icd
        $ sudo ldconfig
    $ sudo clinfo

And that's done! I can use kinect2 with qhd in 30Hz. that's amazing!

Thank you Thiemo!

Number of platforms: 3 Platform Profile: FULL_PROFILE Platform Version: OpenCL 1.2 beignet 1.1.1 Platform Name: Intel Gen OCL Driver Platform Vendor: Intel Platform Extensions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_spir cl_khr_icd Platform Profile: FULL_PROFILE Platform Version: OpenCL 1.2 LINUX Platform Name: Intel(R) OpenCL Platform Vendor: Intel(R) Corporation Platform Extensions: cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir cl_khr_fp64 Platform Profile: FULL_PROFILE Platform Version: OpenCL 1.2 CUDA 7.5.23 Platform Name: NVIDIA CUDA Platform Vendor: NVIDIA Corporation Platform Extensions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts

Platform Name: Intel Gen OCL Driver Number of devices: 1 Device Type: CL_DEVICE_TYPE_GPU Device ID: 32902 Max compute units: 20 Max work items dimensions: 3 Max work items[0]: 512 Max work items[1]: 512 Max work items[2]: 512 Max work group size: 512 Preferred vector width char: 16 Preferred vector width short: 8 Preferred vector width int: 4 Preferred vector width long: 2 Preferred vector width float: 4 Preferred vector width double: 0 Native vector width char: 8 Native vector width short: 8 Native vector width int: 4 Native vector width long: 2 Native vector width float: 4 Native vector width double: 2 Max clock frequency: 1000Mhz Address bits: 32 Max memory allocation: 1073741824 Image support: Yes Max number of images read arguments: 128 Max number of images write arguments: 8 Max image 2D width: 8192 Max image 2D height: 8192 Max image 3D width: 8192 Max image 3D height: 8192 Max image 3D depth: 2048 Max samplers within kernel: 16 Max size of kernel argument: 1024 Alignment (bits) of base address: 1024 Minimum alignment (bytes) for any datatype: 128 Single precision floating point capability Denorms: No Quiet NaNs: Yes Round to nearest even: Yes Round to zero: No Round to +ve and infinity: No IEEE754-2008 fused multiply-add: No Cache type: Read/Write Cache line size: 64 Cache size: 8192 Global memory size: 2147483648 Constant buffer size: 134217728 Max number of constant args: 8 Local memory type: Global Local memory size: 65536 Error correction support: 0 Unified memory for Host and Device: 1 Profiling timer resolution: 80 Device endianess: Little Available: Yes Compiler available: Yes Execution capabilities:
Execute OpenCL kernels: Yes Execute native function: Yes Queue properties:
Out-of-Order: No Profiling : Yes Platform ID: 0x7ffbe7407a60 Name: Intel(R) HD Graphics Haswell GT2 Mobile Vendor: Intel Device OpenCL C version: OpenCL C 1.2 beignet 1.1.1 Driver version: 1.1.1 Profile: FULL_PROFILE Version: OpenCL 1.2 beignet 1.1.1 Extensions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_spir cl_khr_icd

Platform Name: Intel(R) OpenCL Number of devices: 1 Device Type: CL_DEVICE_TYPE_CPU Device ID: 32902 Max compute units: 8 Max work items dimensions: 3 Max work items[0]: 8192 Max work items[1]: 8192 Max work items[2]: 8192 Max work group size: 8192 Preferred vector width char: 1 Preferred vector width short: 1 Preferred vector width int: 1 Preferred vector width long: 1 Preferred vector width float: 1 Preferred vector width double: 1 Native vector width char: 32 Native vector width short: 16 Native vector width int: 8 Native vector width long: 4 Native vector width float: 8 Native vector width double: 4 Max clock frequency: 2500Mhz Address bits: 64 Max memory allocation: 2012067840 Image support: Yes Max number of images read arguments: 480 Max number of images write arguments: 480 Max image 2D width: 16384 Max image 2D height: 16384 Max image 3D width: 2048 Max image 3D height: 2048 Max image 3D depth: 2048 Max samplers within kernel: 480 Max size of kernel argument: 3840 Alignment (bits) of base address: 1024 Minimum alignment (bytes) for any datatype: 128 Single precision floating point capability Denorms: Yes Quiet NaNs: Yes Round to nearest even: Yes Round to zero: No Round to +ve and infinity: No IEEE754-2008 fused multiply-add: No Cache type: Read/Write Cache line size: 64 Cache size: 262144 Global memory size: 8048271360 Constant buffer size: 131072 Max number of constant args: 480 Local memory type: Global Local memory size: 32768 Error correction support: 0 Unified memory for Host and Device: 1 Profiling timer resolution: 1 Device endianess: Little Available: Yes Compiler available: Yes Execution capabilities:
Execute OpenCL kernels: Yes Execute native function: Yes Queue properties:
Out-of-Order: Yes Profiling : Yes Platform ID: 0xd1a7a0 Name: Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz Vendor: Intel(R) Corporation Device OpenCL C version: OpenCL C 1.2 Driver version: 1.2.0.43 Profile: FULL_PROFILE Version: OpenCL 1.2 (Build 43) Extensions: cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir cl_khr_fp64

Platform Name: NVIDIA CUDA Number of devices: 1 Device Type: CL_DEVICE_TYPE_GPU Device ID: 4318 Max compute units: 2 Max work items dimensions: 3 Max work items[0]: 1024 Max work items[1]: 1024 Max work items[2]: 64 Max work group size: 1024 Preferred vector width char: 1 Preferred vector width short: 1 Preferred vector width int: 1 Preferred vector width long: 1 Preferred vector width float: 1 Preferred vector width double: 1 Native vector width char: 1 Native vector width short: 1 Native vector width int: 1 Native vector width long: 1 Native vector width float: 1 Native vector width double: 1 Max clock frequency: 758Mhz Address bits: 64 Max memory allocation: 268402688 Image support: Yes Max number of images read arguments: 256 Max number of images write arguments: 16 Max image 2D width: 16384 Max image 2D height: 16384 Max image 3D width: 4096 Max image 3D height: 4096 Max image 3D depth: 4096 Max samplers within kernel: 32 Max size of kernel argument: 4352 Alignment (bits) of base address: 4096 Minimum alignment (bytes) for any datatype: 128 Single precision floating point capability Denorms: Yes Quiet NaNs: Yes Round to nearest even: Yes Round to zero: Yes Round to +ve and infinity: Yes IEEE754-2008 fused multiply-add: Yes Cache type: Read/Write Cache line size: 128 Cache size: 32768 Global memory size: 1073610752 Constant buffer size: 65536 Max number of constant args: 9 Local memory type: Local Local memory size: 49152 Error correction support: 0 Unified memory for Host and Device: 0 Profiling timer resolution: 1000 Device endianess: Little Available: Yes Compiler available: Yes Execution capabilities:
Execute OpenCL kernels: Yes Execute native function: No Queue properties:
Out-of-Order: Yes Profiling : Yes Platform ID: 0xd160e0 Name: GeForce GT 730M Vendor: NVIDIA Corporation Device OpenCL C version: OpenCL C 1.2 Driver version: 352.63 Profile: FULL_PROFILE Version: OpenCL 1.2 CUDA Extensions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64