An OpenCL error occurred: clBuildProgram failed

[akocabas]

I get the following error when I start the node in a docker container:

Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://1-3659:45857/

SUMMARY
========

PARAMETERS
 * /camera/zivid/zivid_camera/frame_id: zivid_optical_frame
 * /camera/zivid/zivid_camera/serial_number: 22295FBA
 * /rosdistro: noetic
 * /rosversion: 1.15.14

NODES
  /camera/zivid/
    zivid_camera (zivid_camera/zivid_camera_node)

ROS_MASTER_URI=http://localhost:11311

process[camera/zivid/zivid_camera-1]: started with pid [855]
[ INFO] [1668478790.951814691]: Creating a nodelet::Loader
[ INFO] [1668478790.959547132]: Initializing nodelet with 12 worker threads.
[ INFO] [1668478790.960818215]: Loading nodelet 'zivid_camera/nodelet'
[ INFO] [1668478791.039600360]: Zivid ROS driver version 2.3.0
[ INFO] [1668478791.039654439]: Node's namespace is '/camera/zivid'
[ INFO] [1668478791.039679921]: Built towards Zivid Core version 2.8.0+891708ba-1
[ INFO] [1668478791.039699521]: Running with Zivid Core version 2.8.0+891708ba-1
[ INFO] [1668478793.093512247]: 1 cameras found
[ INFO] [1668478793.093611658]: Searching for camera with serial number '22295FBA' ...
[ INFO] [1668478793.136979503]: { Camera info: { Model: zividOnePlusMedium, Model Name: Zivid One Plus M, Serial Number: 22295FBA, Hardware Revision: 3.2, Firmware Version: 1.9.2+6f0325da, USB Path: na, Network Path: na, Internal camera id: 1204475, Projector firmware tag: ZIVID 1.0.5 } }
[ INFO] [1668478793.137039165]: Connecting to camera '22295FBA'
[ERROR] [1668478795.557401863]: Failed to initialize camera driver. Exception: "An OpenCL error occurred: clBuildProgram failed

This may indicate a problem with your OpenCL device and driver, or an incompatibility between the compute kernels and your OpenCL implementation. Try updating your drivers, verify that your OpenCL device and driver supports OpenCL 1.2, and install any pending operating system updates.

For more help, visit the Zivid Knowledge Base at https://support.zivid.com.

[CL_BUILD_PROGRAM_FAILURE]"
[FATAL] [1668478795.557547763]: Failed to load nodelet 'zivid_camera/nodelet'!
[camera/zivid/zivid_camera-1] process has died [pid 855, exit code 1, cmd /home/catkin_ws/devel/lib/zivid_camera/zivid_camera_node __name:=zivid_camera __log:=/root/.ros/log/c2dc39ac-648b-11ed-bdb5-f077c3d6518c/camera-zivid-zivid_camera-1.log].
log file: /root/.ros/log/c2dc39ac-648b-11ed-bdb5-f077c3d6518c/camera-zivid-zivid_camera-1*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done

System

• Ubuntu 20.04
• ROS Noetic

• Nvidia GPU

  Output of clinfo inside the container

  
  Number of platforms                               1
  Platform Name                                   NVIDIA CUDA
  Platform Vendor                                 NVIDIA Corporation
  Platform Version                                OpenCL 3.0 CUDA 11.8.87
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_khr_gl_event cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_nv_kernel_attribute cl_khr_device_uuid cl_khr_pci_bus_info cl_khr_external_semaphore cl_khr_external_memory cl_khr_external_semaphore_opaque_fd cl_khr_external_memory_opaque_fd
  Platform Host timer resolution                  0ns
  Platform Extensions function suffix             NV
  
  Platform Name                                   NVIDIA CUDA
  Number of devices                                 1
  Device Name                                     Quadro RTX 3000
  Device Vendor                                   NVIDIA Corporation
  Device Vendor ID                                0x10de
  Device Version                                  OpenCL 3.0 CUDA
  Driver Version                                  520.56.06
  Device OpenCL C Version                         OpenCL C 1.2 
  Device Type                                     GPU
  Device Topology (NV)                            PCI-E, 01:00.0
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               30
  Max clock frequency                             1380MHz
  Compute Capability (NV)                         7.5
  Device Partition                                (core)
  Max number of sub-devices                     1
  Supported partition types                     None
  Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             1024x1024x64
  Max work group size                             1024
  === CL_PROGRAM_BUILD_LOG ===
  Preferred work group size multiple              <getWGsizes:1200: create kernel : error -45>
  Warp size (NV)                                  32
  Max sub-groups per work group                   0
  Preferred / native vector sizes                 
  char                                                 1 / 1       
  short                                                1 / 1       
  int                                                  1 / 1       
  long                                                 1 / 1       
  half                                                 0 / 0        (n/a)
  float                                                1 / 1       
  double                                               1 / 1        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
  Denormals                                     Yes
  Infinity and NANs                             Yes
  Round to nearest                              Yes
  Round to zero                                 Yes
  Round to infinity                             Yes
  IEEE754-2008 fused multiply-add               Yes
  Support is emulated in software               No
  Correctly-rounded divide and sqrt operations  Yes
  Double-precision Floating-point support         (cl_khr_fp64)
  Denormals                                     Yes
  Infinity and NANs                             Yes
  Round to nearest                              Yes
  Round to zero                                 Yes
  Round to infinity                             Yes
  IEEE754-2008 fused multiply-add               Yes
  Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              6222970880 (5.796GiB)
  Error Correction support                        No
  Max memory allocation                           1555742720 (1.449GiB)
  Unified memory for Host and Device              No
  Integrated memory (NV)                          No
  Shared Virtual Memory (SVM) capabilities        (core)
  Coarse-grained buffer sharing                 Yes
  Fine-grained buffer sharing                   No
  Fine-grained system sharing                   No
  Atomics                                       No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       4096 bits (512 bytes)
  Preferred alignment for atomics                 
  SVM                                           0 bytes
  Global                                        0 bytes
  Local                                         0 bytes
  Max size for global variable                    0
  Preferred total size of global vars             0
  Global Memory cache type                        Read/Write
  Global Memory cache size                        983040 (960KiB)
  Global Memory cache line size                   128 bytes
  Image support                                   Yes
  Max number of samplers per kernel             32
  Max size for 1D images from buffer            268435456 pixels
  Max 1D or 2D image array size                 2048 images
  Max 2D image size                             32768x32768 pixels
  Max 3D image size                             16384x16384x16384 pixels
  Max number of read image args                 256
  Max number of write image args                32
  Max number of read/write image args           0
  Max number of pipe args                         0
  Max active pipe reservations                    0
  Max pipe packet size                            0
  Local memory type                               Local
  Local memory size                               49152 (48KiB)
  Registers per block (NV)                        65536
  Max number of constant args                     9
  Max constant buffer size                        65536 (64KiB)
  Max size of kernel argument                     4352 (4.25KiB)
  Queue properties (on host)                      
  Out-of-order execution                        Yes
  Profiling                                     Yes
  Queue properties (on device)                    
  Out-of-order execution                        No
  Profiling                                     No
  Preferred size                                0
  Max size                                      0
  Max queues on device                            0
  Max events on device                            0
  Prefer user sync for interop                    No
  Profiling timer resolution                      1000ns
  Execution capabilities                          
  Run OpenCL kernels                            Yes
  Run native kernels                            No
  Sub-group independent forward progress        No
  Kernel execution timeout (NV)                 Yes
  Concurrent copy and kernel execution (NV)       Yes
  Number of async copy engines                  3
  IL version                                    (n/a)
  printf() buffer size                            1048576 (1024KiB)
  Built-in kernels                                (n/a)
  Device Extensions                               cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_khr_gl_event cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_nv_kernel_attribute cl_khr_device_uuid cl_khr_pci_bus_info cl_khr_external_semaphore cl_khr_external_memory cl_khr_external_semaphore_opaque_fd cl_khr_external_memory_opaque_fd

NULL platform behavior clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) NVIDIA CUDA clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [NV] clCreateContext(NULL, ...) [default] Success [NV] clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) No platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) No platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) Invalid device type for platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) No platform

ICD loader properties ICD loader Name OpenCL ICD Loader ICD loader Vendor OCL Icd free software ICD loader Version 2.2.11 ICD loader Profile OpenCL 2.1 NOTE: your OpenCL library only supports OpenCL 2.1, but some installed platforms support OpenCL 3.0. Programs using 3.0 features may crash or behave unexpectedly

### Output of nvdia-smi

+-----------------------------------------------------------------------------+ | NVIDIA-SMI 520.56.06 Driver Version: 520.56.06 CUDA Version: 11.8 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Quadro RTX 3000 Off | 00000000:01:00.0 On | N/A | | N/A 48C P8 12W / N/A | 240MiB / 6144MiB | 28% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| +-----------------------------------------------------------------------------+

### These commands are run in the Dockerfile

RUN apt-get install -y \ unzip \ tar \ curl \ xz-utils \ ocl-icd-libopencl1 \ opencl-headers \ clinfo

make opencl available in docker container (zivid)

RUN mkdir -p /etc/OpenCL/vendors && \ echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd ENV NVIDIA_VISIBLE_DEVICES all ENV NVIDIA_DRIVER_CAPABILITIES compute,utility

Install Zivid SDK

RUN wget https://www.zivid.com/hubfs/softwarefiles/releases/2.8.0+891708ba-1/u20/zivid-telicam-driver_3.0.1.1-3_amd64.deb \ https://www.zivid.com/hubfs/softwarefiles/releases/2.8.0+891708ba-1/u20/zivid_2.8.0+891708ba-1_amd64.deb \ https://www.zivid.com/hubfs/softwarefiles/releases/2.8.0+891708ba-1/u20/zivid-studio_2.8.0+891708ba-1_amd64.deb \ https://www.zivid.com/hubfs/softwarefiles/releases/2.8.0+891708ba-1/u20/zivid-tools_2.8.0+891708ba-1_amd64.deb RUN apt install ./zivid-telicam-driver_3.0.1.1-3_amd64.deb \ ./zivid_2.8.0+891708ba-1_amd64.deb \ ./zivid-studio_2.8.0+891708ba-1_amd64.deb \ ./zivid-tools_2.8.0+891708ba-1_amd64.deb

RUN /bin/bash -c ". /opt/ros/noetic/setup.bash \ && cd catkin_ws/src \ && git clone https://github.com/zivid/zivid-ros.git \ && cd zivid-ros \ && git checkout a04aebf \ && cd ../.. \ && rosdep update \ && rosdep install --from-paths src --ignore-src -r -y \ && catkin build"

Can you help me solve this problem? 
Thanks in advance.

[akocabas]

The problem doesn't occur if Nvidia driver and cuda versions are downgraded. Using Driver Version: 510.85.02 and CUDA Version: 11.6 solved the problem in my case. This is more like a Nvidia issue than a Zivid issure. Therefore i'm closing this.