CUDA vs OpenCL

[NicerNewerCar]

Feature parity between CUDA and OpenCL

Feature	CUDA Implementation	OpenCL Implementation
Background Renderer	Yes	Yes
Compositor	Yes	Yes
Contrast Filter	Yes	Yes
DRR Background (DRR Rendering)	Yes	Yes
Guassian Filter	Yes	Yes
HDist	Yes	No
Merger	Yes	Yes
Mult	Yes	Yes
NCC	Yes	Yes
PSO	Yes*	No
Rad Renderer	Yes	Yes
Ray Caster	Yes	Yes
Sharpen Filter	Yes	Yes
Sobel Filter	Yes	Yes

• PSO has a Cuda Implementation however it is not used in favor of a regular C++ implementation located in libautoscoper
• HDist has a Cuda implementation and is used by the Tracker classes load function used on top of the ncc code, OpenCL just uses ncc. See lines 274-278 in Tracker.cpp.
• HDist does not seem terrible to implement in OpenCL, based off of the CUDA implementation, the algorithm description, and this paper.

Build warnings: CUDA vs OpenCL

CUDA

Addressed in Issue #42

Warning	# of Occurrences	File(s)	Link to Docs
C4276	5	Compositor_kernals.cu, RayCaster_kernals.cu, Merger_kernals.cu, DRRBackground_kernals.cu, Mult_kernals.cu	https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4267?view=msvc-170
#1215-D	3	RadRenderer_kernals.cu, RayCaster_kernals.cu, BackgroundRenderer_kernals.cu	https://forums.developer.nvidia.com/t/tex2d-deprecated-how-to-replace-it/175878

copy of C4267:

C:\SAM\Autoscoper\libautoscoper\src\gpu\cuda\Compositor_kernels.cu(70): warning C4267: 'argument': conversion from 'size_t' to 'unsigned int', possible loss of data [C:\SAM\Autoscoper\build-test\Autoscoper-build\libautoscoper\libautoscoper.vcxproj] [C:\SAM\Autoscoper\build-test\Autoscoper.vcxproj]

copy of #1215-D:

C:\SAM\Autoscoper\libautoscoper\src\gpu\cuda\RadRenderer_kernels.cu(123): warning #1215-D: function "tex2D(texture<T, 2, cudaReadModeNormalizedFloat>, float, float) [with T=unsigned char]" [C:\SAM\Autoscoper\build-test\Autoscoper-build\libautoscoper\libautoscoper.vcxproj] [C:\SAM\Autoscoper\build-test\Autosco
per.vcxproj]

OpenCL

Fixed in PR #44

Warning	# of Occurrences	File(s)	Link to Docs
MSB8065	13	all calls to shader_to_headers converting .cl to .cl.h	No official Microsoft Docs

Warning seems to be caused by the shader_to_header creating a file with the extension example.cl.h but the program is looking for just example.h.

Copy of one of the warnings:

 Generating BackgroundRenderer.h
    -- Converting gpu/opencl/kernel/BackgroundRenderer.cl to C:/SAM/Autoscoper/build-test/Autoscoper-build/libautoscope
  r/gpu/opencl/kernel/BackgroundRenderer.cl.h

C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(247,5): w
arning MSB8065: Custom build for item "C:\SAM\Autoscoper\build-test\Autoscoper-build\CMakeFiles\5a64e70afe57404e94303d8
7ad075ff6\BackgroundRenderer.h.rule" succeeded, but specified output "c:\sam\autoscoper\build-test\autoscoper-build\lib
autoscoper\backgroundrenderer.h" has not been created. This may cause incremental build to work incorrectly. [C:\SAM\Au
toscoper\build-test\Autoscoper-build\libautoscoper\shader_to_header.vcxproj] [C:\SAM\Autoscoper\build-test\Autoscoper.v
cxproj]

History

Cuda was last updated in August of 2020 127b063 OpenCL was last updated in April of 2018 fe02d20

Distribution

According to Ben Birdsong:

Anecdotally, I’ve found it easier to build and distribute packages for OpenCL than CUDA. ITKVkFFTBackend builds against the Khronos Group’s OpenCL-ICD-Loader which finds a compatible OpenCL distribution to load on the client system. Note that CUDA typically limits user hardware to NVIDIA devices while OpenCL is platform agnostic.

• OpenCL-ICD-Loader
• OpenCL is open source and supports more than just NVIDIA gpus
• For distribution on MacOS going forward OpenCL is required
• CUDA will require the user to download the correct package
• NVIDIA gpus are most common
• More recent dev work on CUDA
• From preliminary testing it seems that there is no noticeable difference between the runtime of the two backends, however, CUDA appears to build slower than OpenCL

OpenCL-ICD-Loader

Description:

OpenCL defines an Installable Client Driver (ICD) mechanism to allow developers to build applications against an Installable Client Driver loader (ICD loader) rather than linking their applications against a specific OpenCL implementation. The ICD Loader is responsible for:

• Exporting OpenCL API entry points
• Enumerating OpenCL implementations
• Forwarding OpenCL API calls to the correct implementation
• This repo contains the source code and tests for the Khronos official OpenCL ICD Loader.

Note that this repo does not contain an OpenCL implementation (ICD). You will need to obtain and install an OpenCL implementation for your OpenCL device that supports the OpenCL ICD extension cl_khr_icd to run an application using the OpenCL ICD Loader.

The OpenCL Installable Client Driver extension (cl_khr_icd) is described in the OpenCL extensions specification, which may be found on the Khronos OpenCL Registry.

• Simple to build (took me ~1.5 min)
• Can add to SAM's Superbuild
• May have to register the ICD

Basic usage in cmake (from the OpenCL-ICD-Loader repo ):

cmake_minimum_required(VERSION 3.0)
cmake_policy(VERSION 3.0...3.18.4)
project(proj)
add_executable(app main.cpp)
find_package(OpenCLHeaders REQUIRED)
find_package(OpenCLICDLoader REQUIRED)
target_link_libraries(app PRIVATE OpenCL::Headers OpenCL::OpenCL)

Installation

OS/Backend	CUDA	OpenCL
Windows	CUDA tk, Visual Studio, Update GPU drivers, Update PATH	Shipped w/ (May need to update GPU drivers)
Linux	CUDA tk, gcc, Correct kernal, MLNX_OFED, package manager	Shipped w/ (May need to update GPU drivers)
MacOS	CUDA tk, Xcode, command line tools	OSX > 10.6 is shipped with OpenCL
NOTES	CUDA is not on M1/2 macs or any RISC/ARM CPUs

Time analysis

Time were recorded by timing the optimize method in the tracker class. Both OpenCL and CUDA had 800 tracks performed. The following is the result of running this script.

CPU: 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz GPU: NVIDIA RTX A2000 Laptop GPU RAM: 32GB OS: Windows 11 Pro 64-bit

Average CUDA: 9.18217857142857 OpenCL: 10.284844221105526 Median CUDA: 9.023 OpenCL: 10.1995 Standard Deviation CUDA: 0.9538181864603326 OpenCL: 1.1481717995852314

Welch's T Test: t-statistic: -20.7770213974207 p-value: 1.0891601943552252e-84 There is significant difference between CUDA and OpenCL

Cuda	Opencl

[jcfr]

OSX > 10.6 is shipped with OpenCL

After talking with @thewtex, it seems the version shipped on macOS is OpenCL 1.2 (see ITKVkFFTBackend/CMakeLists.txt) and current GPU integration effort are focusing on Metal.

[jcfr]

HDist

The is available in the SegmentComparison module.

Implementation available in vtkSlicerSegmentComparisonModuleLogic::ComputeHausdorffDistances through plastimatch/util/hausdorff_distance.h

[NicerNewerCar]

Hdist info https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.directed_hausdorff.html https://www.slicer.org/wiki/Documentation/Nightly/Extensions/ModelToModelDistance https://www.insight-journal.org/browse/publication/106 https://www.nitrc.org/projects/meshvalmet/ https://www.nitrc.org/docman/view.php/449/919/mesh-icme02.pdf

[BardiyaAk]

• The HDist function (Hausdorff Distance) was an experimental method, and at the moment, it's not being used. I would not worry about the implementation of that in OpenCL. It will be helpful to have it in the future, but not now.
• I believe the main problem with OpenCL version of the code is that the parallelization for matching doesn't happen in it correctly (either it's not written, or the code has to be heavily refactored). --> Per frame optimization took me about 5/6 times longer on OpenCL-compiled version compared to Cuda-compiled version (Suggestion: compile the code in both format, and run the matching process on 1 random frame and compare the time)
• PSO can benefit from the parallelization on Cuda, but the code for that is not being used. The main reason was I wasn't able to access the global cost function defined on the CPU from the PSO Cuda kernel.