holoviz / datashader

Quickly and accurately render even the largest data.
http://datashader.org
BSD 3-Clause "New" or "Revised" License
3.24k stars 363 forks source link

Add CUDA support for std and var reductions #1267

Closed ianthomas23 closed 10 months ago

ianthomas23 commented 11 months ago

This PR adds support for CUDA std and var reductions, both with and without dask. This means that all reductions now work on CPU and GPU, with and without dask, with the exception that some reductions do not yet support antialiasing but this is work in progress which will be completed soon.

The changes here rely on the use of a CUDA mutex that was recently added (#1196, #1212, #1217) which allows us to use more complicated multi-stage per-pixel operations and keep them atomic. This particular PR adds an extra layer of CUDA mutex usage via an enumerated type so that the CUDA mutex use is not just local to a single reduction (UsesCudaMutex.Local) but can also be global to the whole reduction.append pipeline (UsesCudaMutex.Global). It is necessary for std and var as their constituent sub-reductions which calculate the count, sum and variance need to be atomic so that they are not interrupted by another CUDA thread working on the same pixel.

codecov[bot] commented 11 months ago

Codecov Report

Merging #1267 (0d3e4a6) into main (812bcf6) will decrease coverage by 0.08%. The diff coverage is 60.00%.

@@            Coverage Diff             @@
##             main    #1267      +/-   ##
==========================================
- Coverage   83.47%   83.40%   -0.08%     
==========================================
  Files          35       35              
  Lines        9089     9110      +21     
==========================================
+ Hits         7587     7598      +11     
- Misses       1502     1512      +10     
Files Changed Coverage Δ
datashader/reductions.py 80.06% <51.72%> (-0.29%) :arrow_down:
datashader/compiler.py 89.17% <71.42%> (-0.86%) :arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more