search

JuliaGPU / CUDA.jl

CUDA programming in Julia.
https://juliagpu.org/cuda/
Other
1.22k stars 221 forks source link

cuDNN: Provide wrappers for the declarative API #2266

Open maleadt opened 8 months ago

maleadt commented 8 months ago

As part of cuDNN 9.0, the old imperative API has been deprecated, while moving towards a new, graph-based declarative API. That means that our wrappers will have to be rewritten. As I'm not familiar with machine learning / NNlib.jl (which I guess these wrappers would need to be compatible with), I'm hoping that people from the ML community could be able to help with this.

cc @ToucheSir @CarloLucibello @mcabbott @DhairyaLGandhi

ToucheSir commented 8 months ago

I looked into the graph APIs a few months ago for perf- and flexibility-related reasons. The problem I see is that they are lower-level and require far more manual work than the imperative API. There also appear to be exactly zero tutorials or in-depth learning resources on how to use the graph API to accomplish the same functionality as the imperative API? This is less of a problem outside of Julia because NVIDIA provides https://github.com/NVIDIA/cudnn-frontend, but obviously we can't make use of that higher-level C++ layer in Julia.

All this to say that I don't foresee us being able to migrate all of cuDNN.jl to the graph API any time soon. The best we could probably do is fix all the functionality using removed APIs to use just deprecated ones instead before cuDNN 9.0 becomes widespread. In parallel, a good first step to allow people to help with porting imperative -> graph code would be to generate bindings for the graph API (which IIRC existed in v8 as well). Grepping through res/wrap those do not appear to exist right now.

maleadt commented 8 months ago

Generating bindings is trivial, I'll do that as part of the PR upgrading to cuDNN 9.0 here.

The best we could probably do is fix all the functionality using removed APIs to use just deprecated ones instead before cuDNN 9.0 becomes widespread.

Yep, I don't think we have a problem right now, I just wanted to get some people involved/interested so that we have enough time to adapt before the imperative API inevitably gets removed 🙂

maleadt commented 8 months ago

The update to CUDNN 9 was easy enough: https://github.com/JuliaGPU/CUDA.jl/pull/2267

That PR includes updated wrappers, including the graph API: https://github.com/JuliaGPU/CUDA.jl/blob/master/lib/cudnn/src/libcudnn.jl