CUDA

[DmitriyRyazanovsky]

Hello, I use your service, but I do not understand how to use your service together with CUDA, is there a guide or an article where I could familiarize myself with the contents?

[yalue]

CUDA support is in the documentation on pkg.go.dev. Copied from that link:

1. Call NewSessionOptions() to create a SessionOptions struct.
2. Call NewCUDAProviderOptions() to obtain a CUDAProviderOptions struct.
3. Call the CUDAProviderOptions struct's Update(...) function to pass a list of settings to CUDA. (See the documentation on the Update() function.)
4. Pass the CUDA options struct pointer to the sessionOptions' AppendExecutionProviderCUDA(...) function.
5. Call the Destroy() function on the CUDA provider options.
6. Call NewAdvancedSession(...), passing the SessionOptions struct to it.
7. Call the Destroy() function on the SessionOptions struct.

On top of all of that, you'll need to make sure to be using a build of onnxruntime.dll or onnxruntime.so that was compiled with CUDA support, and be running on a system with the correct version of CUDA (only CUDA 11.8 works with onnxruntime 1.16.1). It's a long process, but that's just a limitation of onnxruntime itself.

All of this is explained in the project README and the documentation on pkg.go.dev: https://pkg.go.dev/github.com/yalue/onnxruntime_go. Just search them for the term "CUDA".