jkawamoto / ctranslate2-rs

Rust bindings for OpenNMT/CTranslate2

https://docs.rs/ct2rs

MIT License

22 stars 3 forks source link

Support for CUDA #24

Closed filiptibell closed 5 months ago

filiptibell commented 7 months ago

Similar to #23 , would you be open to adding support for building for CUDA? I have not yet figured out how to do this (don't have much experience with cc/cxx/build scripts) but this would be great for deploying on infra such as HF Inference Endpoints.

jkawamoto commented 7 months ago

It would be nice if we supported CUDA. However, I'm a Mac user, and testing with NVIDIA's GPUs is not easy for me...

filiptibell commented 7 months ago

It seems like bindgen_cuda has some logic for finding proper CUDA paths to link and it can probably be added as an optional dependency similar to intel-mkl-src in #23 . It needs an attached gpu to build but seems like that is also pretty common. I can take a look at this in a week or so

jkawamoto commented 6 months ago

37 added support for CUDA and cuDNN. I agree that it would be beneficial for this package to automatically find and link the CUDA and cuDNN libraries. However, as an initial step, #37 merely checks whether the `CUDA_TOOLKIT_ROOT_DIR` environment variable is set.

jkawamoto / ctranslate2-rs

Support for CUDA #24

37 added support for CUDA and cuDNN. I agree that it would be beneficial for this package to automatically find and link the CUDA and cuDNN libraries. However, as an initial step, #37 merely checks whether the CUDA_TOOLKIT_ROOT_DIR environment variable is set.

37 added support for CUDA and cuDNN. I agree that it would be beneficial for this package to automatically find and link the CUDA and cuDNN libraries. However, as an initial step, #37 merely checks whether the `CUDA_TOOLKIT_ROOT_DIR` environment variable is set.