search

OpenNMT / CTranslate2

Fast inference engine for Transformer models
https://opennmt.net/CTranslate2
MIT License
3.06k stars 273 forks source link

DirectML Support #1615

Open gdiaz384 opened 5 months ago

gdiaz384 commented 5 months ago

Assume that I already followed Microsoft's instructions to Enable PyTorch with DirectML on Windows and the DirectML library loads correctly according to MS's example code.

If I wanted to use PyTorch + DirectML + Ctranslate2 + an AMD GPU, how would I do it? Does Ctranslate2 need any major updates? Shouldn't I just need to pass directml instead of cuda to device= as found in the quickstart?

What is the bottleneck to this not working right now?

Aside: MS's instructions for DirectML's Python library install pytorch==2.0 as of early 2024. MS's documentation suggests it only works with PyTorch 1.3, which seems to be incorrect? So, it should not be a matter of supporting PyTorch 1.x since DirectML runs on 2.x in practice.

faker2048 commented 4 months ago

I need DirectML too.

minhthuc2502 commented 4 months ago

Currently, Ctranslate2 do not support DirectML. To support this, the new implementation for this backend is required.