Does this support CUDA?

skeskinen / bert.cpp

ggml implementation of BERT

MIT License

463 stars 58 forks source link

Does this support CUDA? #40

Open SpaceCowboy850 opened 10 months ago

SpaceCowboy850 commented 10 months ago

I have seen where I can set the GGML_USE_CUBLAS, and I can follow the few #defines that activate the code, but the tensors are all on the CPU. I'm not seeing in bert.cpp where it would transfer the model or the inputs to the GPU.

Is this just not functioning yet?

skeskinen commented 10 months ago

I haven't done anything towards CUDA support. How easy/difficult it is to implement depends on ggml, I guess