search

kbressem / medAlpaca

LLM finetuned for medical question answering
GNU General Public License v3.0
484 stars 56 forks source link

[question] is this model usable with llama.cpp ? #14

Closed Mahgozar closed 1 year ago

kbressem commented 1 year ago

I have not tried it and no experience with llama.cpp. But as it is the same architecture as Alpaca, I assume it should be usable. Probably has to be converted first.

s1lvester commented 1 year ago

@Mahgozar yes it is.

Either convert the 7b-8bit / 13b-8bit Model to ggml yourself, using the scripts form the llama-cpp Respo or find a suitable model on HF Hub - there are at least 3 by now.