We have only tested the llama 2 model series provided thought HF, but I see no reason why the implementation couldn't be generalised to support any HF model (including the firs llama series).
Additionally, you can extend the AbstractLanguageModel class to introduce support for any custom model deployments that you might have.
Hope that helps!
xxm1668
commented
9 months ago
We have only tested the llama 2 model series provided thought HF, but I see no reason why the implementation couldn't be generalised to support any HF model (including the firs llama series). Additionally, you can extend the AbstractLanguageModel class to introduce support for any custom model deployments that you might have. Hope that helps!