Closed surenderssm closed 4 months ago
@surenderssm thanks for opening an issue! I just added the phi 3 3.7b model. see https://sozercan.github.io/aikit/premade-models
this is not the 128k version but the 4k one. if you are interested in contributing, it'll be similar to #210 but for 128k gguf file. I didn't see the gguf for the 128k one at the time
Looks like llama.cpp doesn't support 128k yet due to longrope scaling type: https://github.com/ggerganov/llama.cpp/issues/6849
Looks like https://github.com/ggerganov/llama.cpp/pull/7225 added support for 128k version. aikit v0.9.0 includes the llama.cpp version that has this feature.
What kind of request is this?
New feature
What is your request or suggestion?
I would like to add a support for phi-3 mini model.
https://huggingface.co/microsoft/Phi-3-mini-128k-instruct
I am very new to this ecosystem, however I have a strong desire to learn and contribute in this space, pointers or references will be enough for me to get started. Let me know your thoughts!
Are you willing to submit PRs to contribute to this feature request?