glai - GGUF LLAMA AI - Package for simplified model handling and text generation with Llama models quantized to GGUF format. APIs for downloading and loading models automatically, includes a db with models of various scale and quantizations. With this high level API you need one line to load the model and one to generate text completions.
Default model dir argument for initializing ModelDB was changed to global package dir to enable using models from the package dir in all projects importing one of the glai classes.
However it's still possible to provied a different dir path to be used for this purpose (with or without copying packages model data)
Now since it's reasonable to anticipate that some ggufs might be already downloaded in the default dir, for cases when user is trying to load modes from non-default model dir and the gguf is not present, before downloading it from url it should first check if it's not in the packages model dir and just copy the gguf
laelhalawani
commented
9 months ago
Default model dir argument for initializing ModelDB was changed to global package dir to enable using models from the package dir in all projects importing one of the glai classes. However it's still possible to provied a different dir path to be used for this purpose (with or without copying packages model data) Now since it's reasonable to anticipate that some ggufs might be already downloaded in the default dir, for cases when user is trying to load modes from non-default model dir and the gguf is not present, before downloading it from url it should first check if it's not in the packages model dir and just copy the gguf