llama cpp integration?

[JoseConseco]

Are there any changes for llama.cpp integration? It is using quantized models - allowing running bigger models with less VRAM required ( https://github.com/ggerganov/llama.cpp#quantization) and you can put part of computation of CPU, part on GPU, if whole model cannot be loaded into GPU. It supports server mode too- https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md#testing-with-curl

[JoseConseco]

Ok, forget about it. From ollma docs I can see it is using llamacpp in background. And it supports manual addition of custom gguf models, with various params and etc.