search

logancyang / obsidian-copilot

THE Copilot in Obsidian
https://www.obsidiancopilot.com/
GNU Affero General Public License v3.0
2.95k stars 204 forks source link

how about Llamafile? #613

Open RickySupriyadi opened 1 month ago

RickySupriyadi commented 1 month ago

I'm no expert but i would like to hear many from you all. Since this plugin will support mobile... i thought maybe supporting llamafile might useful? especially there are Small Language Models nowdays that might work in mobile devices also.

miwagner1 commented 1 month ago

I think the openai format for embedding is not working for llamafile and via openwebui proxy https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/server/README.md#api-endpoints I was able to use a llamafile with openai format http://127.0.0.1:8080/v1

{"function":"log_server_request","level":"INFO","line":2781,"method":"POST","msg":"request","params":{},"path":"/embeddings","remote_addr":"","remote_port":-1,"status":404,"tid":"17594334553008","timestamp":1725650100} {"function":"log_server_request","level":"INFO","line":2781,"method":"POST","msg":"request","params":{},"path":"/embeddings","remote_addr":"","remote_port":-1,"status":404,"tid":"17594334547440","timestamp":1725650100}

RickySupriyadi commented 1 month ago

that's cool thanks, I'll try that, Small Language Models time. I would love to see what Andrej Karpathy said one day we might have 1B model with strong cognitive core can surpass bigger model,