search

jacoblee93 / fully-local-pdf-chatbot

Yes, it's another chat over documents implementation... but this one is entirely local!
https://webml-demo.vercel.app
MIT License
1.68k stars 307 forks source link

Feature request: make Wllama an option, next to WebLLM #23

Open flatsiedatsie opened 6 months ago

flatsiedatsie commented 6 months ago

Wllama is a browser-based version of Llama.cpp with low-level capabilities, and has a built-in embedding option too.

https://github.com/ngxson/wllama

While WebLLM only runs on WebGPU-enabled browsers, Wllama can run on all browsers. It could perhaps be a great fallback option? E.g. for Safari / Firefox?

jacoblee93 commented 6 months ago

Interesting! Would love to look into it!