search

andrewnguonly / Lumos

A RAG LLM co-pilot for browsing the web, powered by local LLMs
MIT License
1.42k stars 102 forks source link

WebLLM #156

Open sublimator opened 7 months ago

sublimator commented 7 months ago

Stumbled back on this repo, and was looking at the changes/issues, and then onwards to Web Speech stuff.

Something in the back of my head for a while is WebLLM. Basically adding a polyfill to the page in the MAIN context that adds an LLM api (accessible by webpages)

I really haven't thought about it much at all

  1. Is it even worth doing?
  2. What would it look like?
  3. Requesting specific models? model families/classes ?

Shrug

andrewnguonly commented 7 months ago

Basically adding a polyfill to the page in the MAIN context that adds an LLM api (accessible by webpages)

Can you explain with a use case? Not sure if I'm following.

sublimator commented 7 months ago

Haha, I am not sure I am following, myself! Like I said, I haven't thought this through much!

WebApps that want to use an LLM? All the advantages of web based apps. No need to install etc

I mean, I guess you just configure an app to use a localhost openai compatible url and configure allow origins etc as an alternative.

I guess it really just saves building that permissions UI in multiple places? Anything else? Shrug??