search

n4ze3m / page-assist

Use your locally running AI models to assist you in your web browsing
https://chromewebstore.google.com/detail/page-assist-a-web-ui-for/jfgfiigpkhlkbnfnbobbkinehhfdhndo
MIT License
1.42k stars 137 forks source link

embeddings fail #103

Open oatmealm opened 4 months ago

oatmealm commented 4 months ago

Not sure if to report a problem or not. Does embedding of knowledge base only works locally? I'm running models from a remote ollama but I see in the console app calls for the embeddings are local (127.0.0.1) even though I set it to use an embeddings model on the same remote server

n4ze3m commented 4 months ago

Hey, it was a bug that I hadn't noticed. I will fix it in the next release

oatmealm commented 4 months ago

Was going to test. I see it's calling the remote for embeddings, but all calls return 403... not sure what's happening since the server's origin is set and other client (openweb ui) work fine with the same server... image

Also curl from the maching where page-assist is running:

curl http://10.0.0.56:11434/api/generate -d '{
  "model": "llama3:instruct",
  "prompt": "Why is the sky blue?"
}'
{"model":"llama3:instruct","created_at":"2024-06-10T08:20:58.692906Z","response":"What","done":false}
{"model":"llama3:instruct","created_at":"2024-06-10T08:20:58.726031Z","response":" a","done":false}
{"model":"llama3:instruct","created_at":"2024-06-10T08:20:58.760291Z","response":" great","done":false}
{"model":"llama3:instruct","created_at":"2024-06-10T08:20:58.795941Z","response":" question","done":false}
{"model":"llama3:instruct","created_at":"2024-06-10T08:20:58.83106Z","response":"!\n\n","done":false}
oatmealm commented 4 months ago

BTW, localhost, with page_assist exact settings (unchanged) works

n4ze3m commented 4 months ago

Hey, can you enable Custom Origin URL in the advanced URL settings? Check out these docs https://github.com/n4ze3m/page-assist/blob/main/docs/connection-issue.md#solutions . This may solve the issue. Please let me know if you still encounter an error

oatmealm commented 4 months ago

I'm pretty sure I did, but anyways. I works now. Before setting origina only embeddings api was failing btw... image

SunnyOd commented 2 months ago

Not sure if it's related but when I go to settings > RAG and select an embeddings model, it's not saving it. So when I try to "chat with current page" it throws and error saying "please set an embedding model"

Standard LLM inference seems to work fine, without issue. It's just the RAG/embed issue

I'm on firefox 126.0 with latest plugin, and running on Linux Fedora 38

I can open a new issue if needed?

n4ze3m commented 2 months ago

Hey @SunnyOd , this is not an issue. Can you set an embedding model on the RAG settings page like this? image

SunnyOd commented 2 months ago

Thanks that got it! I don't think I was using the right save button - I wasusing the one at the bottom of the page previously and it wasn't saving. I downloaded the model from you image and pressed save just below in the same section and it's working ! :)

Love this project btw, think integrating it into the browser is a fantastic idea instead of a webpage as usual. Simplicity of it is great too and makes RAG accessible with search so simply - killed lots of birds with one stone for me :)

I love what you've done with the prompts too btw, but maybe it's worth integrating with Fabric? Not sure if you've come across it but the project is fantastic and helps store/access custom prompts from one location locally, then use them from the command line. It's super helpful. I can't help but think your plugin combined with that would be a marriage made in heaven and hopefully hopefully straightforward to implement? :thinking:

This is the github: https://github.com/danielmiessler/fabric Video about it: https://www.youtube.com/watch?v=0qlyCPvmH2Y

Cheers! Sunny

oatmealm commented 2 months ago

Yes I think more projects should pick fabric as a starting / default option. At best better than nothing :)