search

ParisNeo / ollama_proxy_server

A proxy server for multiple ollama instances with Key security
Apache License 2.0
249 stars 36 forks source link

Support for real_ip_header #15

Open Joly0 opened 1 month ago

Joly0 commented 1 month ago

Not sure, if this is something the proxy can support, but i noticed when running it inside docker, that the ip the requests are comming from, are all showing the same ip (not sure which one, i guess its the docker host ip; Ip of proxy is internal 172.17.0.18 and the requests come from 172.17.0.1) I have an nginx reverse proxy in front of it and configured it correctly, to forward the real ip header, but the ollama_proxy_server apparentely doesnt recognizes that or doesnt receive it properly. I am not sure. Maybe i am missing something, and there needs to be some additional configuration.

ParisNeo commented 1 month ago

Hi. If you already have a reverse proxy, why would you even use this proxy? I have built it a while ago when ollama used to be unable to handle multiple generation requests but today this is not really relevant (i don't really use it anymore), the only use now may be just to secure access with multiple api keys or to use it as a reverse proxy in which case, yo udon't need that if you already have one.

Joly0 commented 1 month ago

I use it for load balacing between multiple ollama instances and for authentication.