assafelovic / gpt-researcher

LLM based autonomous agent that conducts in-depth web research on any given topic
https://gptr.dev
Apache License 2.0
14.51k stars 1.91k forks source link

Enable usage of docker networks by using next.js env variables #878

Open olafgeibig opened 2 weeks ago

olafgeibig commented 2 weeks ago

Problem I want to run gpt-researcher on my home server but port 8000 is already taken and moreover I don't like to expose the API to my host network. The compose app could utilize a docker network to avoid that but for that the backend URLs must be configurable. Currently there is no way to set the gpt-researcher backend URLs that are used by the frontend but this is a good practice and there are canonical env variables for this purpose in next.js.

Solution For my deployment case I can modify the docker-compose file so that the backend is exposed not on the host but on a docker network and my reverse proxy is taking care to forward from the proxy host (researcher.example) to the respective docker network host names: gpt-researcher and gptr-nextjs. But for this I need that the frontend app is using the canonical env variables (NEXTPUBLIC* see next.js docs to construct the URLs for the backend that I set in the compose file like this:

    environment:
      - CHOKIDAR_USEPOLLING=true
      - NEXT_PUBLIC_API_URL=http://researcher.example/api
      - NEXT_PUBLIC_WS_URL=ws://researcher.example/ws

For the default compose file it could simply be http://localhost:8000/api

Alternatives Nothing I know except exposing port 8000 on the host network which is undesirable.

ElishaKay commented 2 weeks ago

Good call @olafgeibig

Agreed.

The final step would be to leverage the NextJS env variable anywhere within frontend/nextjs where localhost:8000 is defined explcitly.

Screen Shot 2024-10-02 at 7 53 08

In other words, these 2-3 cases should leverage the getHost function (which leverages the NextJS env)

Love the idea of putting the NextJS env in the docker-compose :) i.e. If it can work out of the box that is ideal & users should be able to easily see (in 1 place) what services are sitting where & what's pointing where

Would be happy to collab with you to take this to Production.

Feel comfortable to create a PR with your ideas & I can follow up with the above commits.