Router can not call subgraphs behind proxy

[rajendersaini]

Component(s)

router

Component version

latest

wgc version

0.61.0

controlplane version

none

router version

latest

What happened?

If possible, please create a PR with a failing test to illustrate the issue clearly. Otherwise, please attach a minimum reproduction through a GitHub repository that includes essential information such as the relevant subgraph SDLs. Please also make sure that the instructions for the reproduction are clear, tested, and fully accurate.

Description

The local router cannot call graphs behind proxy.

Steps to Reproduce

1. Run router with local settings.
2. Run your local proxy server so that all the traffic goes through local-proxy server
3. Open router endpoint in browser. http://localhost:3002
4. Try running any query.

   Expected Result

   Query should be successful

   Actual Result

   Query times out

Tried setting the environment variables http_proxy, https_proxy etc. but this did not help.

Environment information

Environment

OS: (e.g., "Ubuntu 20.04") Package Manager: pnpm, npm, yarn, etc Compiler(if manually compiled): (e.g., "go 14.2")

Router configuration

can use standard simple example. I can not share my file here.

Router execution config

version: "1"

dev_mode: true
execution_config:
  file:
    watch: true
    path: config.json

Log output

07:12:25 AM DEBUG core/websocket.go:129 Epoll is available {"component": "@wundergraph/router", "service_version": "dev"}
07:12:25 AM INFO core/router.go:497 Server listening and serving {"component": "@wundergraph/router", "service_version": "dev", "listen_addr": "0.0.0.0:3002", "playground": true, "introspection": true, "config_version": "fe917e25-6590-46a3-9a40-3a96df77d265"}
07:12:25 AM INFO core/router.go:1042 Watching config file for changes. Router will hot-reload automatically without downtime {"component": "@wundergraph/router", "service_version": "dev", "path": "router.json"}
 query 6862141230738056751 query($a: ID!){field(id: $a){name}}
07:13:12 AM DEBUG retrytransport/retry_transport.go:82 Retrying request {"component": "@wundergraph/router", "service_version": "dev", "retry": 1, "url": "https://location.production.tcc.services/graphql", "sleep": 2.105999953}
07:13:42 AM ERROR recoveryhandler/recovery.go:87 [Recovery from panic] {"component": "@wundergraph/router", "service_version": "dev", "time": "07:13:42 AM", "error": "runtime error: invalid memory address or nil pointer dereference", "request": "POST /graphql HTTP/1.1\r\nHost: localhost:3002\r\nAccept: application/json,

Additional context

No response

[github-actions[bot]]

WunderGraph commits fully to Open Source and we want to make sure that we can help you as fast as possible. The roadmap is driven by our customers and we have to prioritize issues that are important to them. You can influence the priority by becoming a customer. Please contact us here.

[AndreasZeissner]

Hi @rajendersaini thanks for reporting we will have a look.

[AndreasZeissner]

Hi @rajendersaini,

I made a quick check and was able to get the router talking over a proxy by using all upper case HTTPS_PROXY and HTTP_PROXY.

export HTTPS_PROXY="http://localhost:8082"
export HTTP_PROXY="http://localhost:8082"

It seems like that underlying go libraries are supporting it like this. Can you please try it out and let me know?

For wgc you might want to have a look here: https://github.com/wundergraph/cosmo/releases/tag/wgc%400.63.0 we recently released supporting proxies here. You can use the same variables as above.

[rajendersaini]

@AndreasZeissner thanks for the quick response and looking into this. Let me go back and recheck my entire environment and setup. I will post my results here.

[rajendersaini]

Ok - I can tell you the wgc utility works fine I pointed the utility to subgraph introspection endpoint which is sitting behind local-proxy. The wgc works fine.

But the ./router times out and does not work when I am trying to query the subgraph entity via router.

[AndreasZeissner]

Thanks for the feedback.

Is the router behaving different in any way? Would you mind sharing the logs of your router?

[rajendersaini]

@AndreasZeissner - I think I know the fix, the router

router/core/router.go

line no - 1645 is creating http.Transport, to take proxy into account we need to add proxy here

return &http.Transport{
    DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
        return dialer.DialContext(ctx, network, addr)
    },
    // The defaults value 0 = unbounded.
    // We set to some value to prevent resource exhaustion e.g max requests and ports.
    MaxConnsPerHost: 100,
    // The defaults value 0 = unbounded. 100 is used by the default go transport.
    // This value should be significant higher than MaxIdleConnsPerHost.
    MaxIdleConns: 1024,
    // The default value is 2. Such a low limit will open and close connections too often.
    // Details: https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/274
    MaxIdleConnsPerHost: 20,
    ForceAttemptHTTP2:   true,
    IdleConnTimeout:     opts.KeepAliveIdleTimeout,
    // Set more timeouts https://gitlab.com/gitlab-org/gitlab-pages/-/issues/495
    TLSHandshakeTimeout:   opts.TLSHandshakeTimeout,
    ResponseHeaderTimeout: opts.ResponseHeaderTimeout,
    ExpectContinueTimeout: opts.ExpectContinueTimeout,

// add proxy here ...... Proxy: http.ProxyFromEnvironment,

After adding this router works fine.

[rajendersaini]

Let me know if I need to send you a merge request! So that you know, I am pulling the latest code from the repo. I don't know if this code is already merged in the main as you suggested above!

[AndreasZeissner]

Hi @rajendersaini thank you for testing this,

Unfortunately there is nothing yet actively implemented in the router. It's a coincidence that we implemented the proxy handling for wgc see: https://github.com/wundergraph/cosmo/issues/822.

I did some testing and found out, that proto clients respect the HTTPS_PROXY variable by default. E.g. the poller https://github.com/wundergraph/cosmo/blob/main/router/pkg/controlplane/configpoller/config_poller.go.

You should see some logs in the router about configpoller/config_poller.go:78 No new router config available. Trying again .... These calls should be passed properly to your proxy and go against the controlplane or the cdn.

Other calls, unlike the one you fixed, might still not work. Those should be at least the default net/http calls.

Can you please use your adapted version for now, you might encounter issues with some of the features or still see blocked calls in the router logs, it would be great if you can drop them here.

I will go through the relevant parts and open up a pr the next time.

[AndreasZeissner]

Hi @rajendersaini,

Thank you again for initially bringing up the issue!

This went into the latest release: https://github.com/wundergraph/cosmo/releases/tag/router%400.108.0

Feel free to take a look.

[rajendersaini]

@AndreasZeissner - thank for fixing this fast.

[AndreasZeissner]

I'll close this ticket for now.

Feel free to open another one if needed!