Proxy Support - Githubissues

helloanoop commented 9 months ago

User should be able to use proxy at the collection level and app level

Tasks

[x] Store Collection Proxy Settings in bruno.json
[x] Electron layer updates to pickup apply proxy settings
[x] Bump axios at electron layer to 1.5
[x] UI support for configuring proxy settings
[x] CLI support for using proxy settings
[x] Bump axios at cli layer to 1.5
[x] Use proxy config in collection runner in UI
[x] socks5 proxy support
[ ] Support for proxy bypassing by hostname
[ ] Serialize and Deserialize proxy config during import and export of bruno collection as files
[ ] Proxy settings at app level.
[ ] Use OS proxy configuration if found https://github.com/usebruno/bruno/issues/224#issuecomment-1753076833

Below is the axios config to use proxy

proxy: {
    protocol: 'https',
    host: '127.0.0.1',
    // hostname: '127.0.0.1' // Takes precedence over 'host' if both are defined
    port: 9000,
    auth: {
      username: 'mikeymike',
      password: 'rapunz3l'
    }
  }

mirkogolze commented 9 months ago

The proxy must only be enabled for the request, if the request URL does not match the proxy exceptions

mirkogolze commented 9 months ago

Hi @helloanoop, I am not sure if proxy support is perfectly placed on collection level. Someone needs another proxy than another developer. So it is in my current project. In my eyes it should be a global configuration in Bruno preferences. So everybody can enable it for its personal situation. As it is in Postman. And the usage of the CLI will mostly be in CI/CD. Here you have different proxy configuration. So the global env Variables http_proxy, https_proxy and no_proxy should be enough. Or I need any other configuration for runtime environment. And the third thing is. We are currently using axios in our custom scripts to run the OIDC authentication request to get a token. So I would need to have access to the proxy configuration inside global object bru.

Your are too fast with implementing.

We should set the global env variables and use https://github.com/Rob--W/proxy-from-env/blob/master/index.js#L24 to check need for proxy, as axios is doing. It's a pity that the proxy login can't be done via environment variables. So we have to do something in the code and can't delegate the whole proxy work.

helloanoop commented 9 months ago

Hey @mirkogolze !

I am not sure if proxy support is perfectly placed on collection level. Someone needs another proxy than another developer. So it is in my current project. In my eyes it should be a global configuration in Bruno preferences. So everybody can enable it for its personal situation. As it is in Postman.

I feel we should support it at both levels. If a proxy config is defined at collection level, then the runtime uses collection level proxy, else it fallbacks to global proxy config. This has been asked by community to be supported in Postman - https://github.com/postmanlabs/postman-app-support/issues/4739 What do you think?

And the usage of the CLI will mostly be in CI/CD. Here you have different proxy configuration. So the global env Variables http_proxy, https_proxy and no_proxy should be enough. Or I need any other configuration for runtime environment.

Yup agreed. We can provide flags to pass proxy config via cli arguments.

We are currently using axios in our custom scripts to run the OIDC authentication request to get a token. So I would need to have access to the proxy configuration inside global object bru.

If I understand you correctly, you need to use proxy only for one request in the collection right? I think we can support a req.setProxy() api so that you can apply proxy only for certain requests. The proxy values can be picked up from env config. I recently added similar functionality to disable redirect-follows at a request level. You can take a look at this issue and commits under it - https://github.com/usebruno/bruno/issues/200 What do you think?

Your are too fast with implementing.

Yes 😄, This was a bit rushed. Need to slow down.

rmueller83 commented 9 months ago

For me, the current proxy support does not work. When I analyze curl and Postman with Wireshark, they both issue a CONNECT request to the proxy server (see https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT )

But Bruno currently creates a GET request which results in an error page from the proxy.

Edit: seems to be a library issue: https://github.com/axios/axios/issues/4531 https://github.com/axios/axios/issues/5256

Edit2: it works now thanks to https://github.com/usebruno/bruno/pull/374

mirkogolze commented 9 months ago

Hi @helloanoop, please merge https://github.com/usebruno/bruno/pull/468 then I go on with my pull request.

Lotti commented 9 months ago

I think it can be cool to have also the ability to use OS proxy configuration (in windows is possible to set a proxy system wide, that most of the applications ignore :D )

helloanoop commented 9 months ago

I think it can be cool to have also the ability to use OS proxy configuration (in windows is possible to set a proxy system wide, that most of the applications ignore :D )

We may be limited now by axios https://chat.openai.com/share/e36d054d-0428-49da-9595-755da83ceb63

There is still a possibility of Bruno automatically reading OS proxy configuration and passing it on to axios. If it works, we can also provide a toggle to use os proxy configuration.

Lotti commented 9 months ago

https://www.npmjs.com/package/get-proxy-settings This package is old and not maintained but it could be a good starting point to study the problem.

Normally under linux you should just honor env vars http_proxy, https_proxy, no_proxy (easy). Problems arrives with Windows (i can give you some powershell script to obtain the values but don't know how to reproduce them in javascript) and macos (which I ignore how it manages a system wide proxy).

UPDATE from axios npm page:

// `proxy` defines the hostname, port, and protocol of the proxy server.
  // You can also define your proxy using the conventional `http_proxy` and
  // `https_proxy` environment variables. If you are using environment variables
  // for your proxy configuration, you can also define a `no_proxy` environment
  // variable as a comma-separated list of domains that should not be proxied.
  // Use `false` to disable proxies, ignoring environment variables.
  // `auth` indicates that HTTP Basic auth should be used to connect to the proxy, and
  // supplies credentials.
  // This will set an `Proxy-Authorization` header, overwriting any existing
  // `Proxy-Authorization` custom headers you have set using `headers`.
  // If the proxy server uses HTTPS, then you must set the protocol to `https`.
  proxy: {
    protocol: 'https',
    host: '127.0.0.1',
    // hostname: '127.0.0.1' // Takes precedence over 'host' if both are defined
    port: 9000,
    auth: {
      username: 'mikeymike',
      password: 'rapunz3l'
    }
  },

It seems that axios already honor env vars.

Tilogorn commented 7 months ago

I found an issue with Node.js + axios concerning Proxy, HTTPS and proper HTTP CONNECT. I am technically not deep enough into the topic, but this issue may give more details: https://github.com/axios/axios/issues/4531

Injecting e.g. hpagent with proper set proxy configuration into axios as httpsAgent solved the issue.

As I observed the same errors and timeouts in Bruno I got in my own Node.js scripts, I suspect Bruno is also missing such a layer for Proxy + HTTPS. I still have to switch to Insomnia for such edge cases.

This is not a complaint - more a feature request/raising awareness - because I'd like to move completely to Bruno as soon as possible and it probably doesn't just affect me. :-)

mirkogolze commented 7 months ago

Hi @Tilogorn, I am not sure what your use case is. When you use the proxy configuration for collections or global configuration all different proxy variations should work. All is implemented with the httpAgents (https://github.com/usebruno/bruno/blob/main/packages/bruno-electron/src/ipc/network/index.js#L144). Only when you use axios directly in your custom scripts, you don't benefit of the configuration of the request in ipc/network. Then you must do this for your own.

Tilogorn commented 7 months ago

Hi @mirkogolze

with "my scripts" I was rereferring to a completely different Node.js project that I am developing where I encountered problems with axios connecting to the outside https-world. My company proxy always returned 502/Read Error (Squid). It got fixed by using hpagent as httpsAgent (tested https-proxy-agent succesfully as well).

As I wanted to execute the same requests in Bruno that I did via code in my project, I set everything up - also the Bruno proxy settings - but I got the same 502/Read Error as ealier in Node.js before injecting hpagent/https-proxy-agent. That's why I assumed you missed this fix and it would be interesting for you to know about it.

Now that you showed me that you already use https-proxy-agent to work around this issue, I am quite confused why I experience this behavior.

I am now at a point where my custom Node.js scripts work but Bruno with the same setup does not and I am not sure how to debug that further.

mirkogolze commented 7 months ago

When you have time you could try that out with Bruno. To enable local Bruno development is quite easy: https://github.com/usebruno/bruno/blob/main/contributing.md

You could switch to hpagent and try if it is working with your companies proxy.

gajanak commented 2 months ago

It would be nice, if ProxySettings on CollectionLevel can use Environment Variable ? This would easy switch between different proxyServers in different Setups ?

wamfous commented 1 month ago

Hi, bruno looks very promising but I currently can't get it to work from behind my corporate proxy. I've tried adding the proxy configuration to global or collection settings but all of my requests time out. Is there some way to troubleshoot this? Thanks

mirkogolze commented 1 month ago

You can try to look into the dev tools inside Bruno and see if the console is logging something. Second is you can start Bruno from shell. On a windows machine I use "cmd" and got to folder "C:\Users[user]\AppData\Local\Programs\bruno" and start the bruno.exe. Then you should see error messages.

wamfous commented 1 month ago

Thanks for the quick reply. The console shows two errors directly on startup

Failed to load resource: net::ERR_FILE_NOT_FOUND
Error: Error invoking remote method 'renderer:fetch-notifications': Error while fetching notifications!

I'm not sure if that can explain anything. And running from the command line gives a big object dump after the timeout. I won't copy it here but I'm not sure what to look for

lanes100 commented 1 month ago

Thanks for the quick reply. The console shows two errors directly on startup
Failed to load resource: net::ERR_FILE_NOT_FOUND
Error: Error invoking remote method 'renderer:fetch-notifications': Error while fetching notifications!
I'm not sure if that can explain anything. And running from the command line gives a big object dump after the timeout. I won't copy it here but I'm not sure what to look for

I am running into the same error on startup. It's actually blocking me from making https requests where the certificate is invalid even with SSL verification off. This is a huge bug that needs to get fixed ASAP See https://github.com/usebruno/bruno/issues/2391

wamfous commented 1 month ago

OK, I hadn't explored the HTTPS angle. Here are some new findings:

GET http://example.com and https://example.com time out from behind my corporate proxy with the proxy configuration.

GET http://example.com works from behind my corporate proxy if I disable the proxy configuration (it shouldn't). Is Bruno using the underlying system proxy configuration?

GET https://example.com gives me a 502 "Connection reset by peer" with proxy disabled

mirkogolze commented 1 month ago

No. There is no system configuration used. There are three ways to use proxy.

Configure on global level
Configure on collection level to overwrite global configuration
use HTTP_PROXY, HTTPS_PROXY, NO_PROXY environment variables. They are used by axios automatically. But this can influence the other settings.

dsculptor commented 5 days ago

Can we not save the proxy password in the bruno.json file? Is it possible to use a variable from environment as the proxy password - if that is the case, then we don't need to invent anything special to avoid leaking proxy password in a git repo.

usebruno / bruno

Proxy Support #224