Separate proxy settings for dig dug tunnel and intern Proxy server

[bartoszkaczorek]

Currently intern Proxy server seems to be using the same proxy settings as tunnel (defined in tunnelOptions). This is not always wanted behavior - i.e. I would like to specify different proxy settings for my tunnel and different for intern Proxy server (I'd like it to not use proxy at all).

[csnover]

I don’t understand this ticket, since Intern’s Proxy doesn’t make outgoing requests. Could you please provide more specific detail about what you think is happening? Thanks.

[bartoszkaczorek]

Well I'm running tests in a browser using selenium server which is hosted on a different server (and I need to use proxy with authentication to connect with it) - so I need to configure my tunnel proxy settings to get it working, but then the same proxy settings are applied to intern's Proxy which makes my tests error out - I believe it's happening due to the same upstream bug as here: https://github.com/theintern/digdug/issues/17. Even if the mentioned upstream bug would get fixed and my tests start work ok with that setup I would still want to separate those settings since intern's Proxy is on my localhost I actually don't need any proxy settings here applied. Does that makes sense? By the way - that issue did not occured in intern 2, it started to be an issue after update to intern 3.

[jason0x43]

As @csnover pointed out, the Intern proxy doesn't make outgoing requests, and from looking at #519, @bartoszkaczorek's issue isn't with the Intern proxy, but with the WebDriver server.

The current assumption is that if a proxy server is being used, it will be needed for every request to an external URL. In other words, the system running Intern is assumed to be behind the firewall, and everything else is assumed to be outside. If this is the case, then I'm not sure what benefit there in separating the Tunnel proxy config from Leadfoot's.

[bartoszkaczorek]

I am currently using below intern configuration options to run intern tests on Sauce Labs (some of the real values are mocked):

    "tunnel": "dojo/node!digdug-null-sauce-labs-tunnel/lib/NullSauceLabsTunnel",
    "tunnelOptions": {
        "clientUrl": "https://my-sauce-username:my-sauce-access-key@my-sauce-labs-server/wd/hub",
        "proxy": "http://my-proxy-server:80",
        "username": "my-sauce-username",
        "accessKey": "my-sauce-access-key"
    },

You can take a look at the NullSauceLabsTunnel implementation here if you want (in short it's a Sauce Labs tunnel with fixed proxy behind authentication issue): https://github.com/bartoszkaczorek/digdug-null-sauce-labs-tunnel/blob/master/lib/NullSauceLabsTunnel.js

Now if I use these options with intern 3.0.6 it will instantiate intern server as (it will use proxy from tunelOptions):

var server = new Server(tunnel.clientUrl, {
    "proxy" : "http://my-proxy.server:80"
});

This then leads to below error when I try running test:

Created session chrome 41 on WINDOWS (123456)
SUITE ERROR
UnknownError: [POST https://(redacted)@my-sauce-labs-server/wd/hub/session/123456/url / {"url":"http://my-intern-machine-hostname:8090/__intern/client.html?someParametersAddedHere"}] An unknown server-side error occurred while processing the command.
  at runRequest  <node_modules\intern\node_modules\leadfoot\Session.js:88:40>
  at <node_modules\intern\node_modules\leadfoot\Session.js:109:39>
  at new Promise  <node_modules\intern\node_modules\dojo\Promise.ts:411:3>
  at ProxiedSession._post  <node_modules\intern\node_modules\leadfoot\Session.js:63:10>
  at ProxiedSession.Session.get  <node_modules\intern\node_modules\leadfoot\Session.js:575:15>
  at <node_modules\intern\lib\ProxiedSession.js:86:25>
  at getFinalValue  <node_modules\intern\node_modules\dojo\Promise.ts:462:21>
  at <node_modules\intern\node_modules\dojo\Promise.ts:482:10>
  at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
  at runCallbacks  <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
  at Command.target.(anonymous function) [as get]  <node_modules\intern\node_modules\leadfoot\Command.js:610:11>
  at ClientSuite.lang.mixin.run  <node_modules\intern\lib\ClientSuite.js:149:6>
  at runWithCatch  <node_modules\intern\lib\Suite.js:314:21>
  at next  <node_modules\intern\lib\Suite.js:322:18>
  at <node_modules\intern\lib\Suite.js:345:6>
  at new Promise  <node_modules\intern\node_modules\dojo\Promise.ts:411:3>
  at runTests  <node_modules\intern\lib\Suite.js:278:12>
  at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
  at runCallbacks  <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
  at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
[Suite chrome 41 on WINDOWS - unit tests FAILED
UnknownError: [POST https://(redacted)@my-sauce-labs-server/wd/hub/session/123456/url / {"url":"http://my-intern-machine-hostname:8090/__intern/client.html?someParametersAddedHere"}] An unknown server-side error occurred while processing the command.
  at runRequest  <node_modules\intern\node_modules\leadfoot\Session.js:88:40>
  at <node_modules\intern\node_modules\leadfoot\Session.js:109:39>
  at new Promise  <node_modules\intern\node_modules\dojo\Promise.ts:411:3>
  at ProxiedSession._post  <node_modules\intern\node_modules\leadfoot\Session.js:63:10>
  at ProxiedSession.Session.get  <node_modules\intern\node_modules\leadfoot\Session.js:575:15>
  at <node_modules\intern\lib\ProxiedSession.js:86:25>
  at getFinalValue  <node_modules\intern\node_modules\dojo\Promise.ts:462:21>
  at <node_modules\intern\node_modules\dojo\Promise.ts:482:10>
  at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
  at runCallbacks  <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
  at Command.target.(anonymous function) [as get]  <node_modules\intern\node_modules\leadfoot\Command.js:610:11>
  at ClientSuite.lang.mixin.run  <node_modules\intern\lib\ClientSuite.js:149:6>
  at runWithCatch  <node_modules\intern\lib\Suite.js:314:21>
  at next  <node_modules\intern\lib\Suite.js:322:18>
  at <node_modules\intern\lib\Suite.js:345:6>
  at new Promise  <node_modules\intern\node_modules\dojo\Promise.ts:411:3>
  at runTests  <node_modules\intern\lib\Suite.js:278:12>
  at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
  at runCallbacks  <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
  at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>[0m
0/0 tests failed
No unit test coverage for chrome 41 on WINDOWS[0m
[chrome 41 on WINDOWS: 0/0 tests failed; fatal error occurred[0m

This error does not occur if intern server is instantiated without proxy options (and this is how I currently fix this, but there's no option to fix it without changing intern source files which is sad) so this is basically where I come from.

[jason0x43]

Hmmm...I see. Presumably the assumption is that a proxy wouldn't proxy connections to localhost, which is what Server uses for SauceLabs. Assuming that's not the case, though, then it does make sense for Server to not use the proxy.

Just out of curiosity, do you need to specify clientUrl? If you're using a non-standard Sauce host, you should just be able to set hostname to my-sauce-server.

[bartoszkaczorek]

I think our proxy does indeed bypass localhost connections, but in our intern config proxyUrl is not pointing to localhost but to machine hostname actually (setting it to localhost would not work with Sauce Labs for us):

"proxyPort": 8090,
"proxyUrl": "http://my-intern-machine-hostname:8090",

Regarding clientUrl - I think the reason I used it was only because I thought it's more readable that way as it seemed a bit confusing to guess if hostname, protocol & port parameters are used to build clientUrl or maybe rather proxy url and it's easier to spot misspellings when looking at whole URL.

[dylans]

PR at https://github.com/theintern/intern/pull/519