Export methods to control ntlm-proxy

[bjowes]

The Node module API currently only supports running ntlm-proxy + cypress. For those who intend to use the ntlm-proxy with other solutions than cypress, it would be helpful if the Node module API included methods to start, stop and configure a ntlm-proxy.

Based on the discussion with @creage in #124

[bjowes]

@creage's request:

And, will your methods support picking random port and all the goodies of new multi-instance approach?

Yes it will.

Ideally, I'd like to have a way to start a new proxy (with new user/credentials) giving me back address:port I'll use in browser instance; the way to stop a particular proxy; and the way to stop them all (if multiple were started). Could we have such API please?

I am still considering the best approach, but I think the API methods will be something like this

• start ntlm-proxy
• send configuration to the ntlm-proxy (so you don't have to write the HTTP requests yourself)
• send reset to ntlm-proxy
• send quit to ntlm-proxy

Since there is no global store of ntlm-proxy instances, there is no way to globally stop all ntlm-proxy instances. The intent is that each node process should start only one ntlm-proxy instance, and stop it when done. If you see a need to handle multiple instances from within a single node process, please let me know the use case.

[creage]

Thank you for bringing this idea on the table - I'm sure such change will give great value to all QA community!

If you see a need to handle multiple instances from within a single node process, please let me know the use case.

Here is an example - a test of chat functionality, which includes running two isolated browser instances using different users, running in parallel. Both users would require their own ntlm-proxy.

Another example - a test of online editing of a document (like a google docs) - I need to see both users online at the same time, having first user making changes, reflected on second user's screen.

I'm not sure about Cypress, but PlayWright (which I'm using for automation for now) has a concept of Multiple contexts allowing to run multiple browser instances from within single node process. This is a perfect candidate/reason for starting multiple proxy instances from within one node process.

And, as an idea - if we can't store proxy instances in a global scope, we can probably inverse the control to user - giving him back PIDs of proxies he has started, and letting him store these in his scope. And later, accept a list of PIDs of proxies to stop.

[bjowes]

Hey @creage - I have published a beta for you to try out, cypress-ntlm-auth@3.1.0-beta.1 The docs aren't done, but the methods have XMLdoc so they should provide some intellisense. Simple example:

const cypressNtlmAuth = require("cypress-ntlm-auth");

async function run() {
  let proxy1 = await cypressNtlmAuth.startNtlmProxy();
  console.info(proxy1.ports);
  await proxy1.reset();
  let ntlmConfig = {
    ntlmHosts: ["localhost:5000"],
    username: "karl",
    password: "password",
    domain: "testdmn",
    workstation: "workstation",
    ntlmVersion: 2,
  };
  await proxy1.ntlm(ntlmConfig);

  let proxy2 = await cypressNtlmAuth.startNtlmProxy();
  console.info(proxy2.ports);
  let ntlmSso = {
    ntlmHosts: ["localhost:5006"],
  };
  try {
    await proxy2.ntlmSso(ntlmSso);
  } catch (err) {
    console.log("Got error from ntlmSso: " + err);
    console.log("Expected error on non-windows");
  }

  await proxy2.alive();
  await proxy1.stop();
  await proxy2.stop();
}

run();

[creage]

@bjowes I can't be more happy playing with this new API - it works super well!

The only method I'm missing is the ability to close all of open proxies, but I solve it by storing proxy instances in global array var, and close them manually on tests teardown.

Thank you so much!

[creage]

@bjowes apparently I was too excited with this new API, thus missed one important issue.

It seems that the last started proxy always wins, despite using different ports.

I start proxy#1 as user#1, I start browser#1 with this proxy. I start proxy#2 as user#2, I start browser#2 with this proxy.

I switch to browser#1, try to run requests - they all run as user#2.

[bjowes]

When starting up a proxy, it configures the node process to use it through the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables. This is adapted to the needs of cypress.

If your browsers are all running in the same node context, then the environment variables are overridden each time you start a new proxy, which would lead to the behavior you describe. Either you need to launch the browsers as separate processes, or you need to pass proxy settings to the browsers which they use instead of environment variables.

Could you clarify how you are launching the browsers?

[creage]

I start browsers from single node context, but I set their proxy addresses explicitly using proxy.ports.ntlmProxyUrl (and not env vars) that I receive from startNtlmProxy() method. And these URLs are different for every proxy I start, and these proxies are alive.

But still, requests to proxy#1are sent to the server as user#2.

[bjowes]

The it seems that your browser uses the environment variables to override the settings you pass in. Could you attempt to clear the HTTP_PROXY and HTTPS_PROXY environment variables before starting the browser? Just set process.env.HTTP_PROXY = undefined(and similar for HTTPS)

[bjowes]

I'm thinking that maybe I should add an option to startNtlmProxy() to disable modification of environment variables.

[creage]

Clearing env vars leads to 401 in requests. This is strange...

[bjowes]

Well, maybe not that strange. It means that the proxy setting you are passing in isn't used. Maybe something wrong with the variable name? That would also explain why the environment overrides it, as it shouldn't when you set the proxy explicitly.

[creage]

@bjowes it appears that Playwright does not support proxy-per-context yet, so, it is not a problem of your lib, but a launcher that I use.

There is a feature request on their repo https://github.com/microsoft/playwright/issues/3534, and until it is done - I cannot test multi-user scenarios 😢

But at least single-user NTLM scenarios are now WAAAY easier! Thank you one more time 🤝 !

[bjowes]

Great, then I will work on finalizing this with some docs and then make a release.

[bjowes]

Hey @creage - just a wild idea how you might be able to do multi instance testing with one proxy: Is it possible for you to make your chat server listen to multiple ports, or multiple hostnames? If this is possible, you can configure different credentials in the ntlm-proxy for each URL to the chat server. And even if you don't have native support for this, I would presume you can fake it by overriding the DNS using the hosts file.

Maybe this is something you already considered and discarded. Otherwise just ask if anything of the above is unclear.

[creage]

@bjowes sorry for late reply. You are right - DNS trick might do the job, but we are forced to use the 'real world' environments, and altering system settings is forbidden here.

And BTW, the missing feature in PlayWright is merged now, so I'm waiting for the release to test it.

P.S. And thanks to your new API, I was able to use your lib in webpack-dev-server chain, where we also need this NTLM proxying. It's so much easier to work with NTLM now!

[creage]

I finally could get my hands into this story, but whatever I do, I cannot pass the issue I faced with before:

@bjowes apparently I was too excited with this new API, thus missed one important issue.

It seems that the last started proxy always wins, despite using different ports.

I start proxy#1 as user#1, I start browser#1 with this proxy. I start proxy#2 as user#2, I start browser#2 with this proxy.

I switch to browser#1, try to run requests - they all run as user#2.

Could it be because of you setting process.env.HTTP_PROXY globally for every new proxy started? could this somehow affect previously started proxies?

UPDATE: I got this scenario working, by starting each proxy in a forked child process, which makes their environments independent.