Failing to Authenticate with Multiple Test Files

[MSIH]

1. if I run test individually, they all complete successfully
2. if I run test together --spec cypress/integration/* , then some test fail
3. if I change the order then different tests fail
4. the sites that fail seem to be sharepoint sites

The content-type of the response we received from your web server was:

text/plain

This was considered a failure because responses must have content-type: 'text/html'

However, you can likely use cy.request() instead of cy.visit().

cy.request() will automatically get and set cookies and enable you to parse responses. at http://localhost:63480/__cypress/runner/cypress_runner.js:157827:25 at visitFailedByErr (http://localhost:63480/__cypress/runner/cypress_runner.js:157203:12) at http://localhost:63480/__cypress/runner/cypress_runner.js:157807:13 at tryCatcher (http://localhost:63480/__cypress/runner/cypress_runner.js:10584:23) at Promise._settlePromiseFromHandler (http://localhost:63480/__cypress/runner/cypress_runner.js:8519:31) at Promise._settlePromise (http://localhost:63480/__cypress/runner/cypress_runner.js:8576:18) at Promise._settlePromise0 (http://localhost:63480/__cypress/runner/cypress_runner.js:8621:10) at Promise._settlePromises (http://localhost:63480/__cypress/runner/cypress_runner.js:8697:18) at _drainQueueStep (http://localhost:63480/__cypress/runner/cypress_runner.js:5291:12) at _drainQueue (http://localhost:63480/__cypress/runner/cypress_runner.js:5284:9) at Async.../../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:63480/__cypress/runner/cypress_runner.js:5300:5) at Async.drainQueues (http://localhost:63480/__cypress/runner/cypress_runner.js:5170:14) From Your Spec Code: at Context.eval (http://localhost:63480/__cypress/tests?p=cypress\integration\fic-org\fic-insider.js:116:8)

2) fic-insider Load SharePoint Search Page: https://foo/_layouts/15/osssearchresults.aspx?k=home: CypressError: cy.visit() failed trying to load:

https://testinsider.fic.nih.gov/_layouts/15/osssearchresults.aspx?k=home

The response we received from your web server was:

401: Unauthorized

This was considered a failure because the status code was not 2xx.

If you do not want status codes to cause failures pass the option: failOnStatusCode: false at http://localhost:63480/__cypress/runner/cypress_runner.js:157827:25 at visitFailedByErr (http://localhost:63480/__cypress/runner/cypress_runner.js:157203:12) at http://localhost:63480/__cypress/runner/cypress_runner.js:157807:13 at tryCatcher (http://localhost:63480/__cypress/runner/cypress_runner.js:10584:23) at Promise._settlePromiseFromHandler (http://localhost:63480/__cypress/runner/cypress_runner.js:8519:31) at Promise._settlePromise (http://localhost:63480/__cypress/runner/cypress_runner.js:8576:18) at Promise._settlePromise0 (http://localhost:63480/__cypress/runner/cypress_runner.js:8621:10) at Promise._settlePromises (http://localhost:63480/__cypress/runner/cypress_runner.js:8697:18) at _drainQueueStep (http://localhost:63480/__cypress/runner/cypress_runner.js:5291:12) at _drainQueue (http://localhost:63480/__cypress/runner/cypress_runner.js:5284:9) at Async.../../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:63480/__cypress/runner/cypress_runner.js:5300:5) at Async.drainQueues (http://localhost:63480/__cypress/runner/cypress_runner.js:5170:14) From Your Spec Code: at Context.eval (http://localhost:63480/__cypress/tests?p=cypress\integration\fic-org\fic-insider.js:123:8)

[MSIH]

The fix was to add cy.ntlmReset();

cy.ntlmReset(); 
cy.ntlm(env.url, env.user, env.pass);

[bjowes]

Nice that you resolved it yourself @MSIH !

Generally, I recommend calling cy.ntlmReset() in an afterEach step in your specfiles, so that you have a clean slate at the start of each test. Calling cy.ntlmReset() and cy.ntlm() causes very little overhead to test execution time so I think it makes sense to keep it clean.

In theory, cy.ntlmReset() shouldn't be required when you don't need to use different credentials for different tests, but it does not only clear the NTLM config. It also closes all connections, which helps in preparing a clean slate for the next test.

I have a revised version of ntlmReset in the works, that will have options to only close connections and not clear the NTLM config. This can be helpful when you are performing test setup/cleanup operations in before and after steps that also require authentication. When the site under test has pending requests that has not yet been sent when the test ends, you may receive the 401 if the ntlmReset "comes too early" so auth fails. I hope the more advanced ntlmReset will mitigate this.