cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.66k stars 3.16k forks source link

Fetching a list of unique superdomains to test and cypress fails on second call #835

Closed joacim-boive closed 6 years ago

joacim-boive commented 6 years ago

Is this a Feature or Bug?

Feature

Current behavior:

I configure my tests using a Google Sheet so that non techs can configure what URLs the wish to test, that includes a number of different domains as we run the same code (with different config, hence the testing) in different countries. Ex; https://www.lensway.se https://www.lensway.no ...

My code fails when I try to test https://www.lensway.no

cypress:driver emitted: 'fail' to '0' listeners - with args: Error: cy.visit() failed because you are attempting to visit a second unique domain.

You may only visit a single unique domain per test.

Different subdomains are okay, but unique domains are not.

The previous domain you visited was: 'https://www.lensway.se'

You're attempting to visit this new domain: 'https://www.lensway.no'

You may need to restructure some of your code to prevent this from happening.

https://on.cypress.io/cannot-visit-second-unique-domain

Desired behavior:

Don't fail. :)

How to reproduce:

Test code:

import * as utils from '../../support/utils';

let sheetData = [];

before(() => {
  return utils.getSheetData('megamenu!A2:A500').then(data => {
    return (sheetData = data);
  });
});

describe('Megamenu top links', () => {
  it('Should have ONE canonical link that points to the correct URL', () => {
    sheetData.forEach(([url]) => {
      cy
        .visit(url)
        .get('.header-large__nav-main a')
        .each(link => {
          utils.testCanonical(link[0].href, link[0].href);
          utils.testCanonical(link[0].href, link[0].href, `cid=${new Date().getTime()}`);
        });
    });
  });
});
jennifer-shehane commented 6 years ago

We should definitely write a better description of why this is the case in our doc link.

You can visit separate superdomains in separate tests, so if you move your logic up so that each domain creates a test, this will work fine. Something like:


describe('Megamenu top links', () => {
  sheetData.forEach((url) => {
    it('Should have ONE canonical link that points to' + url, () => {
      cy
        .visit(url)
        .get('.header-large__nav-main a')
        .each(link => {
          utils.testCanonical(link[0].href, link[0].href);
          utils.testCanonical(link[0].href, link[0].href, `cid=${new Date().getTime()}`);
        });
    });
  });
});
brian-mann commented 6 years ago

Also I will just throw out there that if you're trying to do spidering then you probably don't want to use cy.visit you could just use cy.request to programmatically make all the requests you want and then parse the HTML and then find the <a> and then continue to recurse.

Doing it that way will prob be at least 10x faster since it avoids all the page loads.

joacim-boive commented 6 years ago

Thanks @jennifer! That was too easy! 😉

Great tip @Brian! I’ll definitely look into that as I was currently investigating how to only load the HTML.

joacim-boive commented 6 years ago

@jennifer-shehane Unfortunately I cant run the test with your approach, it says:

"No tests found in your file:

/Users/joaboi/Intellij/test/cypress/integration/verify-canonicals_spec.js We could not detect any tests in the above file. Write some tests and re-run."

jennifer-shehane commented 6 years ago

Sorry, may have been a typo here, url should not be wrapped in brackets - I've updated my pasted example above:

  sheetData.forEach(([url]) => {

I also did not test the before logic since I don't have your sheetData, so you may need to debug that code.

joacim-boive commented 6 years ago

No I understand.

But even if I simplify the test case I still get the same problem:

let sheetData = [];

before(() => {
  debugger;
  return (sheetData = ['https://www.lensway.se', 'https://wwww.lensway.no']);
});

describe('Megamenu top links', () => {
  sheetData.forEach(url => {
    it('Should have ONE canonical link that points to' + url, () => {
      debugger;
      cy.visit(url);
    });
  });
});
joacim-boive commented 6 years ago

before is never triggered.

brian-mann commented 6 years ago

@joacim-boive this isn't anything on Cypress - this is how Mocha works.

You cannot create dynamic tests if you're relying on the code inside of a hook to run first. Hooks are not run until Mocha understands how many tests there are and their structure.

In other words - it's impossible for before to run "before" Mocha parses the tests. What would the hook be running for? It would be unclear.

If you move your sheetData assignment outside of the describe block then it works correctly.

If you put in a debugger above your sheetData.forEach you would see that sheetData is an empty array.

joacim-boive commented 6 years ago

@brian-mann - The original request was to get different URLs using cy.visit and the solution would be the below, according to @jennifer-shehane

describe('Megamenu top links', () => {
  sheetData.forEach((url) => {
    it('Should have ONE canonical link that points to' + url, () => {
      cy
        .visit(url)
        .get('.header-large__nav-main a')
        .each(link => {
          utils.testCanonical(link[0].href, link[0].href);
          utils.testCanonical(link[0].href, link[0].href, `cid=${new Date().getTime()}`);
        });
    });
  });
});

That doesn't work as Cypress tells me there are no tests (or if it's Mocha, someone does anyway :)

What I CAN do is to use cy.request:

let sheetData = [];

before(() => {
  return (sheetData = ['https://www.lensway.se', 'https://www.lensway.no']);
});

describe('Megamenu top links', () => {
  it('Should have ONE canonical link that points to current URL', () => {
    sheetData.forEach(url => {
      cy.request(url);
    });
  });
});

The above works with different domains as well, that cy.visit doesn't allow. But it would be nice to get cy.visit to work as that provides a nice video of the test and cy.request won't provide any useful video as the page isn't rendered. (Although this is expected)