Testing waitForPageToLoad with SPAs

[tommedema]

First let me say that I very much appreciate your work. Am planning to contribute in the future, perhaps this is a good start. I could write a test for waitForPageToLoad with different single page applications that load data from a backend.

Before writing the test I would like to actually understand how I can make this work. So far it seems to work for react+redux, react+mobx, angular1, and angular4. However, it does not seem to work with elm and a CMS called Chrizmas.

I've been trying this out as follows:

const HeadlessChrome = require('simple-headless-chrome');
const fs = require('asfs');

// TODO: move to config
const targets = [
  {name: 'react-redux', url: 'https://react-redux.realworld.io'},
  {name: 'react-mobx', url: 'https://react-mobx.realworld.io/'},
  {name: 'angular-4', url: 'https://angular2.realworld.io/'},
  {name: 'angular-1', url: 'https://angularjs.realworld.io/'},
  {name: 'elm-spa', url: 'http://rtfeldman.github.io/elm-spa-example/'},
  {name: 'chrizmas', url: 'https://crizmas-mvc.realworld.io/'}
];

const browser = new HeadlessChrome({
  headless: true,
  tabs: {
    maxTabs: 1
  }
});

async function getStaticDomHTML(url, screenshotPath) {
  try {
    await browser.init();

    const mainTab = await browser.newTab({ privateTab: true });

    await mainTab.goTo(url, {
      timeout: false,
      bypassCertificate: true
    });
    await mainTab.waitForPageToLoad();

    const html = (await mainTab.evaluate(function() {
      return document.documentElement.innerHTML;
    })).result.value;

    if (screenshotPath) {
      await mainTab.saveScreenshot(screenshotPath);
    }

    await browser.close();

    return html;
  }
  catch (err) {
    await browser.close();
    throw err;
  }
}

(async function() {
  for (let target of targets) {
    try {
        const html = await getStaticDomHTML(target.url, `${__dirname}/../output/${target.name}`);
        await fs.writeFileAsync(`${__dirname}/../output/${target.name}-static.html`, html, 'utf8');
    }
    catch (err) {
      console.error(err);
    }
  }
}());

Results are promising but not perfect.

react-redux:

react-mobx:

angular1

angular4

elm

chrizmas

The URLs come from the realworld project. My plan is to make these work first, then write a test case for it. If you have other SPA example urls, that would be great.

Also, are you ok with tests (integration tests I guess) that depend on these realworld.io projects and backends? They will probably stay alive but this is not guaranteed.

[LucianoGanga]

Hi! It would be interesting to discover why it didn't worked on elm and Chrizmas. I think it may be a timing problem, but i'm not sure.

Did you tried navigating them without headless mode on to see what happens?

Cheers! Lucho

[LucianoGanga]

Hi! Any news about this?

Thanks! Luciano

[tommedema]

@LucianoGanga yes, I tried that, and it is indeed a timing issue. But the question is why await mainTab.waitForPageToLoad(); does not seem to resolve this timing issue. What do you think?

[LucianoGanga]

Hi @tommedema !

It may be because of how waitForPageToLoad() works. It waits for the browser event of "page loaded", and in the case that is failing, maybe the page tells the browser that it already loaded all but do some stuff asynchronously. What I think is that you can try using the .waitForSelectorToLoad() and wait for a specific selector you need to be loaded.

Thanks! Lucho