NimaSoroush / differencify

Differencify is a library for visual regression testing
MIT License
634 stars 46 forks source link

Second Jest test failed #109

Closed patriksima closed 5 years ago

patriksima commented 5 years ago

Hi,

I am not sure if i understand jest testing correctly. I need run test before I am going to change production website. And after I've made these changes. But...If i run test twice, second test failed due to timeout (toMatchSnapshot).

What's wrong?

const Differencify = require('differencify');
const differencify = new Differencify({ debug: false });

describe('Test1', () => {
    it('Homepage', async () => {
        await differencify
            .init()
            .launch()
            .newPage()
            .setViewport({
                width: 2800,
                height: 2800
            })
            .goto("https://example.com/")
            .screenshot()
            .toMatchSnapshot()
            .close()
            .end();
    }, 10000);
});
NimaSoroush commented 5 years ago

Hey @wrongware , I just tried your script a couple of times locally and I couldn't reproduce it. Possibly your machine is just a bit slow to process the Image comparison as that only happens on the second run. What I suggest is to increase the timeout to 30000 and see if that helps.

You test is very similar to the integration test that already exists in Differencify here and as you can see the timeout is around 3secs. https://github.com/NimaSoroush/differencify/blob/84489a38ca44a22e3c1e574f7a12fc88f6975e3f/src/integration.tests/integration.test.js#L12

patriksima commented 5 years ago

Hi, i've figured it out. I guess, Puppeteer is waiting for loading some external scripts like Facebook, so I've disabled Javascript and resized viewport. And it works.

it('Homepage', async () => {
        await differencify
            .init({
                testName: "homepage"
            })
            //.launch() // cuz shared browser (multiple tests)
            .newPage()
            **.setJavaScriptEnabled(false)**
            .setViewport({
                **width: 1200,
                height: 1000**
            })
            .goto("https://example.com/")
            .wait(500)
            .screenshot()
            .toMatchSnapshot()
            .close()
            .end();
    });

I think there are some performance issues because I got gaming notebook w 16GB, 8-core Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz, SSD disk etc. It is enough power to solve image comparsion.

NimaSoroush commented 5 years ago

Good to hear that you figured it out!