NimaSoroush / differencify

Differencify is a library for visual regression testing
MIT License
634 stars 46 forks source link

target.toMatchSnapshot() not behaving as expected #74

Closed tankalaharish closed 6 years ago

tankalaharish commented 6 years ago

I was trying to do the following:

        const target = differencify.init({
          chain: false
        })
        await target.launch({ headless: true })
        const page = await target.newPage()
        await page.setViewport({
          width: 1366,
          height: 766
        })
        await page.goto(key, { waitUntil: 'networkidle0' })
        await page.content()
        const bodyHandle = await page.$('body')
        const { width, height } = await bodyHandle.boundingBox()
        await console.log(width, height)
        const imag = await fs.readFileSync(
          require.resolve('../../snapshots/' + value + '_screenshot.png')
        )
        const result = await target.toMatchSnapshot(imag)

Here( imag = local screenshot )and when im trying to match the reference screenshot with the(imag), the same local screenshot taken earlier is being saved as the present reference snapshot.

NimaSoroush commented 6 years ago

Hey @tankalaharish , Can you post the whole code to demonstrate how you actually make screenshot? I don't see anywhere in your code call screenshot function.

If you want a working example of your test, differencify integration tests would probably help. https://github.com/NimaSoroush/differencify/blob/master/src/integration.tests/integration.test.js#L24

grv190 commented 6 years ago

Hey @NimaSoroush , we are trying to compare the local screenshot with the live website reference screenshot.

` await console.log('inside async') const target = differencify.init({ testName: "test1", chain: false }) await target.launch({ headless: true })

    const page = await target.newPage()
    await page.setViewport({
      width: 1366,
      height: 766
    })
    await page.goto(key, { waitUntil: 'networkidle0' })
    await page.content()
    const bodyHandle = await page.$('body')
    const { width, height } = await bodyHandle.boundingBox()
    const imag = await fs.readFileSync(
      require.resolve('../../snapshots/' + value + '_screenshot.png')
    )
    await console.log('read the file')
    const result = await target.toMatchSnapshot(imag)
    await console.log('After')
    await bodyHandle.dispose()
    await expect(result).toEqual(true)
    await console.log(result)
    await target.close()
  } catch (error) {
    await console.log(error)
  }

`

NimaSoroush commented 6 years ago

I am afraid that is not how currently differencify works. But I am happy to look over this feature with and you and see how we can integrate it to differencify

grv190 commented 6 years ago

@NimaSoroush can you kindly explain what is wrong with the code? are you saying that we cannot compare local saved screenshot with the live website reference screenshot?

NimaSoroush commented 6 years ago

Hey @grv190 , you are actually right. I misread your code. That is actually possible and should work as expected. I'll run your code locally to see what is wrong

grv190 commented 6 years ago

@NimaSoroush Did you run the code? Is the code working or having any issue?

NimaSoroush commented 6 years ago

Hey @grv190: Yeah, there is an issue with Differencify which is not handling your usecase very well. I am just putting a fix for it. Will update here when it is out Also wondering if you are using this code with jest?

grv190 commented 6 years ago

@NimaSoroush can you briefly explain what is the issue? Yeah we are using this code with puppeteer, differencify and jest.

NimaSoroush commented 6 years ago

The issue is when you specify testName in init() function. It always get overridden by jest testName and as a result the same image always gets overridden

NimaSoroush commented 6 years ago

@grv190 : This should be fixed now. One example being added to integration tests which is similar to your usecase. https://github.com/NimaSoroush/differencify/blob/654fd66cb69a6bc4de526e1767f6ea2fddfc1b0b/src/integration.tests/integration.test.js#L174