Open therealpecus opened 5 years ago
Hi @therealpecus, I am not sure what would be the best approach for unique ID generation. There is already an incremental test ID generation in place but I think for anything more sophisticated you need to have your own ID generator as part of your test and pass the test name to Differencify somehow
e.g.
const Differencify = require('differencify');
const differencify = new Differencify(GlobalOptions);
const prefix = 'test name';
let ID = 0;
const testNameGenerator = () => {
const testName = `${testName} ${ID + 1}`
return testName;
};
(async () => {
await differencify
.init({ testName: testNameGenerator() })
.launch()
.newPage()
.setViewport({ width: 1600, height: 1200 })
.goto('https://github.com/NimaSoroush/differencify')
.waitFor(1000)
.screenshot()
.toMatchSnapshot()
.result((result) => {
console.log(result); // Prints true or false
})
.close()
.end();
await differencify
.init({ testName: testNameGenerator() })
.launch()
.newPage()
.setViewport({ width: 1600, height: 1200 })
.goto('https://github.com/NimaSoroush/differencify')
.waitFor(1000)
.screenshot()
.toMatchSnapshot()
.result((result) => {
console.log(result); // Prints true or false
})
.close()
.end();
})();
Hi @NimaSoroush
thanks for your reply. There is an issue with the current code, whereas the internal testId
is always added when tests are run outside Jest.
See this relevant snippet:
https://github.com/NimaSoroush/differencify/blob/5ec7eef9405bbca0a721cc26bebd9826beb3f042/src/target.js#L232-L240
The else
adds the testId
even when the testConfig.testName
has been provided. It happens even when testId
is undefined
, because of:
https://github.com/NimaSoroush/differencify/blob/5ec7eef9405bbca0a721cc26bebd9826beb3f042/src/index.js#L60
which returns NaN
which in turns evaluates to true
.
Running unchained allows to bypass this test. We already generate unique test names for each test, and pass it as a config to init()
:
const asyncTest = async ([vid, breakpoint, filename, width, height, title], current = 'N/A', total = 'N/A') => {
// avoid Differencify chaining to ensure test filenames are not polluted by ids
const target = differencify.init({ testName: `${vid}-${width}`, chain: false });
const page = await target.newPage();
await page.setViewport({ width, height });
await page.goto(`http://localhost:3000/_variants/${theme}/${filename}.html`, { waitUntil: 'networkidle2' });
const image = await page.screenshot();
// override filenames including a test counter
target.testConfig.isJest = false;
differencify.testId = undefined;
target.testConfig.testId = undefined;
const test = await target.toMatchSnapshot(image);
console.info(
`[${String(current).padStart(String(total).length)}/${total}] ${
test ? '✅' : '❌'
} ${title} 'http://localhost:3000/_variants/${theme}/${filename}.html' (${breakpoint})`
);
await page.close();
};
We solved it by explicitely dropping out of Jest, and nullying both the global testId
and test specific testId
.
Hi @therealpecus , Sorry for the confusion. Yes, you are right and this needs to be fixed. Are you happy putting a fix for this? otherwise, I will prioritize a fix for at some point
Sure! I'll make a PR.
Hi!
We have over 6000 tests to run and are avoiding Jest since it adds an unbereable overhead. Running without Jest prevents consistent test names, since the testId is automatically added.
The responsibility of the test name being unique should be in the hands of the implementator for manual tests, since changing the order of tests invalidates them (the original snapshot would not be found).
Is there a quick workaround?
Tx, M