[Discussion] addContext in .then() closure of Cypress (multiple screenshots)

The original problem

So, just like many of us, I wanted to add screenshots 📷 to my mochawesome report. And I quickly found several guides online on how to do so. Eventually, I ended up with something like this to solve my problem;

Cypress.on('test:after:run', (test, runnable) => {
    if (test.state == "failed" && Cypress.config('screenshotOnRunFailure')){
        const screenshot = encodeURI(`cypress/screenshots/${Cypress.spec.name}/${runnable.parent.title} -- ${test.title} (failed).png`)
        addContext({test}, screenshot)
    }
})

Done, works fine right? Well, what if you have a test that has several attempts, adding screenshots with (attempt 2, attempt 3, etc). Are you happy just getting one of those screenshots? And what if you have a beforeEach() hook and it fails there? Then the screenshot name becomes ${test.title} -- before each hook (failed).png So I set out to add all the screenshots. 📷 📷 📷 📷

My idea💡

Instead of statically defining the name(s) of the screenshot(s) that may or may not be there, I wanted to use fs.readdirSync() to find and list all screenshots in the folder and match them to the test.title. Like so;

Cypress.on('test:after:run', (test, runnable) => {
    if (test.state == "failed" && Cypress.config('screenshotOnRunFailure')) {
        cy.task('getFilesFromFolder', `/cypress/screenshots/${Cypress.spec.name}`).then((files: string[]) => {
            files.forEach(function (filename: string) {
                console.log("Checking: " + filename) //successfully logs filename
                if (filename.includes(`${runnable.parent.title} -- ${test.title}`)) {
                    console.log("Trying to add: " + filename) //successfully logs filename
                    addContext({test}, encodeURI(`../cypress/screenshots/${Cypress.spec.name}/${filename}`))
                }
            })
        })
    }
})

I am utilizing cy.task() because you cannot use fs-functions within Cypress test code itself. So I use the task below to return a string[] of filenames. From config file;

setupNodeEvents(on, config){
            on('task', {
                getFilesFromFolder(folder: string): string[] {
                    return fs.readdirSync(path.join(process.cwd(), folder));
                }
            })
        }

Unfortunately, this does not work...

The problem now

While encapsulated in the cy.task().then(), the addContext() function doesn't seem to do anything... Its doesnt even add the broken image link that you see if there is an issue with the url itself. I have tries moving the const addContext = require("mochawesome/addContext.js") import inside the .then() closure to ensure that it is accessible in its context without it making a difference. Any ideas or inputs on how to solve this is more than welcome!

Alternate solutions

Could just addContext once for each possible screenshot, but that would just leave me with a bunch of broken image links in the mochawesome report...
Are there other in the runnable or test objects that could be used to determine number of attempts or it the test failed in the it() test og beforeEach() hook?

I'm having the exact same problem. It doesn't seem to be specific to readdir() as substituting cy.wait() gets the same results, so perhaps related to a callback within the event handler. The following does not work:

Cypress.on("test:after:run", (test, runnable) => {
  if (test.state === "failed" || test.state === "passed") {
    //this does not work
    cy.wait(1).then(function () {
      addContext({ test } as Mocha.Context, "CONTEXT ADDED WITHIN test:after:run");
    });
  }
});

...but the same structure outside of the event handler does work:

it("should log context within callback", function () {
    cy.addContext("outer").then(() => {
      cy.wait(1).then(() => {
        cy.addContext("within wait");
      });
    });
  });

//added a custom command to get test context, as using _this_ was not working for me
Cypress.Commands.add("addContext", (context) => {
  cy.once("test:after:run", (test) => addContext({ test } as Mocha.Context, context));
});

adamgruber / mochawesome