seeInThisFile, Playwright, is broken. freezes process

[nateGarcia]

What are you trying to achieve?

use I.seeInThisFile("some string"); and run my tests in a pipeline.

What do you get instead?

Test failures that crash & freeze the worker node. It fails every test at I.seeInThisFile, and the CSV file clearly contains the string it is attempting to search for. Additionally, a notice that says an after all hook fails, which causes the process to freeze up. I don't have an "after all" hook in any of my test setup, so this is most likely a failure in a plugin, (videoOnFail most likely)... Since this happens in a CI pipeline, it is very frustrating because the node will run frozen until it times out.

Provide console output if related. Use --verbose mode for more details.

• Playwright helper
• custom I function including this seeInThisFile call.

Details

• CodeceptJS version: 3.3.4+
• NodeJS Version: v16.17.1
• Operating System: MacOS, linux ubuntu
• playwright: 1.23.4

exports.config = {
  tests: "./*/test_*.ts",
  output: "./results",
  helpers: {
    Playwright: {
      url: process.env.E2E_HOST || "https://localhost:8080",
      show: true,
      browser: "chromium",
      windowSize: "1600x1200",
      waitForNavigation: "load",
      waitForAction: 0,
      timeout: 60000,
      restart: false,
      ignoreHTTPSErrors: process.env.IGNORE_HTTPS === "true" || true,
      video: process.env.RECORD_FAILED === "true" || false,
      keepCookies: true,
      chromium: {
        // Launch args that make chrome run faster
        args: [
          "--blink-settings=imagesEnabled=false",
          "--proxy-server='direct://'",
          "--proxy-bypass-list=*",
        ],
      },
    },
    FileSystem: {},
    diffingHelper: {
      require: "./setup/diffingHelper",
    },
    attachVideoAllure: {
      require: "./setup/attachVideoAllure",
    },
    dataManipulation: {
      require: "./setup/dataManipulation",
    },
    ChaiWrapper: {
      require: "codeceptjs-chai",
    },
  },
  plugins: {
    allure: {
      // https://codecept.io/plugins/#allure
      enabled: process.env.ALLURE === "true" || false,
      outputDir: "./reports/allure",
    },
    coverage: {
      enabled: process.env.COVERAGE === "true" || false,
      coverageDir: "./reports/playwrightCoverageData",
      uniqueFileName: true,
    },
    autoDelay: {
      enabled: false,
    },
    eachElement: {
      enabled: true,
    },
    pauseOnFail: {
      enabled: false,
    },
    tryTo: {
      enabled: true,
    },
    retryTo: {
      enabled: true,
    },
    screenshotOnFail: {
      enabled: true,
      uniqueScreenshotNames: true,
    },
    autoLogin: {
    },
  },
  include: {
    I: "./setup/helperFunctions.ts",
  },
  bootstrap: null,
  mocha:
    process.env.MOCHA === "true"
      ? {
          reporter: "mocha-junit-reporter",
          reporterOptions: {
            testsuitesTitle: "E2E Tests",
            mochaFile: xmlDir,
            attachments: true,
            suiteTitleSeparatedBy: ".",
          },
        }
      : {},
  name: "codecept",
};

[nateGarcia]

I've attached the verbose output, if useful

[AWolf81]

@nateGarcia Can you please create a minimal example that is showing your issue?

I couldn't recreate the mentioned issue but I'm not seeing every part of your configuration. The following part is difficult to create without seeing your code:

    // ... rest of config ...
   helpers: {
     diffingHelper: {
       require: "./setup/diffingHelper",
      },
      attachVideoAllure: {
        require: "./setup/attachVideoAllure",
      },
      dataManipulation: {
        require: "./setup/dataManipulation",
      },
   }
   // rest of config
}

Your exact test code that is failing could also help to recreate your issue.

[nateGarcia]

@AWolf81 Hi there, thanks for the reply. Those helpers aren't in scope of the bug. The bug is occurring in a custom function that extends I like the following. (perhaps you can attempt this with a pre-existing CSV in the results dir)

// ./setup/helperFunctions.ts
export = function () {
  return actor({
    downloadAndCheckCsv: async function (){
      // Download and check file
      this.handleDownloads(fileName, 5);
      this.click("Download");
      this.amInPath("./results/");
      this.waitForFile(fileName, 10);
      this.wait(2);
      this.seeInThisFile("some string");
      this.seeInThisFile("some string");
     }
  });
};

// in steps.d.ts
type steps_file = typeof import('./setup/helperFunctions');

declare namespace CodeceptJS {
  interface SupportObject { I: I, current: any, login: any }
  interface Methods extends Playwright, .... other helpers .... {}
  interface I extends ReturnType<steps_file>, ... other stuff ... {}
  namespace Translation {
    interface Actions {}
  }
}

Also FYI, I have ALLURE and COVERAGE enabled in my run. And FWIW, this bug seemingly came out of no where. From what I can tell, my Playwright, codeceptjs, and other dependency versions haven't changed since a run that succeeded with the same function & tests.

[AWolf81]

I haven't worked with custom helpers yet. But have you tried to follow the documentation here?

I think you could access Playwright, FileSystem, and other helpers in your helper like this:

const { FileSystem, Playwright } = this.helpers;
FileSystem.handleDownloads(fileName, 5);
Playwright.click("Download");
// ...

I'm not sure about the actor function (where is it documented?) but the access should work like in the mentioned link (I haven't tested it yet).

[StanislavGrishaev]

@nateGarcia hi there, not about your issue, but still - could you share code of your helper attachVideoAllure ? that's exactly what i'm looking for.

[nateGarcia]

@StasGrishaevTutu Sure no problem. I think I took most of this from another user, but I forget their name. Note that it kills the browserContext in order to process the video. This could have some side effects depending on your setup.

In setup/attachVideoAllure.ts

import { readFileSync } from "fs";
import { Helper } from "codeceptjs";

/**
 * Helper class for the Playwright Report
 */
class PlaywrightHelper extends Helper {
  /**
   * Constructor
   * @param config
   */
  constructor(config) {
    super(config);
  }

  /**
   * Attach Video to allure report
   * @param test
   */
  async _attachVideo(test) {
    const Playwright = codeceptjs.container.helpers("Playwright");
    if (Playwright) {
      const allure = codeceptjs.container.plugins("allure");
      if (allure) {
        await Playwright.browserContext.close(); // Need to close browser context to get video to generate
        const FORMAT = "video/webm";
        const TITLE = "Execution Video";
        const video =
          test.artifacts?.video || test._retriedTest?.artifacts?.video;
        if (video) {
          codeceptjs.output.debug(`Attaching Video: ${video}`);
          allure.addAttachment(TITLE, readFileSync(video), FORMAT);
        }
      }
    }
  }

  /**
   * Run on test failure
   * @param test
   */
  async _failed(test) {
    await this._attachVideo(test);
  }
}

export = PlaywrightHelper;

in steps.d.ts, add in the helper

...
interface Methods extends Playwright, attachVideoAllure {}
interface I extends ReturnType<steps_file>, WithTranslation<attachVideoAllure> {}
...

[StanislavGrishaev]

@nateGarcia thank you, really appreciate it ! with a bit changes it works on my side !

[kobenguyent]

Closed for now! Feel free to reopen if you still encounter the issue with latest version and would be nice to provide a sample code to reproduce the issue.