Setting up a pipeline to run memlab tests

[TaylorShane]

I'm not very experienced with setting up CI/CD and I'm having troubling creating a stage to run a memlab scenario. I can run scenarios locally just fine, but my pipeline fails.

I have an Angular application in Gitlab that pulls a docker image with node:16.10.0 The memlab stage of my .gitlab-ci-.yml file looks like this:

memlab:
  stage: memlab
  extends: .prod-build
  services:
    - selenium/standalone-chrome:90.0-20210517
  script:
    - 'PUPPETEER_SKIP_DOWNLOAD=true' # this was added as an attempt to resolve my errors
    - yarn
    - sed -i "s/hash = 'UNKNOWN'/hash = '${GIT_HASH}'/g" src/app/app.constants.ts
    - yarn test:leak-pipeline
    - mkdir memlab
    - memlab --clean-up-snapshot
    - mv e2e/memlab/results/ memlab
    - yarn build:prod --progress=false
  artifacts:
    expire_in: 3h
    paths:
      - dist
  tags:
    - DOCKER

Here is what the script looks like (which runs locally just fine): "test:leak-pipeline": "memlab run --scenario e2e/memlab/scenarios/my-scenario.js --trace-object-size-above=100000 --skip-screenshot --work-dir e2e/memlab/results/"

The pipeline error I get looks like this:

[4/4] Building fresh packages...
error /builds/my-project/node_modules/puppeteer: Command failed.
Exit code: 1
Command: node install.js
Arguments: 
Directory: /builds/my-project/node_modules/puppeteer
Output:
ERROR: Failed to set up Chromium r982053! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
Error: connect ETIMEDOUT 1.1.1.1:443
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1146:16) {
  errno: -110,
  code: 'ETIMEDOUT',
  syscall: 'connect',
  address: '1.1.1.1',
  port: 443
}

I thought maybe this error was telling me that I need puppeteer as a dependency in my project, but when I did that I still get the following error:

[4/4] Building fresh packages...
error /builds/my-project/node_modules/@memlab/api/node_modules/puppeteer, /builds/my-project/node_modules/@memlab/cli/node_modules/puppeteer, /builds/my-project/node_modules/@memlab/core/node_modules/puppeteer, /builds/my-project/node_modules/@memlab/e2e/node_modules/puppeteer, /builds/my-project/node_modules/@memlab/heap-analysis/node_modules/puppeteer, /builds/my-project/node_modules/memlab/node_modules/puppeteer: Command failed.
Exit code: 1
Command: node install.js
Arguments: 
Directory: /builds/my-project/node_modules/@memlab/api/node_modules/puppeteer
Output:
ERROR: Failed to set up Chromium r982053! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
Error: connect ETIMEDOUT 1.1.1.1:443
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1146:16) {
  errno: -110,
  code: 'ETIMEDOUT',
  syscall: 'connect',
  address: '1.1.1.1',
  port: 443
}

I feel like my entire approach might be wrong. Should I instead be looking at ways of somehow incorporating memlab tests into my existing karma tests?

[JacksonGL]

ERROR: Failed to set up Chromium r982053! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.

This error is likely triggered by downloading Chromium binary when running npm install puppeteer in the CI/CD environment, which may not support downloading from certain external urls.

ERROR: Failed to set up Chromium r982053! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.

You may skip the chromium binary download via the PUPPETEER_SKIP_DOWNLOAD env variable, but memlab does not support setting an external Chromium binary yet. Maybe we can add a new option to specify an external chromium binary, something like:

memlab run --scenario <FILE> --chromium-binary <BINARY_FILE>

NOTE: I'm not 100% sure if a --chromium-binary flag would fix the issue from the CI/CD environment you are running.

[JacksonGL]

memlab --clean-up-snapshot

memlab does not provide a --clean-up-snapshot CLI option.

You can use memlab reset --work-dir /your/work/dir to clean up the working directory.

(This is unrelated to the Chromium setup error)

[TaylorShane]

Thanks for you help @JacksonGL ! I'm still trying to resolve my issues, but I appreciate your help.

[TaylorShane]

Hi @JacksonGL - I think I'm getting closer, but memlab isn't finding chrome.

yarn run v1.22.5
$ memlab run --scenario e2e/memlab/scenarios/test-scenario.js --chromium-binary @build3002-DEVELOPMENT.b64d9fda.zip '--chrome-flags=--headless --no-sandbox --disable-gpu'
Could not find expected browser (chrome) locally. Run `npm install` to download the correct Chromium revision (982053).
total time: 4ms
snapshot meta data invalid or missing
Use `memlab help` or `memlab <COMMAND> -h` to get helper text
error Command failed with exit code 1.

I realize this is probably not an issue with memlab but rather my gitlab-ci.yml file, which now looks like this.

memlab:
  stage: memlab
  extends: .special-build
  services:
    - selenium/standalone-chrome:90.0-20210517
  before_script:
    - yarn install
  script:
    - sed -i "s/hash = 'UNKNOWN'/hash = '${GIT_HASH}'/g" src/app/app.constants.ts
    - yarn test:leak-pipeline --chromium-binary @${ZIP_RELEASE}
    - mkdir memlab
    - mv e2e/memlab/results/ memlab
    - yarn build:prod --progress=false
  artifacts:
    expire_in: 3h
    paths:
      - dist
      - memlab-report.html
  tags:
    - DOCKER

Any suggestions would be greatly appreciated, thanks.

[JacksonGL]

The concept of --chromium-binary is an idea that may help the issue you faced, but it hasn't been incorporated into memlab yet.

[TaylorShane]

Thanks @JacksonGL ! I see the commit that was pushed recently, is there a plan to push a new version with these changes soon?

[JacksonGL]

I have released memlab@1.1.39 with the new flag.