Support @jest-runner/electron

[UberMouse]

Issue description or question

I'm trying to get Wallaby setup to test our code designed to run in an Electron renderer process environment, so it needs to use @jest-runner/electron as the testRunner/testEnvironment.

Problem 1

Wallaby seems to only function when the testRunner is specifically jest-runner/jest-runner-groups (based on examining the loadJest function of server.js). I worked around this by making the function that filters the configs just return true so it would load our jest config.

But I'm also guessing there is a reason this only loads the jest-runner... runner and that could be the source of my problem 2

Problem 2

Now that I have Wallaby booting happily and running Jest, it doesn't actually run any tests. I'm guessing that is because of this error

[Info]  2021-07-07T02:05:28.923Z workers Scheduling Jest Test Run (xo7fp): 2021-07-07T02:05:28.919Z
[Info]  2021-07-07T02:05:28.923Z workers Jest Test Error: 2021-07-07T02:05:28.921Z
[Info]  console.error: TypeError: Cannot destructure property 'config' of 'context' as it is undefined.
[Info]      at /home/ubermouse/code/kawaka/common/temp/node_modules/.pnpm/@jest+core@26.6.3/node_modules/@jest/core/build/TestScheduler.js:257:14
[Info]      at Set.forEach (<anonymous>)
[Info]      at TestScheduler.scheduleTests (/home/ubermouse/code/kawaka/common/temp/node_modules/.pnpm/@jest+core@26.6.3/node_modules/@jest/core/build/TestScheduler.js:256:14)
[Info]  2021-07-07T02:05:28.929Z workers Jest Test Run Complete (xo7fp): 2021-07-07T02:05:28.921Z
[Info]  2021-07-07T02:05:28.930Z workers [xo7fp] Run 0 test(s), skipped 0 test(s)
[Info]  2021-07-07T02:05:28.933Z workers [xo7fp] Sandbox is responsive, closing it
[Info]  2021-07-07T02:05:28.933Z project Test run finished

Running npx jest works fine

So I guess my question is:

1. Does Wallaby support using alternative Jest testRunners?
2. If so, is there anything else I need to do?

or is this something you are willing to add support for?

Let me know if you need any further details, we have a somewhat complicated environment because we use @microsoft/rush along with their Heft toolchain for building/testing which isn't really compatible with how Wallaby needs to work so I've had to write some glue code.

Wallaby diagnostics report

{
  editorVersion: '1.57.1',
  pluginVersion: '1.0.297',
  editorType: 'VSCode',
  osVersion: 'linux 5.8.0-59-generic',
  nodeVersion: 'v12.22.1',
  coreVersion: '1.0.1102',
  checksum: 'NDU4YjUwNzU3NDhjNWU3ZGNhNzg2OGNiNzIyMWI0YWIsMTYzNDc3NDQwMDAwMCww',
  config: {
    testFramework: { version: 'jest@24.8.0', configurator: 'jest@24.8.0', reporter: 'jest@24.8.0', starter: 'jest@24.8.0', path: './node_modules/jest', configFile: './jest.config.js', autoDetected: true },
    trace: true,
    diagnostics: {
      jest: {
        config: {
          configs: [
            {
              automock: false,
              cache: true,
              cacheDirectory: '/tmp/jest_rs',
              clearMocks: false,
              coveragePathIgnorePatterns: [ '/node_modules/' ],
              cwd: '<homeDir>/code/kawaka/packages/rimu/shared',
              dependencyExtractor: undefined,
              detectLeaks: undefined,
              detectOpenHandles: undefined,
              displayName: undefined,
              errorOnDeprecated: false,
              extraGlobals: [],
              filter: undefined,
              forceCoverageMatch: [],
              globalSetup: undefined,
              globalTeardown: undefined,
              globals: [Object: null prototype] {
                __DEV__: 'true',
                SENTRY_DSN: '',
                SENTRY_RELEASE: '',
                SENTRY_ENVIRONMENT: '',
                [Symbol(configuration-file-field-annotation)]: {
                  configurationFilePath: '<homeDir>/code/kawaka/packages/rimu/shared/config/jest.config.json',
                  originalValues: { __DEV__: 'true', SENTRY_DSN: '', SENTRY_RELEASE: '', SENTRY_ENVIRONMENT: '' }
                }
              },
              haste: { computeSha1: false, throwOnModuleCollision: false },
              injectGlobals: true,
              moduleDirectories: [ 'node_modules' ],
              moduleFileExtensions: [
                'ts',
                'tsx',
                'cjs',
                'js',
                'json',
                'node',
                [Symbol(configuration-file-field-annotation)]: {
                  configurationFilePath: '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@rushstack+heft-jest-plugin@0.1.3/node_modules/@rushstack/heft-jest-plugin/includes/jest-shared.config.json',
                  originalValues: { '0': 'ts', '1': 'tsx', '2': 'cjs', '3': 'js', '4': 'json', '5': 'node' }
                }
              ],
              moduleLoader: undefined,
              moduleNameMapper: [ [ '^~/(.*)$', '<homeDir>/code/kawaka/packages/rimu/shared/src/$1' ] ],
              modulePathIgnorePatterns: [],
              modulePaths: undefined,
              name: 'b8bbcac0d1a5c0aa52946f9b7b350d1c',
              prettierPath: 'prettier',
              resetMocks: false,
              resetModules: false,
              resolver: '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@rushstack+heft-jest-plugin@0.1.3/node_modules/@rushstack/heft-jest-plugin/lib/exports/jest-improved-resolver.js',
              restoreMocks: false,
              rootDir: '<homeDir>/code/kawaka/packages/rimu/shared',
              roots: [ '<homeDir>/code/kawaka/packages/rimu/shared/src' ],
              runner: '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@jest-runner+electron@3.0.1_electron@12.0.4/node_modules/@jest-runner/electron/index.js',
              setupFiles: [ '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@rushstack+heft-jest-plugin@0.1.3/node_modules/@rushstack/heft-jest-plugin/lib/exports/jest-global-setup.js' ],
              setupFilesAfterEnv: [ '<homeDir>/code/kawaka/packages/rimu/shared/src/jest.setup.ts' ],
              skipFilter: false,
              skipNodeResolution: undefined,
              slowTestThreshold: 5,
              snapshotResolver: undefined,
              snapshotSerializers: [],
              testEnvironment: '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@jest-runner+electron@3.0.1_electron@12.0.4/node_modules/@jest-runner/electron/environment.js',
              testEnvironmentOptions: {},
              testLocationInResults: false,
              testMatch: [ '<homeDir>/code/kawaka/packages/rimu/shared/src/**/*.test.{ts,tsx}', '<homeDir>/code/kawaka/packages/rimu/shared/src/**/?(*.)(spec|test|integration).ts?(x)' ],
              testPathIgnorePatterns: [ '/node_modules/' ],
              testRegex: [],
              testRunner: '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/jest-jasmine2@26.6.3/node_modules/jest-jasmine2/build/index.js',
              testURL: 'http://localhost/',
              timers: 'real',
              transform: [
                [ '\\.(ts|tsx)$', '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/babel-jest@26.6.3_@babel+core@7.14.6/node_modules/babel-jest/build/index.js', {} ],
                [
                  '\\.(css|sass|scss)$',
                  '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@rushstack+heft-jest-plugin@0.1.3/node_modules/@rushstack/heft-jest-plugin/lib/exports/jest-identity-mock-transform.js',
                  {}
                ],
                [
                  '\\.(aac|eot|gif|jpeg|jpg|m4a|mp3|mp4|oga|otf|png|svg|ttf|wav|webm|webp|woff|woff2)$',
                  '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@rushstack+heft-jest-plugin@0.1.3/node_modules/@rushstack/heft-jest-plugin/lib/exports/jest-string-mock-transform.js',
                  {}
                ],
                [ 'node_modules.*\\.jsx?$', '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/babel-jest@26.6.3_@babel+core@7.14.6/node_modules/babel-jest/build/index.js', {} ]
              ],
              transformIgnorePatterns: [
                '/node_modules(?!.*(intl-messageformat|intl-messageformat-parser|@kx|dnd-core|react-dnd|react-dnd-html5-backend|react-dnd-test-backend|react-dnd-test-utils|/ol|ol-mapbox-style|ol-ext)/).+\\.(js|ts|tsx)$'
              ],
              unmockedModulePathPatterns: undefined,
              watchPathIgnorePatterns: []
            }
          ],
          globalConfig: {
            bail: 0,
            changedFilesWithAncestor: false,
            changedSince: undefined,
            collectCoverage: true,
            collectCoverageFrom: [ 'src/**/*.{ts,tsx}', '!src/**/*.d.ts', '!src/**/*.test.{ts,tsx}', '!src/**/test/**', '!src/**/__tests__/**', '!src/**/__fixtures__/**', '!src/**/__mocks__/**' ],
            collectCoverageOnlyFrom: undefined,
            coverageDirectory: '<homeDir>/code/kawaka/packages/rimu/shared/temp/coverage',
            coverageProvider: 'babel',
            coverageReporters: [ 'json', 'text', 'lcov', 'clover' ],
            coverageThreshold: undefined,
            detectLeaks: undefined,
            detectOpenHandles: undefined,
            enabledTestsMap: undefined,
            errorOnDeprecated: false,
            expand: false,
            filter: undefined,
            findRelatedTests: false,
            forceExit: false,
            globalSetup: undefined,
            globalTeardown: undefined,
            json: false,
            lastCommit: false,
            listTests: undefined,
            logHeapUsage: false,
            maxConcurrency: 5,
            maxWorkers: 11,
            noSCM: undefined,
            noStackTrace: false,
            nonFlagArgs: undefined,
            notify: false,
            notifyMode: 'failure-change',
            onlyChanged: false,
            onlyFailures: false,
            outputFile: undefined,
            passWithNoTests: undefined,
            projects: [],
            replname: undefined,
            reporters: undefined,
            rootDir: '<homeDir>/code/kawaka/packages/rimu/shared',
            runTestsByPath: false,
            silent: false,
            skipFilter: false,
            testFailureExitCode: 1,
            testNamePattern: undefined,
            testPathPattern: '',
            testResultsProcessor: undefined,
            testSequencer: '<homeDir>/code/kawaka/common/temp/node_modules/.pnpm/@jest+test-sequencer@26.6.3/node_modules/@jest/test-sequencer/build/index.js',
            testTimeout: undefined,
            updateSnapshot: 'new',
            useStderr: false,
            verbose: false,
            watch: false,
            watchAll: false,
            watchPlugins: undefined,
            watchman: true
          },
          hasDeprecationWarnings: false,
          wallaby: {
            roots: [ 'src' ],
            watchPathIgnorePatterns: [ '/node_modules/', '\\./dist/|\\./build/|\\./coverage/|\\./git/|/\\..+/' ],
            testPathIgnorePatterns: [ '/node_modules/', '\\./dist/|\\./build/|\\./coverage/|\\./git/|/\\..+/' ],
            testMatch: [ 'src/**/*.test.{ts,tsx}', 'src/**/?(*.)(spec|test|integration).ts?(x)' ],
            testRegex: []
          }
        }
      }
    },
    filesWithCoverageCalculated: [ 'src/**/*.{ts,tsx}', '!src/**/*.d.ts', '!src/**/*.test.{ts,tsx}', '!src/**/test/**', '!src/**/__tests__/**', '!src/**/__fixtures__/**', '!src/**/__mocks__/**' ],
    filesWithNoCoverageCalculated: [],
    globalSetup: false,
    micromatch: true,
    files: [
      { pattern: '/node_modules/', regexp: /\/node_modules\//, ignore: true, trigger: true, load: true },
      { pattern: '\\./dist/|\\./build/|\\./coverage/|\\./git/|/\\..+/', regexp: /\.\/dist\/|\.\/build\/|\.\/coverage\/|\.\/git\/|\/\..+\//, ignore: true, trigger: true, load: true },
      { pattern: '**/**', ignore: false, trigger: true, load: true, order: 1 },
      { pattern: 'src/**', ignore: false, trigger: true, load: true, order: 2 },
      { pattern: 'src/**/*.test.{ts,tsx}', ignore: true, trigger: true, load: true },
      { pattern: 'src/**/?(*.)(spec|test|integration).ts?(x)', ignore: true, trigger: true, load: true }
    ],
    tests: [
      { pattern: '/node_modules/', regexp: /\/node_modules\//, ignore: true, trigger: true, load: true, test: true },
      {
        pattern: '\\./dist/|\\./build/|\\./coverage/|\\./git/|/\\..+/',
        regexp: /\.\/dist\/|\.\/build\/|\.\/coverage\/|\.\/git\/|\/\..+\//,
        ignore: true,
        trigger: true,
        load: true,
        test: true
      },
      { pattern: 'src/**/*.test.{ts,tsx}', ignore: false, trigger: true, load: true, test: true, order: 3 },
      { pattern: 'src/**/?(*.)(spec|test|integration).ts?(x)', ignore: false, trigger: true, load: true, test: true, order: 4 }
    ],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    addModifiedTestFileToExclusiveTestRun: true,
    compilers: {},
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    workers: { initial: 0, regular: 0, recycle: false },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    mapConsoleMessagesStackTrace: false,
    extensions: {},
    env: { type: 'node', params: {}, runner: '<homeDir>/.nvm/versions/node/v12.22.1/bin/node', viewportSize: { width: 800, height: 600 }, options: { width: 800, height: 600 }, bundle: true },
    reportUnhandledPromises: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    loose: true,
    configCode: 'module.exports = {\n  autoDetect: true,\n  testFramework: {\n    configFile: "./jest.config.js",\n    path: "./node_modules/jest",\n  },\n  trace: true,\n};\n'
  },
  packageJSON: {
    dependencies: {
      electron: '12.0.4',
      '@sentry/electron': '2.4.1',
      '@kx/kart-client': 'workspace:*',
      '@kx/kart-track': 'workspace:*',
      '@kx/utils': 'workspace:*',
      '@kx/diff-pipeline': 'workspace:*',
      '@kx/rimu-gql-client': 'workspace:*',
      '@kx/sno-shovel': 'workspace:*',
      '@kx/xstate-tree': 'workspace:*',
      lodash: '4.17.15',
      '@kx/snohub-ui': 'workspace:*',
      '@apollo/client': '3.3.11',
      rxjs: '6.6.2',
      zod: '1.10.2',
      'utility-types': '3.10.0',
      xstate: '4.20.1',
      moment: '2.27.0',
      'node-fetch': '2.6.1',
      uuid: '8.3.0',
      '@kx/models': 'workspace:*',
      'electron-updater': '4.3.1',
      graphql: '15.3.0',
      '@kx/moss': 'workspace:*',
      react: '16.14.0',
      'markdown-to-jsx': '6.11.4'
    },
    devDependencies: {
      '@kx/eslint-config': 'workspace:*',
      eslint: '7.23.0',
      '@kx/rush-rig': 'workspace:*',
      '@rushstack/heft': '0.33.0',
      '@types/heft-jest': '1.0.2',
      '@types/lodash': '4.14.149',
      '@xstate/test': '0.4.0',
      keytar: '6.0.1',
      'electron-log': '4.2.0',
      '@types/uuid': '8.3.0',
      '@types/node-fetch': '2.5.7',
      'electron-store': '7.0.2',
      '@types/react': '16.9.46',
      '@types/markdown-to-jsx': '6.11.0',
      '@emotion/styled': '10.0.27',
      'tsconfig-replace-paths': '0.0.5',
      '@jest-runner/electron': '3.0.1',
      jest: '26.6.3',
      'ts-jest': '26.4.0'
    }
  },
  fs: { numberOfFiles: 89 },
  debug: [
    '2021-07-07T02:05:28.187Z angular/cli config Angular CLI not found.\n',
    '2021-07-07T02:05:28.361Z jest/config Detected Jest.\n',
    '2021-07-07T02:05:28.362Z jest/config Configured Jest.\n',
    '2021-07-07T02:05:28.363Z project Wallaby Node version: v12.22.1\n',
    '2021-07-07T02:05:28.363Z project Wallaby config: <homeDir>/code/kawaka/packages/rimu/shared/wallaby.js\n',
    '2021-07-07T02:05:28.503Z project File cache: <homeDir>/.vscode/extensions/wallabyjs.wallaby-vscode-1.0.297/projects/eefd0780397dcb85\n',
    '2021-07-07T02:05:28.558Z uiService Listening port 51235\n',
    '2021-07-07T02:05:28.564Z project package.json file change detected, invalidating local cache\n',
    '2021-07-07T02:05:28.584Z workers Parallelism for initial run: 10, for regular run: 5\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #0\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #1\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #2\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #3\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #4\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #5\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #6\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #7\n',
    '2021-07-07T02:05:28.584Z workers Starting run worker instance #8\n',
    '2021-07-07T02:05:28.585Z workers Starting run worker instance #9\n',
    '2021-07-07T02:05:28.585Z workers Web server is listening at 34147\n',
    '2021-07-07T02:05:28.585Z project File cache requires some updates, waiting required files from IDE\n',
    '2021-07-07T02:05:28.694Z project Stopping process pool\n',
    '2021-07-07T02:05:28.695Z project Test run started; run priority: 3\n',
    '2021-07-07T02:05:28.696Z project Running all tests\n',
    '2021-07-07T02:05:28.700Z workers Starting test run, priority: 3\n',
    '2021-07-07T02:05:28.701Z nodeRunner Starting sandbox [worker #0, session #xo7fp]\n',
    '2021-07-07T02:05:28.701Z nodeRunner Preparing sandbox [worker #0, session #xo7fp]\n',
    '2021-07-07T02:05:28.716Z workers Started run worker instance (delayed) #1\n',
    '2021-07-07T02:05:28.722Z workers Started run worker instance (delayed) #2\n',
    '2021-07-07T02:05:28.722Z workers Started run worker instance (delayed) #0\n',
    '2021-07-07T02:05:28.722Z nodeRunner Prepared sandbox [worker #0, session #xo7fp]\n',
    '2021-07-07T02:05:28.722Z workers [worker #0, session #xo7fp] Running tests in sandbox\n',
    '2021-07-07T02:05:28.739Z workers Started run worker instance (delayed) #3\n',
    '2021-07-07T02:05:28.739Z workers Started run worker instance (delayed) #4\n',
    '2021-07-07T02:05:28.753Z workers Started run worker instance (delayed) #5\n',
    '2021-07-07T02:05:28.759Z workers Started run worker instance (delayed) #6\n',
    '2021-07-07T02:05:28.776Z workers Started run worker instance (delayed) #8\n',
    '2021-07-07T02:05:28.784Z workers Started run worker instance (delayed) #7\n',
    '2021-07-07T02:05:28.795Z workers Started run worker instance (delayed) #9\n',
    '2021-07-07T02:05:28.923Z workers Scheduling Jest Test Run (xo7fp): 2021-07-07T02:05:28.919Z\n',
    '2021-07-07T02:05:28.923Z workers Jest Test Error: 2021-07-07T02:05:28.921Z\n',
    '2021-07-07T02:05:28.929Z workers Jest Test Run Complete (xo7fp): 2021-07-07T02:05:28.921Z\n',
    '2021-07-07T02:05:28.930Z workers [xo7fp] Run 0 test(s), skipped 0 test(s)\n',
    '2021-07-07T02:05:28.933Z workers [xo7fp] Sandbox is responsive, closing it\n',
    '2021-07-07T02:05:28.933Z project Test run finished\n',
    '2021-07-07T02:05:28.934Z project Processed console.log entries\n',
    '2021-07-07T02:05:28.934Z project Processed loading sequences\n',
    '2021-07-07T02:05:28.934Z project Processed executed tests\n',
    '2021-07-07T02:05:28.934Z project Processed code coverage\n',
    '2021-07-07T02:05:28.937Z project Test run result processed and sent to IDE\n'
  ]
}

[ArtemGovorov]

Does Wallaby support using alternative Jest testRunners?

At the moment Wallaby doesn't support @jest-runner/electron (the testRunner is specifically limited to jest-runner and jest-runner-groups because of that).

or is this something you are willing to add support for?

It's not something that we have in our plans at the moment.

Mind sharing specific reasons behind your decision to use @jest-runner/electron instead of the jest-runner?

[UberMouse]

Mind sharing specific reasons behind your decision to use @jest-runner/electron instead of the jest-runner?

I need to test code that is designed to be run inside of the Electron Renderer process environment, which is what @jest-runner/electron provides.