Tests runs with Karma but are failing with wallaby

[IT-CASADO]

Issue description or question

I'm trying to use the monaco-editor in an angular app. Unfortunately I'm getting following error while running tests with wallaby. This error itself is coming from Monaco-Editor.

    Failed: Uncaught (in promise): Error: Unexpected usage 
    Error: Unexpected usage 
        at EditorSimpleWorker.loadForeignModule (http://localhost:65376/____wallaby-bundle.js?1663837776139&wallabyFileId=bundle:117317:31) 
        at http://localhost:65376/____wallaby-bundle.js?1663837776139&wallabyFileId=bundle:241098:30 
        at _ZoneDelegate.invoke (http://localhost:65376/____wallaby-bundle.js?1663837776139&wallabyFileId=bundle:3572:30) 
...

But if I run the same tests with Karma I don't get any errors.

Repro steps:

1. Clone this repository: https://github.com/IT-CASADO/monaco-wallaby-error
2. Open folder "src\evo" in VS Code
3. start wallaby --> some unit tests are red (failing)
4. run "yarn test code-editors" -- all unit test are green

I don't know if this issue is only related to Wallaby or also something with Monaco-Editor. It seems that Monaco is using some hacks with ESM comments.

Do you expect that my tests should also run in wallaby with automatic configuration?

Wallaby diagnostics report

{
  editorVersion: '1.71.2',
  pluginVersion: '1.0.347',
  editorType: 'VSCode',
  osVersion: 'win32 10.0.19044',
  nodeVersion: 'v16.15.0',
  coreVersion: '1.0.1330',
  checksum: 'YWIyZjM0OWFiOWY3MDRhNmZlOTdlZTI3NDZmM2Y1NjksMTY4OTM3OTIwMDAwMCww',
  config: {
    diagnostics: {
      angular: {
        workspace: {
          '$schema': './node_modules/nx/schemas/nx-schema.json',
          version: 1,
          projects: {
            'code-editors': {
              root: 'libs/code-editors',
              projectType: 'library',
              sourceRoot: 'libs/code-editors/src',
              prefix: 'evo',
              architect: {
                build: {
                  builder: '@nrwl/angular:ng-packagr-lite',
                  outputs: [ 'dist/libs/code-editors' ],
                  options: { project: 'libs/code-editors/ng-package.json' },
                  configurations: { production: { tsConfig: 'libs/code-editors/tsconfig.lib.prod.json' }, development: { tsConfig: 'libs/code-editors/tsconfig.lib.json' } },
                  defaultConfiguration: 'production'
                },
                test: {
                  builder: '@angular-devkit/build-angular:karma',
                  options: {
                    main: 'libs/code-editors/src/test.ts',
                    tsConfig: 'libs/code-editors/tsconfig.spec.json',
                    karmaConfig: 'libs/code-editors/karma.conf.js',
                    webWorkerTsConfig: 'apps/magic-code/tsconfig.worker.json'
                  }
                },
                lint: { builder: '@nrwl/linter:eslint', options: { lintFilePatterns: [ 'libs/code-editors/**/*.ts', 'libs/code-editors/**/*.html' ] } },
                storybook: {
                  builder: '@storybook/angular:start-storybook',
                  options: { port: 4400, configDir: 'libs/code-editors/.storybook', browserTarget: 'code-editors:build-storybook', compodoc: false },
                  configurations: { ci: { quiet: true } }
                },
                'build-storybook': {
                  builder: '@storybook/angular:build-storybook',
                  outputs: [ '{options.outputPath}' ],
                  options: {
                    outputDir: 'dist/storybook/code-editors',
                    configDir: 'libs/code-editors/.storybook',
                    browserTarget: 'code-editors:build-storybook',
                    compodoc: false,
                    styles: [ 'libs/code-editors/src/styles/scss/common.scss' ]
                  },
                  configurations: { ci: { quiet: true } }
                }
              },
              tags: []
            },
            'magic-code': {
              root: 'apps/magic-code',
              '$schema': '..\\..\\node_modules\\nx\\schemas\\project-schema.json',
              projectType: 'application',
              sourceRoot: 'apps/magic-code/src',
              prefix: 'evo',
              architect: {
                build: {
                  builder: '@nrwl/angular:webpack-browser',
                  outputs: [ '{options.outputPath}' ],
                  options: {
                    outputPath: 'dist/apps/magic-code',
                    index: 'apps/magic-code/src/index.html',
                    main: 'apps/magic-code/src/main.ts',
                    polyfills: 'apps/magic-code/src/polyfills.ts',
                    tsConfig: 'apps/magic-code/tsconfig.app.json',
                    customWebpackConfig: { path: 'apps/magic-code/webpack.config.js' },
                    inlineStyleLanguage: 'scss',
                    assets: [ 'apps/magic-code/src/favicon.ico', 'apps/magic-code/src/assets' ],
                    styles: [ 'apps/magic-code/src/styles/scss/styles.scss', 'libs/code-editors/src/styles/scss/common.scss' ],
                    scripts: [],
                    webWorkerTsConfig: 'apps/magic-code/tsconfig.worker.json'
                  },
                  configurations: {
                    production: {
                      budgets: [ { type: 'initial', maximumWarning: '500kb', maximumError: '5mb' }, { type: 'anyComponentStyle', maximumWarning: '2kb', maximumError: '4kb' } ],
                      fileReplacements: [ { replace: 'apps/magic-code/src/environments/environment.ts', with: 'apps/magic-code/src/environments/environment.prod.ts' } ],
                      optimization: { scripts: true, fonts: true, styles: { minify: true, inlineCritical: false } },
                      outputHashing: 'all'
                    },
                    development: { buildOptimizer: false, optimization: false, vendorChunk: true, extractLicenses: false, sourceMap: true, namedChunks: true }
                  },
                  defaultConfiguration: 'production'
                },
                serve: {
                  builder: '@nrwl/angular:webpack-server',
                  configurations: { production: { browserTarget: 'magic-code:build:production' }, development: { browserTarget: 'magic-code:build:development' } },
                  defaultConfiguration: 'development'
                },
                'extract-i18n': { builder: '@angular-devkit/build-angular:extract-i18n', options: { browserTarget: 'magic-code:build' } },
                lint: { builder: '@nrwl/linter:eslint', options: { lintFilePatterns: [ 'apps/magic-code/**/*.ts', 'apps/magic-code/**/*.html' ] } },
                test: {
                  builder: '@angular-devkit/build-angular:karma',
                  options: {
                    main: 'apps/magic-code/src/test.ts',
                    tsConfig: 'apps/magic-code/tsconfig.spec.json',
                    karmaConfig: 'apps/magic-code/karma.conf.js',
                    webWorkerTsConfig: 'apps/magic-code/tsconfig.worker.json'
                  }
                },
                storybook: {
                  builder: '@storybook/angular:start-storybook',
                  options: { port: 4400, configDir: 'apps/magic-code/.storybook', browserTarget: 'magic-code:build', compodoc: false },
                  configurations: { ci: { quiet: true } }
                },
                'build-storybook': {
                  builder: '@storybook/angular:build-storybook',
                  outputs: [ '{options.outputPath}' ],
                  options: {
                    outputDir: 'dist/storybook/magic-code',
                    configDir: 'apps/magic-code/.storybook',
                    browserTarget: 'magic-code:build',
                    compodoc: false,
                    styles: [ 'apps/magic-code/src/styles/scss/styles.scss' ]
                  },
                  configurations: { ci: { quiet: true } }
                }
              },
              tags: []
            }
          },
          npmScope: 'evo',
          affected: { defaultBase: 'master' },
          cli: { defaultCollection: '@nrwl/angular', packageManager: 'npm' },
          implicitDependencies: { 'package.json': { dependencies: '*', devDependencies: '*' }, '.eslintrc.json': '*' },
          tasksRunnerOptions: {
            default: { runner: 'nx/tasks-runners/default', options: { cacheableOperations: [ 'build', 'lint', 'test', 'e2e', 'build-storybook' ] } }
          },
          targetDependencies: { build: [ { target: 'build', projects: 'dependencies' } ] },
          schematics: {
            '@nrwl/angular:application': { style: 'scss', linter: 'eslint', unitTestRunner: 'karma', e2eTestRunner: 'cypress' },
            '@nrwl/angular:library': { linter: 'eslint', unitTestRunner: 'karma' },
            '@nrwl/angular:component': { style: 'scss' }
          },
          defaultProject: 'magic-code',
          configFileName: 'C:\\Code\\EVO\\GIT\\Prototyping.CodeEditors\\src\\evo\\angular.json',
          nxWorkspace: true
        },
        tsConfig: '{"compileOnSave":false,"compilerOptions":{"rootDir":".","sourceMap":true,"declaration":false,"moduleResolution":"node","allowSyntheticDefaultImports":true,"emitDecoratorMetadata":true,"experimentalDecorators":true,"importHelpers":true,"target":"es2015","module":"esnext","lib":["es2018","dom"],"skipLibCheck":true,"skipDefaultLibCheck":true,"baseUrl":".","paths":{"@evo/code-editors":["libs/code-editors/src/index.ts"]},"types":["jasmine","node"]},"exclude":["node_modules","tmp"],"files":["libs/code-editors/src/test.ts","apps/magic-code/src/test.ts"],"include":["libs/code-editors/**/*.spec.ts","libs/code-editors/**/*.test.ts","libs/code-editors/**/*.d.ts","apps/magic-code/**/*.test.ts","apps/magic-code/**/*.spec.ts","apps/magic-code/**/*.d.ts"]}',
        polyfill: {
          path: 'C:\\Code\\EVO\\GIT\\Prototyping.CodeEditors\\src\\evo\\wallaby-polyfills.js',
          code: "require('C:/Code/EVO/GIT/Prototyping.CodeEditors/src/evo/node_modules/@angular-devkit/build-angular/node_modules/core-js/proposals/reflect-metadata.js');"
        },
        main: [
          {
            path: 'C:/Code/EVO/GIT/Prototyping.CodeEditors/src/evo/libs/code-editors/src/test.ts',
            code: '// This file is required by karma.conf.js and loads recursively all the .spec and framework files\r\n' +
              '\r\n' +
              "import 'zone.js/dist/zone';\r\n" +
              '\r\n' +
              "import 'zone.js/dist/zone-testing';\r\n" +
              "import { getTestBed } from '@angular/core/testing';\r\n" +
              "import { BrowserDynamicTestingModule, platformBrowserDynamicTesting } from '@angular/platform-browser-dynamic/testing';\r\n" +
              '\r\n' +
              'declare const require: any;\r\n' +
              '\r\n' +
              '// First, initialize the Angular testing environment.\r\n' +
              'getTestBed().initTestEnvironment(BrowserDynamicTestingModule, platformBrowserDynamicTesting());\r\n' +
              '// Then we find all the tests.\r\n' +
              '\r\n' +
              '// And load the modules.\r\n' +
              '\r\n'
          },
          {
            path: 'C:/Code/EVO/GIT/Prototyping.CodeEditors/src/evo/apps/magic-code/src/test.ts',
            code: '// This file is required by karma.conf.js and loads recursively all the .spec and framework files\r\n' +
              '\r\n' +
              "import 'zone.js/dist/zone';\r\n" +
              '\r\n' +
              "import 'zone.js/dist/zone-testing';\r\n" +
              '\r\n' +
              '\r\n' +
              '\r\n' +
              '\r\n' +
              '\r\n' +
              '// First, initialize the Angular testing environment.\r\n' +
              '\r\n' +
              '// Then we find all the tests.\r\n' +
              '\r\n' +
              '// And load the modules.\r\n' +
              '\r\n'
          }
        ]
      }
    },
    testFramework: { version: 'jasmine@3.10.0', configurator: 'jasmine@2.1.3', reporter: 'jasmine@2.1.3', starter: 'jasmine@2.1.3', autoDetected: true },
    env: { kind: 'chrome', type: 'browser', params: {}, viewportSize: { width: 800, height: 600 }, options: { width: 800, height: 600 }, bundle: true },
    files: [
      { pattern: 'libs/code-editors/src/test.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'libs/code-editors/src/test.base.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'libs/code-editors/src/test.wallaby.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'libs/code-editors/src/**/*.spec.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'libs/code-editors/src/**/*.+(ts|js)', load: false, ignore: false, trigger: true, order: 1 },
      { pattern: 'libs/code-editors/src/**/*.+(css|less|scss|sass|styl|html|json|svg)', instrument: false, load: false, ignore: false, trigger: true, order: 2 },
      { pattern: 'apps/magic-code/src/test.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'apps/magic-code/src/main.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'apps/magic-code/src/index.html', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'apps/magic-code/src/test.base.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'apps/magic-code/src/test.wallaby.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'apps/magic-code/src/**/*.spec.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'apps/magic-code/src/**/*.+(ts|js)', load: false, ignore: false, trigger: true, order: 3 },
      { pattern: 'apps/magic-code/src/**/*.+(css|less|scss|sass|styl|html|json|svg)', instrument: false, load: false, ignore: false, trigger: true, order: 4 }
    ],
    tests: [
      { pattern: 'libs/code-editors/src/**/*.spec.ts', load: false, ignore: false, trigger: true, test: true, order: 5 },
      { pattern: 'apps/magic-code/src/**/*.spec.ts', load: false, ignore: false, trigger: true, test: true, order: 6 }
    ],
    filesWithNoCoverageCalculated: [],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    compilers: {},
    logLimits: { inline: { depth: 5, elements: 5000 }, values: { default: { stringLength: 200 }, autoExpand: { elements: 5000, stringLength: 8192, depth: 10 } } },
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    workers: { initial: 0, regular: 0, recycle: false },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    mapConsoleMessagesStackTrace: false,
    extensions: {},
    reportUnhandledPromises: false,
    throwOnBeforeUnload: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    loose: true,
    configCode: 'auto.detect#-51327910'
  },
  packageJSON: {
    dependencies: {
      '@angular/animations': '~13.3.0',
      '@angular/common': '~13.3.0',
      '@angular/compiler': '~13.3.0',
      '@angular/core': '~13.3.0',
      '@angular/forms': '~13.3.0',
      '@angular/platform-browser': '~13.3.0',
      '@angular/platform-browser-dynamic': '~13.3.0',
      '@angular/router': '~13.3.0',
      '@nrwl/angular': '14.1.9',
      '@codemirror/view': '~6.2.2',
      '@codemirror/state': '~6.1.1',
      '@codemirror/language': '~6.2.1',
      '@codemirror/lang-javascript': '~6.0.2',
      '@codemirror/lang-json': '~6.0.0',
      codemirror: '~6.0.1',
      'monaco-editor': '^0.34.0',
      'monaco-editor-core': '^0.34.0',
      'monaco-editor-webpack-plugin': '^7.0.1',
      rxjs: '^6.6.7',
      subsink: '^1.0.2',
      tslib: '^2.0.0',
      'zone.js': '~0.11.4'
    },
    devDependencies: {
      '@angular-devkit/build-angular': '~13.3.0',
      '@angular-eslint/eslint-plugin': '~13.1.0',
      '@angular-eslint/eslint-plugin-template': '~13.1.0',
      '@angular-eslint/template-parser': '~13.1.0',
      '@angular/cli': '~13.3.0',
      '@angular/compiler-cli': '~13.3.0',
      '@angular/language-service': '~13.3.0',
      '@nrwl/cli': '14.1.9',
      '@nrwl/cypress': '14.1.9',
      '@nrwl/eslint-plugin-nx': '14.1.9',
      '@nrwl/jest': '14.1.9',
      '@nrwl/linter': '14.1.9',
      '@nrwl/storybook': '14.1.9',
      '@nrwl/workspace': '14.1.9',
      '@storybook/addon-essentials': '~6.5.10',
      '@storybook/angular': '~6.5.10',
      '@storybook/builder-webpack5': '~6.5.10',
      '@storybook/core-server': '~6.5.10',
      '@storybook/manager-webpack5': '~6.5.10',
      '@types/jasmine': '~3.5.0',
      '@types/jest': '27.4.1',
      '@types/lodash-es': '^4.17.6',
      '@types/node': '16.11.7',
      '@typescript-eslint/eslint-plugin': '~5.18.0',
      '@typescript-eslint/parser': '~5.18.0',
      antlr4ts: '^0.5.0-alpha.4',
      'antlr4ts-cli': '0.5.0-alpha.4',
      cypress: '^9.1.0',
      eslint: '~8.12.0',
      'eslint-config-prettier': '8.1.0',
      'eslint-plugin-cypress': '^2.10.3',
      'eslint-plugin-filenames': '^1.3.2',
      'eslint-plugin-import': '^2.26.0',
      'eslint-plugin-rxjs': '^5.0.2',
      'eslint-plugin-rxjs-angular': '^2.0.0',
      'eslint-plugin-simple-import-sort': '^8.0.0',
      'eslint-plugin-unused-imports': '^2.0.0',
      'jasmine-core': '~3.10.0',
      'jasmine-spec-reporter': '~5.0.0',
      jest: '27.5.1',
      'jest-preset-angular': '11.1.1',
      karma: '~6.3.0',
      'karma-chrome-launcher': '~3.1.0',
      'karma-coverage': '~2.2.0',
      'karma-jasmine': '~4.0.0',
      'karma-jasmine-html-reporter': '~1.7.0',
      'karma-spec-reporter': '~0.0.34',
      'karma-summary-reporter': '~3.1.1',
      'lodash-es': '^4.17.21',
      'ng-packagr': '^13.0.0',
      nx: '14.1.9',
      postcss: '^8.4.5',
      'postcss-import': '~14.1.0',
      'postcss-preset-env': '~7.5.0',
      'postcss-url': '~10.1.3',
      prettier: '2.5.1',
      'ts-jest': '27.1.4',
      'ts-node': '9.1.1',
      typescript: '~4.6.2',
      'vscode-languageserver-types': '^3.17.2',
      'wallaby-webpack': '^3.9.16',
      webpack: '^5.64.0'
    }
  },
  fs: { numberOfFiles: 67 },
  debug: [
    '2022-09-22T07:41:24.607Z config Attempting automatic configuration for angular\n',
    '2022-09-22T07:41:24.792Z angular/cli config Detected Angular CLI.\n',
    '2022-09-22T07:41:24.796Z angular/cli config Angular currentDirPath: "C:\\Code\\EVO\\GIT\\Prototyping.CodeEditors\\src\\evo", workspaceDirPath: "C:\\Code\\EVO\\GIT\\Prototyping.CodeEditors\\src\\evo"\n',
    '2022-09-22T07:41:27.499Z angular/cli karma workspace config Using virtual C:/Code/EVO/GIT/Prototyping.CodeEditors/src/evo/libs/code-editors/src/test.ts.\n',
    '2022-09-22T07:41:27.501Z angular/cli karma workspace config Using virtual C:/Code/EVO/GIT/Prototyping.CodeEditors/src/evo/apps/magic-code/src/test.ts.\n',
    '2022-09-22T07:41:27.503Z angular/cli karma workspace config Using virtual C:\\Code\\EVO\\GIT\\Prototyping.CodeEditors\\src\\evo\\tsconfig.wallaby.spec.json.\n',
    '2022-09-22T07:41:28.583Z config Finished attempting automatic configuration for angular (3976ms)\n',
    '2022-09-22T07:41:28.587Z project Wallaby Node version: v16.15.0\n',
    '2022-09-22T07:41:28.587Z project Wallaby config: C:\\Code\\EVO\\GIT\\Prototyping.CodeEditors\\src\\evo\\auto.detect\n',
    '2022-09-22T07:41:28.603Z fs File system starting\n',
    '2022-09-22T07:41:28.742Z fs File system scan completed\n',
    '2022-09-22T07:41:28.749Z project File cache: C:\\Users\\CasadoDaniel\\.vscode\\extensions\\wallabyjs.wallaby-vscode-1.0.347\\projects\\fbe36356901da563\n',
    '2022-09-22T07:41:28.916Z uiService Listening port 51236\n',
    '2022-09-22T07:41:28.925Z project package.json file change detected, invalidating local cache\n',
    '2022-09-22T07:41:28.971Z workers Parallelism for initial run: 14, for regular run: 7\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #0\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #1\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #2\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #3\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #4\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #5\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #6\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #7\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #8\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #9\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #10\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #11\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #12\n',
    '2022-09-22T07:41:28.972Z workers Starting run worker instance #13\n',
    '2022-09-22T07:41:28.974Z workers Web server is listening at 60146\n',
    '2022-09-22T07:41:28.983Z project Stopping process pool\n',
    '2022-09-22T07:41:28.983Z project File cache is up-to-date, starting full test run\n',
    '2022-09-22T07:41:28.986Z project Running postprocessor\n',
    '2022-09-22T07:41:28.990Z postprocessor Processing started.\n',
    '2022-09-22T07:41:28.990Z postprocessor Initializing.\n',
    '2022-09-22T07:41:28.990Z postprocessor Initializing entry.\n',
    '2022-09-22T07:41:28.991Z postprocessor Sanitizing configuration.\n',
    '2022-09-22T07:41:28.991Z postprocessor Adjusting configuration.\n',
    '2022-09-22T07:41:28.995Z postprocessor Creating compiler.\n',
    '2022-09-22T07:41:29.346Z postprocessor Customizing compiler.\n',
    '2022-09-22T07:41:29.347Z postprocessor Compilation started.\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #0\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #1\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #2\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #3\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #4\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #5\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #6\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #7\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #8\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #9\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #10\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #11\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #12\n',
    '2022-09-22T07:41:37.112Z workers Started run worker instance (immediate) #13\n',
    '2022-09-22T07:41:46.195Z postprocessor Compilation finished.\n',
    '2022-09-22T07:41:46.905Z postprocessor Emitting 2054 files.\n',
    '2022-09-22T07:41:46.917Z postprocessor Processing finished.\n',
    '2022-09-22T07:41:47.053Z project Postprocessor execution finished\n',
    '2022-09-22T07:41:47.053Z project Test run started; run priority: 3\n',
    '2022-09-22T07:41:47.056Z project Running all tests\n',
    '2022-09-22T07:41:47.078Z workers Starting test run, priority: 3\n',
    '2022-09-22T07:41:47.078Z workers Distributing tests between 14 workers\n',
    '2022-09-22T07:41:47.078Z workers Running tests in parallel\n',
    '2022-09-22T07:41:47.079Z chromeRunner Starting sandbox [worker #0, session #odwb2]\n',
    '2022-09-22T07:41:47.079Z chromeRunner Starting sandbox [worker #1, session #xg3l1]\n',
    '2022-09-22T07:41:47.079Z chromeRunner Starting sandbox [worker #2, session #0kteo]\n',
    '2022-09-22T07:41:47.079Z chromeRunner Starting sandbox [worker #3, session #mcdoa]\n',
    '2022-09-22T07:41:47.079Z chromeRunner Starting sandbox [worker #4, session #1crvl]\n',
    '2022-09-22T07:41:47.079Z chromeRunner Starting sandbox [worker #5, session #z739t]\n',
    '2022-09-22T07:41:47.079Z chromeRunner Starting sandbox [worker #6, session #gim5v]\n',
    '2022-09-22T07:41:47.079Z chromeRunner Preparing sandbox [worker #0, session #odwb2]\n',
    '2022-09-22T07:41:47.081Z browserRunner Total files to load in sandbox: 5\n',
    '2022-09-22T07:41:47.083Z browserRunner Sandbox is generated [worker #0, session #odwb2]: http://localhost:60146/wallaby_sandbox0.html\n',
    '2022-09-22T07:41:47.083Z chromeRunner Preparing sandbox [worker #1, session #xg3l1]\n',
    '2022-09-22T07:41:47.084Z browserRunner Total files to load in sandbox: 5\n',
    '2022-09-22T07:41:47.085Z browserRunner Sandbox is generated [worker #1, session #xg3l1]: http://localhost:60146/wallaby_sandbox1.html\n',
    '2022-09-22T07:41:47.085Z chromeRunner Preparing sandbox [worker #2, session #0kteo]\n',
    '2022-09-22T07:41:47.086Z browserRunner Total files to load in sandbox: 5\n',
    '2022-09-22T07:41:47.088Z browserRunner Sandbox is generated [worker #2, session #0kteo]: http://localhost:60146/wallaby_sandbox2.html\n',
    '2022-09-22T07:41:47.088Z chromeRunner Preparing sandbox [worker #3, session #mcdoa]\n',
    '2022-09-22T07:41:47.089Z browserRunner Total files to load in sandbox: 5\n',
    '2022-09-22T07:41:47.091Z browserRunner Sandbox is generated [worker #3, session #mcdoa]: http://localhost:60146/wallaby_sandbox3.html\n',
    '2022-09-22T07:41:47.091Z chromeRunner Preparing sandbox [worker #4, session #1crvl]\n',
    '2022-09-22T07:41:47.091Z browserRunner Total files to load in sandbox: 5\n',
    '2022-09-22T07:41:47.094Z browserRunner Sandbox is generated [worker #4, session #1crvl]: http://localhost:60146/wallaby_sandbox4.html\n',
    '2022-09-22T07:41:47.094Z chromeRunner Preparing sandbox [worker #5, session #z739t]\n',
    '2022-09-22T07:41:47.096Z browserRunner Total files to load in sandbox: 5\n',
    '2022-09-22T07:41:47.099Z browserRunner Sandbox is generated [worker #5, session #z739t]: http://localhost:60146/wallaby_sandbox5.html\n',
    '2022-09-22T07:41:47.099Z chromeRunner Preparing sandbox [worker #6, session #gim5v]\n',
    '2022-09-22T07:41:47.100Z browserRunner Total files to load in sandbox: 5\n',
    '2022-09-22T07:41:47.102Z browserRunner Sandbox is generated [worker #6, session #gim5v]: http://localhost:60146/wallaby_sandbox6.html\n',
    '2022-09-22T07:41:47.103Z chromeRunner Prepared sandbox [worker #0, session #odwb2]\n',
    '2022-09-22T07:41:47.103Z chromeRunner Prepared sandbox [worker #1, session #xg3l1]\n',
    '2022-09-22T07:41:47.103Z chromeRunner Prepared sandbox [worker #2, session #0kteo]\n',
    '2022-09-22T07:41:47.103Z chromeRunner Prepared sandbox [worker #3, session #mcdoa]\n',
    '2022-09-22T07:41:47.103Z chromeRunner Prepared sandbox [worker #4, session #1crvl]\n',
    '2022-09-22T07:41:47.103Z chromeRunner Prepared sandbox [worker #5, session #z739t]\n',
    '2022-09-22T07:41:47.103Z chromeRunner Prepared sandbox [worker #6, session #gim5v]\n',
    '2022-09-22T07:41:47.103Z workers [worker #0, session #odwb2] Running tests in sandbox\n',
    '2022-09-22T07:41:47.104Z workers [worker #1, session #xg3l1] Running tests in sandbox\n',
    '2022-09-22T07:41:47.104Z workers [worker #2, session #0kteo] Running tests in sandbox\n',
    '2022-09-22T07:41:47.105Z workers [worker #3, session #mcdoa] Running tests in sandbox\n',
    '2022-09-22T07:41:47.105Z workers [worker #4, session #1crvl] Running tests in sandbox\n',
    '2022-09-22T07:41:47.105Z workers [worker #5, session #z739t] Running tests in sandbox\n',
    '2022-09-22T07:41:47.105Z workers [worker #6, session #gim5v] Running tests in sandbox\n',
    '2022-09-22T07:41:49.212Z workers [mcdoa] Loaded 5 test(s)\n',
    '2022-09-22T07:41:49.220Z workers [1crvl] Loaded 1 test(s)\n',
    '2022-09-22T07:41:49.225Z workers [z739t] Loaded 1 test(s)\n',
    '2022-09-22T07:41:49.235Z workers [0kteo] Loaded 1 test(s)\n',
    '2022-09-22T07:41:49.255Z workers [0kteo] Test executed: should have a module definition\n',
    '2022-09-22T07:41:49.257Z workers [z739t] Test executed: should create\n',
    '2022-09-22T07:41:49.310Z workers [z739t] Run 1 test(s), skipped 0 test(s)\n',
    '2022-09-22T07:41:49.313Z workers [0kteo] Run 1 test(s), skipped 0 test(s)\n',
    '2022-09-22T07:41:49.316Z workers [z739t] Sandbox is responsive, closing it\n',
    '2022-09-22T07:41:49.321Z workers [0kteo] Sandbox is responsive, closing it\n',
    '2022-09-22T07:41:49.363Z workers [mcdoa] Test executed: should create\n',
    '2022-09-22T07:41:49.378Z workers [1crvl] Test executed: t1\n',
    '2022-09-22T07:41:49.404Z workers [mcdoa] Test executed: should initialize internal value with initial value\n',
    '2022-09-22T07:41:49.420Z workers [1crvl] Run 1 test(s), skipped 0 test(s)\n',
    '2022-09-22T07:41:49.423Z workers [1crvl] Sandbox is responsive, closing it\n',
    '2022-09-22T07:41:49.448Z workers [mcdoa] Test executed: should initialize internal value with initial value\n',
    '2022-09-22T07:41:49.527Z workers [mcdoa] Test executed: should update internal value\n',
    '2022-09-22T07:41:49.545Z workers [mcdoa] Test executed: should create\n',
    '2022-09-22T07:41:49.581Z workers [mcdoa] Run 5 test(s), skipped 0 test(s)\n',
    '2022-09-22T07:41:49.583Z workers [mcdoa] Sandbox is responsive, closing it\n',
    '2022-09-22T07:41:49.645Z workers [gim5v] Loaded 5 test(s)\n',
    '2022-09-22T07:41:49.707Z workers [xg3l1] Loaded 3 test(s)\n',
    '2022-09-22T07:41:49.718Z workers [odwb2] Loaded 1 test(s)\n',
    '2022-09-22T07:41:49.862Z workers [odwb2] Test executed: should create the app\n',
    '2022-09-22T07:41:49.894Z workers [odwb2] Run 1 test(s), skipped 0 test(s)\n',
    '2022-09-22T07:41:49.896Z workers [odwb2] Sandbox is responsive, closing it\n',
    '2022-09-22T07:41:49.958Z workers [gim5v] Test executed: should create\n',
    '2022-09-22T07:41:49.993Z workers [xg3l1] Test executed: should initialize internal value with initial value\n',
    '2022-09-22T07:41:50.016Z workers [gim5v] Test executed: should update internal value\n',
    '2022-09-22T07:41:50.058Z workers [gim5v] Test executed: should initialize internal value with initial value\n',
    '2022-09-22T07:41:50.069Z workers [xg3l1] Test executed: should create\n',
    '2022-09-22T07:41:50.108Z workers [gim5v] Test executed: should initialize internal value with initial value\n',
    '2022-09-22T07:41:50.124Z workers [xg3l1] Test executed: should update internal value\n',
    '2022-09-22T07:41:50.143Z workers [gim5v] Test executed: should create\n',
    '2022-09-22T07:41:50.161Z workers [xg3l1] Run 3 test(s), skipped 0 test(s)\n',
    '2022-09-22T07:41:50.163Z workers [xg3l1] Sandbox is responsive, closing it\n',
    '2022-09-22T07:41:50.179Z workers [gim5v] Run 5 test(s), skipped 0 test(s)\n',
    '2022-09-22T07:41:50.180Z workers [gim5v] Sandbox is responsive, closing it\n',
    '2022-09-22T07:41:50.181Z workers Merging parallel test run results\n',
    '2022-09-22T07:41:50.188Z project Test run finished\n',
    '2022-09-22T07:41:50.189Z project Processed console.log entries\n',
    '2022-09-22T07:41:50.190Z project Processed loading sequences\n',
    '2022-09-22T07:41:50.192Z project Processed executed tests\n',
    '2022-09-22T07:41:50.198Z project Processed code coverage\n',
    '2022-09-22T07:41:50.241Z project Test run result processed and sent to IDE\n'
  ]
}

[smcenlly]

It looks like right now Wallaby's webpack integration doesn't support what you're trying to do with Monaco editor and how it loads/runs.

This is going to require a deeper investigation by our team. We'll look at it next week and get back to you.

[smcenlly]

Our team has spent most of the day investigating your issue and it looks like the fact that it's working for you may be a bug or problem with nx caching.

When we originally ran yarn test code-editors it was working for us but we think that the result was a cached version of a previous test that had previously passed. When we run:

yarn test code-editors --skip-nx-cache

The tests always fail. We tried running npx nx run-many --target=test --all and then for some reason the tests start passing after multiple executions. We're not sure why.

We need you to check whether

yarn test code-editors --skip-nx-cache

is also failing for you?

If so, we need you to fix the project so that these tests pass without using the cache before we can continue troubleshooting. Once they're working, you can try Wallaby again and it may work without any issues. If it doesn't work, at least we will understand how your project works from the command-line and will be able to proceed with troubleshooting Wallaby.

[IT-CASADO]

Skipping NX cache on my side doesn't change anything for me (same for one of my colleague). All test are always green (success) with Karma. No matter how often I run the tests....

[IT-CASADO]

Okay, I just cloned and run the project on my private machine and curiously I see immediately 3 of 7 tests failing.

And furthermore on this machine it is ongoing repeatable both with

yarn test code-editors --skip-nx-cache

and also with

yarn test code-editors.

Need to do more investigation on this (although I don't really know where to start ;))

[smcenlly]

although I don't really know where to start

I'm guessing that somewhere someone else has configured NX to run/test with Monaco. The webpack plugin I saw in your project looked like it was what you might have needed (i.e. custom webpack configuration).

It's a little outside providing standard support, but if you can't work it out in a day or two, let us know and someone in our team can take a look for you. Would probably take a day or two for us to work out.

[IT-CASADO]

I noticed that I see those errors also permanently on my work machine when I run this:

yarn test code-editors --skip-nx-cache --watch

Side note: To prevent caching issues I opened browser developers tools for Karma window ;)

Now I'm applying the same custom webpack config to all Karma tests and all tests run now fine with Karma.

Unfortunately Wallaby is still showing me failing tests.

[smcenlly]

Quick update on this issue for you @IT-CASADO. We have been investigating why Wallaby isn't working for you.

[NikGovorov]

Unfortunately Monaco does not support loading workers on the main thread at the moment. On the other hand, Wallaby does not support running code outside of the main thread, there are several limitations with our Webpack integration and coverage collecting issues for the code running in a worker context.

Therefore our recommendation is to mock ITextModel and IStandaloneCodeEditor in your tests to make them less integration tests but more unit tests. Additionally you may also exclude the tests which load Monaco from Wallaby execution.