Wallaby now errors on code/tests that haven't changed

[jupav]

Wallaby now errors on code that hasn't changed

Using Angular 17.3.0 and Wallaby 1.0.292 (JetBrains plugin), we're now getting consistent errors where we weren't getting errors before: Error: Timeout - Async function did not complete within 5000ms.

Tests all run fine with karma/jasmine, but wallaby fails.

We have also checked out a branch from over a week ago, which used to pass 100% but that now fails with the same errors also, so we assume something in Wallaby has changed to cause these errors.

Wallaby diagnostics report

{
  editorVersion: 'IntelliJ IDEA 2024.2.0.1',
  pluginVersion: '1.0.292',
  editorType: 'IntelliJ',
  osVersion: 'win32 10.0.22631',
  nodeVersion: 'v20.12.1',
  coreVersion: '1.0.1594',
  pnp: '<not set>',
  pnpEsm: '<not set>',
  checksum: 'MGRmNThkNzA3ODFiOWNmMDhiNGQ4OTczMTEzYzZhMGMsMTcyNzgyNzIwMDAwMCww',
  config: {
    diagnostics: {
      angular: {
        workspace: {
          '$schema': './node_modules/@angular/cli/lib/config/schema.json',
          version: 1,
          newProjectRoot: 'projects',
          projects: {
            app: {
              projectType: 'application',
              schematics: { '@schematics/angular:component': { style: 'sass' }, '@nrwl/schematics:component': { styleext: 'sass', spec: true, flat: false, changeDetection: 'OnPush' } },
              root: '',
              sourceRoot: 'src',
              prefix: 'app',
              i18n: { sourceLocale: 'en-US', locales: { 'en-GB': 'locale/rule.en-GB.xlf', nl: 'locale/rule.nl.xlf' } },
              architect: {
                build: {
                  builder: '@angular-devkit/build-angular:browser',
                  options: {
                    allowedCommonJsDependencies: [ 'lodash', 'uuid' ],
                    outputPath: 'dist/app',
                    index: 'src/index.html',
                    main: 'src/main.ts',
                    polyfills: 'src/polyfills.ts',
                    tsConfig: 'tsconfig.app.json',
                    localize: false,
                    assets: [
                      'src/favicon.ico',
                      'src/assets',
                      'src/manifest.webmanifest',
                      { glob: 'pdf.worker.min.mjs', input: './node_modules/pdfjs-dist/build', output: './assets/' },
                      { glob: '*.*', input: './node_modules/pdfjs-dist/cmaps', output: './assets/cmaps' },
                      { glob: '*.*', input: './node_modules/pdfjs-dist/standard_fonts', output: './assets/standard_fonts' }
                    ],
                    styles: [ 'src/styles.sass' ],
                    stylePreprocessorOptions: { includePaths: [ 'src/sass/' ] },
                    scripts: [],
                    vendorChunk: true,
                    extractLicenses: false,
                    buildOptimizer: false,
                    sourceMap: true,
                    optimization: false,
                    namedChunks: true
                  },
                  configurations: {
                    production: {
                      fileReplacements: [
                        { replace: 'src/environments/environment.ts', with: 'src/environments/environment.prod.ts' },
                        { replace: 'src/build-specifics/index.ts', with: 'src/build-specifics/index.prod.ts' }
                      ],
                      optimization: { scripts: true, styles: { minify: true, inlineCritical: false }, fonts: true },
                      outputHashing: 'all',
                      sourceMap: { hidden: true, scripts: true, styles: true },
                      namedChunks: false,
                      extractLicenses: true,
                      vendorChunk: false,
                      buildOptimizer: true,
                      budgets: [ { type: 'initial', maximumWarning: '2mb', maximumError: '5mb' }, { type: 'anyComponentStyle', maximumWarning: '6kb' } ],
                      serviceWorker: false,
                      ngswConfigPath: 'ngsw-config.json'
                    },
                    'en-GB': { outputPath: 'dist/app/en-GB/', localize: [ 'en-GB' ], i18nMissingTranslation: 'warning', baseHref: '/en-GB/' },
                    nl: { outputPath: 'dist/app/nl/', localize: [ 'nl' ], i18nMissingTranslation: 'warning', baseHref: '/nl/' }
                  },
                  defaultConfiguration: ''
                },
                serve: {
                  builder: '@angular-devkit/build-angular:dev-server',
                  options: { buildTarget: 'app:build' },
                  configurations: { production: { buildTarget: 'app:build:production' }, 'en-GB': { buildTarget: 'app:build:en-GB' }, nl: { buildTarget: 'app:build:nl' } }
                },
                'extract-i18n': { builder: '@angular-devkit/build-angular:extract-i18n', options: { buildTarget: 'app:build' } },
                test: {
                  builder: '@angular-devkit/build-angular:karma',
                  options: {
                    main: 'src/test.ts',
                    polyfills: 'src/polyfills.ts',
                    tsConfig: 'tsconfig.spec.json',
                    karmaConfig: 'karma.conf.js',
                    assets: [ 'src/favicon.ico', 'src/assets', 'src/manifest.webmanifest' ],
                    styles: [ 'src/styles.sass' ],
                    scripts: []
                  }
                },
                e2e: {
                  builder: '@angular-devkit/build-angular:protractor',
                  options: { protractorConfig: 'e2e/protractor.conf.js', devServerTarget: 'app:serve' },
                  configurations: { production: { devServerTarget: 'app:serve:production' } }
                },
                lint: { builder: '@angular-eslint/builder:lint', options: { lintFilePatterns: [ 'src/**/*.ts', 'src/**/*.html' ] } }
              }
            }
          },
          cli: { analytics: false, schematicCollections: [ '@angular-eslint/schematics', '@ngneat/spectator' ] }
        },
        main: '// This file is required by karma.conf.js and loads recursively all the .spec and framework files\r\n' +
          '\r\n' +
          "import 'zone.js/testing';\r\n" +
          "import { getTestBed } from '@angular/core/testing';\r\n" +
          'import {\r\n' +
          '  BrowserDynamicTestingModule,\r\n' +
          '  platformBrowserDynamicTesting\r\n' +
          "} from '@angular/platform-browser-dynamic/testing';\r\n" +
          "import { registerLocaleData } from '@angular/common';\r\n" +
          "import localeGB from '@angular/common/locales/en-GB';\r\n" +
          '\r\n' +
          '// First, initialize the Angular testing environment.\r\n' +
          'getTestBed().initTestEnvironment(\r\n' +
          '  BrowserDynamicTestingModule,\r\n' +
          '  platformBrowserDynamicTesting(), {\r\n' +
          '    teardown: { destroyAfterEach: false }\r\n' +
          '}\r\n' +
          ');\r\n' +
          '\r\n' +
          'registerLocaleData(localeGB);\r\n'
      }
    },
    testFramework: { version: 'jasmine@5.1.1', configurator: 'jasmine@4.6.0', reporter: 'jasmine@2.1.3', starter: 'jasmine@2.1.3', autoDetected: true },
    env: { kind: 'chrome', type: 'browser', params: {}, viewportSize: { width: 800, height: 600 }, options: { width: 800, height: 600 }, bundle: true },
    files: [
      { pattern: 'src/**/*.spec.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'src/polyfills.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'src/test.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'src/main.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'src/index.html', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'src/test.base.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'src/test.wallaby.ts', ignore: false, trigger: true, load: true, order: 1 },
      { pattern: 'src/**/*.+(ts|js)', load: false, ignore: false, trigger: true, order: 2 },
      { pattern: 'src/**/*.+(css|less|scss|sass|styl|html|json|svg)', instrument: false, load: false, ignore: false, trigger: true, order: 3 }
    ],
    tests: [ { pattern: 'src/**/*.spec.ts', load: false, ignore: false, trigger: true, test: true, order: 4 } ],
    captureConsoleLog: true,
    filesWithNoCoverageCalculated: [],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    compilers: {},
    logLimits: { inline: { depth: 5, elements: 5000 }, values: { default: { stringLength: 8192 }, autoExpand: { elements: 5000, stringLength: 8192, depth: 10 } } },
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    maxTraceSteps: 999999,
    maxTraceStepsForWatchExpressionPrefetch: 0,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    workers: { initial: 0, regular: 0, recycle: false },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore|c8 ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    mapConsoleMessagesStackTrace: false,
    extensions: {},
    reportUnhandledPromises: false,
    throwOnBeforeUnload: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    runAllTestsWhenNoAffectedTests: false,
    configCode: 'auto.detect#1313788472'
  },
  packageJSON: {
    dependencies: {
      '@ag-grid-community/angular': '~30.2.1',
      '@ag-grid-community/client-side-row-model': '~30.2.1',
      '@ag-grid-community/core': '~30.2.1',
      '@ag-grid-community/styles': '~30.2.1',
      '@ag-grid-enterprise/clipboard': '~30.2.1',
      '@ag-grid-enterprise/column-tool-panel': '~30.2.1',
      '@ag-grid-enterprise/core': '~30.2.1',
      '@ag-grid-enterprise/menu': '~30.2.1',
      '@ag-grid-enterprise/range-selection': '~30.2.1',
      '@ag-grid-enterprise/server-side-row-model': '~30.2.1',
      '@ag-grid-enterprise/status-bar': '~30.2.1',
      '@angular-eslint/schematics': '^17.3.0',
      '@angular/animations': '^17.3.0',
      '@angular/cdk': '^17.2.2',
      '@angular/common': '^17.3.0',
      '@angular/compiler': '^17.3.0',
      '@angular/core': '^17.3.0',
      '@angular/forms': '^17.3.0',
      '@angular/localize': '^17.3.0',
      '@angular/platform-browser': '^17.3.0',
      '@angular/platform-browser-dynamic': '^17.3.0',
      '@angular/router': '^17.3.0',
      '@auth0/auth0-spa-js': '^2.0.0',
      '@datadog/browser-rum': '^5.1.0',
      '@fortawesome/angular-fontawesome': '^0.14.1',
      '@ngrx/component-store': '^17.1.1',
      '@ngrx/effects': '^17.1.1',
      '@ngrx/entity': '^17.1.1',
      '@ngrx/router-store': '^17.1.1',
      '@ngrx/schematics': '^17.1.1',
      '@ngrx/store': '^17.1.1',
      '@panzoom/panzoom': '^4.5.1',
      '@popperjs/core': '^2.11.4',
      '@types/lodash': '^4.14.192',
      'chrono-node': '^2.7.5',
      'jasmine-marbles': '^0.9.2',
      'libphonenumber-js': '^1.10.15',
      litepicker: '^2.0.12',
      lodash: '^4.17.21',
      luxon: '^3.4.4',
      'pdfjs-dist': '^4.4.168',
      rxjs: '~7.8.0',
      tslib: '^2.3.0',
      uuid: '^9.0.1',
      'zone.js': '~0.14.4'
    },
    devDependencies: {
      '@angular-devkit/build-angular': '^17.3.0',
      '@angular-eslint/builder': '16.3.1',
      '@angular-eslint/eslint-plugin': '16.3.1',
      '@angular-eslint/eslint-plugin-template': '16.3.1',
      '@angular-eslint/template-parser': '16.3.1',
      '@angular/cli': '^17.3.0',
      '@angular/compiler-cli': '^17.3.0',
      '@angular/language-service': '^17.3.0',
      '@fortawesome/fontawesome-pro': '^6.1.0',
      '@fortawesome/fontawesome-svg-core': '^6.1.0',
      '@fortawesome/pro-duotone-svg-icons': '^6.1.0',
      '@fortawesome/pro-regular-svg-icons': '^6.1.0',
      '@fortawesome/pro-solid-svg-icons': '^6.1.0',
      '@ngneat/spectator': '^17.1.0',
      '@ngrx/store-devtools': '^17.1.1',
      '@types/jasmine': '~5.1.0',
      '@types/luxon': '^3.4.2',
      '@types/node': '^18.18.0',
      '@types/offscreencanvas': '^2019.7.0',
      '@types/uuid': '^8.3.4',
      '@typescript-eslint/eslint-plugin': '5.62.0',
      '@typescript-eslint/parser': '5.62.0',
      cronstrue: '^2.23.0',
      eslint: '^8.51.0',
      faker: '^4.1.0',
      'jasmine-core': '~5.1.0',
      karma: '~6.4.0',
      'karma-chrome-launcher': '~3.2.0',
      'karma-coverage': '~2.2.0',
      'karma-jasmine': '~5.1.0',
      'karma-jasmine-html-reporter': '^2.1.0',
      'karma-junit-reporter': '^2.0.1',
      'ng-mocks': '^14.12.1',
      prettier: '^1.19.1',
      'ts-node': '~8.3.0',
      tslint: '^6.1.3',
      'tslint-config-prettier': '^1.18.0',
      typescript: '~5.3.3',
      'wallaby-webpack': '^3.9.16'
    }
  },
....
}

[smcenlly]

We've reviewed our recent changes for the last month, and could not identify a possible change that would cause your project to stop working.

It looks like your diagnostic report was truncated, so without the log information, we can't troubleshoot much further. If you're concerned about sharing the rest of the report publicly for privacy reasons, you can email it to us directly: hello@wallabyjs.com.

Is it possible that you updated your dependencies and didn't reinstall them when you reverted to a previous branch?

You may also try to downgrade your Wallaby version to confirm that is the cause of your problem. You may do this by following our Offline Installation instructions and using a previous version. If you do find that a previous version is working, can you please let us know which version starts breaking for you?

[smcenlly]

@jupav - were you able to resolve your issue? We're keen to help if you can provide us with additional information.

[smcenlly]

Closing this issue since we haven't heard back. If you're still having the problem, please respond to our previous comment: https://github.com/wallabyjs/public/issues/3394#issuecomment-2327570069.