alphagov / govuk-design-system

One place for service teams to find styles, components and patterns for designing government services.
https://www.gov.uk/design-system
MIT License
499 stars 231 forks source link

Fix intermittently failing test #859

Closed 36degrees closed 5 years ago

36degrees commented 5 years ago

The tests are failing intermittently on CI, with the error Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

Example build output:

$ npm test -- --runInBand
> govuk_design_system@ test /home/travis/build/alphagov/govuk-design-system
> npm run lint && jest "--runInBand"
> govuk_design_system@ lint /home/travis/build/alphagov/govuk-design-system
> npm run lint:js && npm run lint:scss
> govuk_design_system@ lint:js /home/travis/build/alphagov/govuk-design-system
> standard
> govuk_design_system@ lint:scss /home/travis/build/alphagov/govuk-design-system
> sass-lint '**/*.scss' --verbose --ignore 'node_modules/**/*.scss'
 PASS  __tests__/search.test.js
 PASS  lib/get-macro-options/__tests__/index.js
 PASS  __tests__/tabs.test.js
 PASS  lib/metalsmith-lunr-index/index.test.js
 FAIL  __tests__/accessiblity_audit.test.js (7.941s)
  ● Accessibility Audit › Component page - layout-pane.njk › validates
    Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.
      38 | 
      39 |   describe('Component page - layout-pane.njk', () => {
    > 40 |     it('validates', async () => {
         |     ^
      41 |       await page.goto(baseUrl + '/components/radios/', { waitUntil: 'load' })
      42 |       const results =
      43 |         await new AxePuppeteer(page)
      at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
      at Suite.it (__tests__/accessiblity_audit.test.js:40:5)
  ● Accessibility Audit › Patterns page - layout-pane.njk › validates
    Protocol error (Runtime.callFunctionOn): Execution context was destroyed.
      at Promise (node_modules/puppeteer/lib/Connection.js:183:56)
      at CDPSession.send (node_modules/puppeteer/lib/Connection.js:182:12)
      at ExecutionContext.evaluateHandle (node_modules/puppeteer/lib/ExecutionContext.js:106:44)
      at ExecutionContext.<anonymous> (node_modules/puppeteer/lib/helper.js:111:23)
      at ExecutionContext.evaluate (node_modules/puppeteer/lib/ExecutionContext.js:48:31)
      at ExecutionContext.<anonymous> (node_modules/puppeteer/lib/helper.js:111:23)
      at DOMWorld.evaluate (node_modules/puppeteer/lib/DOMWorld.js:105:20)
        -- ASYNC --
      at Frame.<anonymous> (node_modules/puppeteer/lib/helper.js:110:27)
      at AxePuppeteer.<anonymous> (node_modules/axe-puppeteer/src/axePuppeteer.ts:227:43)
      at fulfilled (node_modules/axe-puppeteer/dist/axePuppeteer.js:4:58)
 PASS  __tests__/component-options.test.js
 PASS  __tests__/mobile-navigation.test.js
 PASS  __tests__/back-to-top.test.js
 PASS  __tests__/example.test.js
 PASS  lib/extract-page-headings/index.test.js
Test Suites: 1 failed, 9 passed, 10 total
Tests:       2 failed, 61 passed, 63 total
Snapshots:   0 total
Time:        18.923s
Ran all test suites.
npm ERR! Test failed.  See above for more details.
The command "npm test -- --runInBand" exited with 1.
36degrees commented 5 years ago

I believe (touch wood) that this was addressed in #862.