search

webpack / webpack-dev-server

Serves a webpack app. Updates the browser on changes. Documentation https://webpack.js.org/configuration/dev-server/.
MIT License
7.8k stars 1.44k forks source link

Puppeteer tests are flaky on CI #2843

Open ylemkimon opened 3 years ago

ylemkimon commented 3 years ago 
FAIL test/e2e/ClientOptions.test.js
  ● Client console.log › liveReload disabled (ws)
  ● Client console.log › liveReload & hot are disabled (ws)
  ● sockjs client port, no path › browser client › uses correct port and path
FAIL test/e2e/Socket-injection.test.js
  ● ws websocket client injection › testing when hot is enabled and liveReload is disabled › should be injected

    Failed to launch the browser process!
    Inconsistency detected by ld.so: ../elf/dl-tls.c: 481: _dl_allocate_tls_init: Assertion `listp->slotinfo[cnt].gen <= GL(dl_tls_generation)' failed!

    TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

      at onClose (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:193:20)
      at ChildProcess.<anonymous> (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:184:79)
FAIL test/e2e/TransportMode.test.js
  ● transportMode client › custom client › on browser client › logs correctly

    expect(received).toMatchSnapshot()

    Snapshot name: `transportMode client custom client on browser client logs correctly 1`

    - Snapshot  - 5
    + Received  + 1

      Array [
        "Hey.",
    -   "open",
    -   "liveReload",
    -   "[webpack-dev-server] Live Reloading enabled.",
    -   "hash",
    -   "ok",
    +   "Failed to load resource: net::ERR_CONNECTION_REFUSED",
        "close",
        "[webpack-dev-server] Disconnected!",
      ]

      82 |                       }
      83 |                     }
    > 84 |                     expect(res).toMatchSnapshot();
         |                                 ^
      85 |                     done();
      86 |                   });
      87 |                 });

      at test/e2e/TransportMode.test.js:84:33
FAIL test/e2e/TransportMode.test.js (16.054 s)
  ● transportMode client › sockjs › on browser client › logs correctly

    expect(received).toMatchSnapshot()

    Snapshot name: `transportMode client sockjs on browser client logs correctly 1`

    - Snapshot  - 1
    + Received  + 1

      Array [
        "Hey.",
    -   "[webpack-dev-server] Live Reloading enabled.",
    +   "Failed to load resource: net::ERR_CONNECTION_REFUSED",
        "[webpack-dev-server] Disconnected!",
      ]

      82 |                       }
      83 |                     }
    > 84 |                     expect(res).toMatchSnapshot();
         |                                 ^
      85 |                     done();
      86 |                   });
      87 |                 });

      at test/e2e/TransportMode.test.js:84:33
FAIL test/e2e/TransportMode.test.js (16.816 s)
  ● transportMode client › custom client › on browser client › logs correctly
  ● transportMode client › ws › on browser client › logs correctly

    Protocol error (Page.navigate): Target closed.

      58 |           runBrowser().then(({ page, browser }) => {
      59 |             const res = [];
    > 60 |             page.goto(`http://localhost:${port}/main`);
         |             ^
      61 |             page.on('console', ({ _text }) => {
      62 |               res.push(_text);
      63 |             });

      at node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
      at CDPSession.send (node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
      at navigate (node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:106:47)
      at FrameManager.navigateFrame (node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:89:13)
      at Frame.goto (node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:405:41)
      at Page.goto (node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:826:53)
      at test/e2e/TransportMode.test.js:60:13
          at runMicrotasks (<anonymous>)
alexander-akait commented 3 years ago

Any ideas?

shaodahong commented 3 years ago

Maybe we can use docker to run test, better than local puppeteer to run

hiroppy commented 3 years ago

Can we migrate to https://github.com/microsoft/playwright?

alexander-akait commented 3 years ago

Maybe it is not bad idea to try playwright

ylemkimon commented 3 years ago

Maybe we can use docker to run test, better than local puppeteer to run

I've used Selenium in other projects and had a not bad experience. Its error rate is about one in two hundred.

Can we migrate to https://github.com/microsoft/playwright?

I haven't tried playwright, but the low number of issues seems nice.

hiroppy commented 3 years ago

playwright provides chromium, firefox, and webkit and we have some browser issues such as https://github.com/webpack/webpack-dev-server/issues/2466#issuecomment-706117588. yes so it's better to try it.

alexander-akait commented 3 years ago

If somebody want to try it and help to migrate feel free to send a PR, I think it will be a good experience

snitin315 commented 2 years ago

@alexander-akait How should we do this, migrate all test at once or one by one in separate PRs then remove puppeteer in last?

alexander-akait commented 2 years ago

I think we can do it in one PR, because API is almost same, we can encapsulate some helpers functions if it will make migration easy

mahdikhashan commented 8 months ago

@snitin315 @alexander-akait hey, I'm an applicant for the gsoc-2024. I started a PR for this issue which has been label relevant to gsoc as well. could you let me know this can be considered as a patch or an early contribution which is required in the proposal?

PR: https://github.com/webpack/webpack-dev-server/pull/5080/files

mahdikhashan commented 8 months ago

@alexander-akait hey, will this issue still be a project for gsoc-2024? I could not find it listed in the document attached on the gsoc website. I had opened a PR a few weeks ago here.

alexander-akait commented 8 months ago

/cc @evenstensberg

evenstensberg commented 8 months ago

@mahdikhashan yes you could use this for a google project. We dont require you to use the ideas from the ideas list. This project is as good as any from the ideas document.

mahdikhashan commented 7 months ago

@alexander-akait would it be possible to share the invite link to Discord?

evenstensberg commented 7 months ago

https://discord.gg/44rxCZxP

mahdikhashan commented 7 months ago

joined, also shared my draft proposal both on twitter and discord.

Swapnilden commented 7 months ago

i think we should ensure Puppeteer and Chromium setup is correct, webpack-dev-server is running, handle any connection issues, and check for premature browser closures during navigation in test scripts.