vitejs / vite-plugin-react-swc

Speed up your Vite dev server with SWC
MIT License
778 stars 50 forks source link

chore: fix flaky test #71

Closed sapphi-red closed 1 year ago

sapphi-red commented 1 year ago

playground/shadow-export/__tests__/shadow-export.spec.ts seems to be flaky.

I guess the file is edited before import.meta.hot.accept is called.

This PR is a POC of a fix.

ArnaudBarre commented 1 year ago

Thanks ! I'm using waitForLogs "connected" in other tests I don't know why I forgot to add it here

sapphi-red commented 1 year ago

This is different from "connected" log. I think this "registered" log can happen after "connected" log.

ArnaudBarre commented 1 year ago

In theory yes but fetching two files & evaluating them has always been faster than connected the socket in the last months (and the second one is also needed). Are you ok with keeping the same wait conditions than other tests?

sapphi-red commented 1 year ago

This PR didn't fail when I ran the test 10 times. But the following code failed 2 times.

import { expect, test } from "@playwright/test";
import { setupDevServer, setupWaitForLogs } from "../../utils";

test("Shadow export HMR", async ({ page }) => {
  const { testUrl, server, editFile } = await setupDevServer("shadow-export");
  const waitForLogs = await setupWaitForLogs(page);
  await page.goto(testUrl);
  await waitForLogs("[vite] connected.");

  editFile("src/App.tsx", ["Shadow export", "Shadow export updates!"]);
  await expect(page.locator("body")).toHaveText("Shadow export updates!");

  await server.close();
});

So I guess there's something.

dominikg commented 1 year ago

vite-plugin-svelte uses a helper for editing files that waits until the hmr update received message is logged. (it contains the filename so it just checks for that)

https://github.com/sveltejs/vite-plugin-svelte/blob/main/packages/e2e-tests/testUtils.ts#L167

editing files during testing, esp multiple times can be error prone due to fs inconsistencies or problems on github ci, esp. windows is notoriously slow there. Using the "polling" option for chokidar and even a retry write if no update happened after X made vite-plugin-sveltes suite very stable for hmr testing.

ArnaudBarre commented 1 year ago

@sapphi-red Thanks for the investigation. Effectively we need to wait for an assertion the app code being evaluated before starting the edits.

@dominikg Thanks for the link! For now I don't run the CI on Windows, but this will be really useful in the future.

ArnaudBarre commented 1 year ago

Added by waiting for initial text to be in the DOM. This avoid to add extra logs for testing. Green 3 times without issues