testing-library / svelte-testing-library

:chipmunk: Simple and complete Svelte DOM testing utilities that encourage good testing practices
https://testing-library.com/docs/svelte-testing-library/intro
MIT License
620 stars 33 forks source link

Svelte 5 feedback and support #284

Open stefanhoelzl opened 11 months ago

stefanhoelzl commented 11 months ago

Hello, @mcous has hijacked this post! We've launched ~experimental~ Svelte 5 support in @testing-library/svelte. ~See the Svelte 5 section in the README for setup instructions.~ No special setup is required to use Svelte 5 if you are using Vitest.

If you find bugs or have other Svelte 5 testing feedback, please add it to this thread! Svelte 5 is still changing rapidly at the time of writing, so feedback is appreciated!

Original post below


Hi! šŸ‘‹

Firstly, thanks for your work on this project! šŸ™‚

Today I used patch-package to patch @testing-library/svelte@4.0.5 for the project I'm working on to use svelte 5.

This patches some breaking changes but there might be more work to do for a proper PR (are tests still working? support svelte 4 and 5, maybe more)

But for everyone who likes to try out svelte 5 this patch should be a good starting point.

Here is the diff that solved my problem:

diff --git a/node_modules/@testing-library/svelte/src/pure.js b/node_modules/@testing-library/svelte/src/pure.js
index 04d3cb0..2f041e0 100644
--- a/node_modules/@testing-library/svelte/src/pure.js
+++ b/node_modules/@testing-library/svelte/src/pure.js
@@ -3,7 +3,7 @@ import {
   getQueriesForElement,
   prettyDOM
 } from '@testing-library/dom'
-import { tick } from 'svelte'
+import { tick, createRoot } from 'svelte'

 const containerCache = new Set()
 const componentCache = new Set()
@@ -54,18 +54,15 @@ const render = (
     return { props: options }
   }

-  let component = new ComponentConstructor({
+  let component = createRoot(ComponentConstructor, {
     target,
-    ...checkProps(options)
+    ...checkProps(options),
+    ondestroy: () => componentCache.delete(component)
   })

   containerCache.add({ container, target, component })
   componentCache.add(component)

-  component.$$.on_destroy.push(() => {
-    componentCache.delete(component)
-  })
-
   return {
     container,
     component,
@@ -73,18 +70,14 @@ const render = (
     rerender: (options) => {
       if (componentCache.has(component)) component.$destroy()

-      // eslint-disable-next-line no-new
-      component = new ComponentConstructor({
+      component = createRoot(ComponentConstructor, {
         target,
-        ...checkProps(options)
+        ...checkProps(options),
+        ondestroy: () => componentCache.delete(component)
       })

       containerCache.add({ container, target, component })
       componentCache.add(component)
-
-      component.$$.on_destroy.push(() => {
-        componentCache.delete(component)
-      })
     },
     unmount: () => {
       if (componentCache.has(component)) component.$destroy()

This issue body was partially generated by patch-package.

jsorb84 commented 11 months ago

Thank you for this, this works flawlessly, you should def. make a pull request , cheers my friend.

jakebottrall commented 11 months ago

Thanks @stefanhoelzl, this is working a treat! Any ideas on how to test snippets? So far I haven't managed to figure out how to provide a snippet to the props API. So it's back to the old slots hack of making a wrapping component for tests.

sureshjoshi commented 9 months ago

Anyone have any luck with snippets? The wrapper components are boilerplate-laden - was hoping to not have to do that.

If anyone has any pointers, I'm happy to give it a shot myself.

sureshjoshi commented 9 months ago

EDIT: Ignore this part - after a LOT of debugging, this seems to be a problem when using happy-dom instead of jsdom - which is insane, since I use happy-dom in all of my other projects... But that's a problem for another day :)

~Also, @yanick - for this patch, does screen.debug() work anymore? I can't seem to even get the most basic Svelte component to work due to:~

PrettyFormatPluginError: Cannot read properties of null (reading 'toLowerCase')

~And that's even with a component which is just <div></div>~

mcous commented 9 months ago

I've been messing around with Svelte 5 in our existing test suite, and have encountered the following issues so far. I didn't see any of these explicitly called out in the change log, but I'm assuming at the moment that they're all intentional.

sysmat commented 9 months ago
Instantiating a component with `new` is no longer valid in Svelte 5
mcous commented 9 months ago

Rudimentary support for Svelte 5 has been released on the next tag

npm install ā€”-save-dev @testing-library/svelte@next

Since Svelte 5 itself is not ready, expect issues and breaking changes if you try it out!

sysmat commented 9 months ago

@mcous thx, so @testing-library/svelte@next is "@testing-library/svelte": "^4.2.0",

with this version it works

mcous commented 9 months ago

FYI for those following along: as of the latest svelte prerelease, our own Svelte 5 support will no longer work due to https://github.com/sveltejs/svelte/pull/10516, which removed the API we were using to mount components. Will do our best to adjust accordingly

yanick commented 9 months ago

as of the latest svelte prerelease, our own Svelte 5 support will no longer work due to https://github.com/sveltejs/svelte/pull/10516

:darth vader big NOOO:

sureshjoshi commented 9 months ago

@mcous If you have any suggestions for directed ways to help, I'm all ears :) Though, less familiar with the internals of the lib.

Is the issue that they removed the API call we use? And that it needs to be migrated to the new one? Or is the new mount API a no-go?

pboling commented 7 months ago

Anyone know of any updates?

mcous commented 7 months ago

Experimental Svelte v5 support is available on the next dist tag

npm install --save-dev @testing-library/svelte@next

In your test files:

import { render } from '@testing-library/svelte/svelte5'

@yanick how would you feel about bumping the next branch to 5 (to account for breaking changes semantic-release missed) and merging into main? Are there any more breaks - e.g. dropping svelte@3 - that you'd like to collect on next first?

The svelte@5 support is appropriately marked as experimental and it seems like something we can promote in a minor release whenever svelte@5 goes into production

pboling commented 7 months ago

Your previous update noted that Svelte 5 support did not work at all. It isn't clear from this update if it has been fixed.

yanick commented 7 months ago

Fwiw, I'm kinda busy today, but I'll try to turn my attention to deployment tomorrow.

mcous commented 7 months ago

Your previous update noted that Svelte 5 support did not work at all. It isn't clear from this update if it has been fixed.

It has been fixed for now, it may be broken tomorrow. With the very-in-progress nature of Svelte v5, the best way to find out if it's currently working will be to try it yourself

sureshjoshi commented 7 months ago

When I tried testing-lib out a week or so ago, it was working - but also note that there are still bugs depending on whether you're using JSDom vs Happy-Dom, or from svelte itself, or any related libraries.

I've been working on a Svelte 5 headless UI port, and Svelte5 + interop is a bit messy at the moment (to say the least) :)

mcous commented 7 months ago

Experimental Svelte 5 support has landed in main! We'll do our best to stay on top of things, but here there be dragons šŸ‰

npm install --save-dev @testing-library/svelte@latest now includes Svelte v5 support

MatthewTFarley commented 7 months ago

When using svelteTesting Vitest setup function with Svelte 5, the autoCleanup option does not work because the function imports the Svelte 4 cleanup function by way of vitest.js.

I have also run into the Element.animate issue @mcous identified. Here's an example repository demonstrating it. It also has some comments mentioning the cleanup issue in the vite.config.ts file. Using vitest globals: true is a workaround since it will cause the correct cleanup to run here.

mcous commented 7 months ago

When using svelteTesting Vitest setup function with Svelte 5, the `autoCleanup option does not work because the function imports the Svelte 4 cleanup function by way of vitest.js.

...Using vitest globals: true is a workaround...

@MatthewTFarley auto-cleanup via plugin works if you use the suggested Svelte 5 alias setup from the README! Otherwise, globals mode is probably your best bet. This awkwardness will go away once Svelte 5 is out of prerelease and the svelte5 import is no longer needed.

export default defineConfig({
  plugins: [svelte(), svelteTesting()],
  test: {
    alias: {
      '@testing-library/svelte': '@testing-library/svelte/svelte5',
    },
  },
})

I have also run into the Element.animate issue

I wonder what's going to happen here! The animations work in the browser, but neither jsdom nor happy-dom support Element.animate. I don't know if Svelte's intention is to require Element.animate or include some sort of fallback.

We may have to start thinking about shipping our own Element.animate stub, but faking out browser APIs in this library gets into some iffy-mocking-practices territory

pboling commented 6 months ago

@mcous I don't know enough to know if what I have should be working, or is expected to be broken. I am on latest versions as of this second of all packages, and latest "next" versions where possible (e.g. Svelte 5 next 118). I have some basic tests working, following the patterns of the internal tests of this actual library (testing-library/svelte-testing-library). I'm trying to ensure my config remains working as Svelte5 nears release, and as I add more complex tests to my test suite, so I am following this issue.

When I use the alias mentioned above and in the readme, my test suite doesn't run:

 FAIL  src/lib/components/forms/input.test.ts [ src/lib/components/forms/input.test.ts ]
Error: Missing "./svelte5/vitest" specifier in "@testing-library/svelte" package
 āÆ e node_modules/.pnpm/vite@5.2.10_@types+node@20.12.7/node_modules/vite/dist/node/chunks/dep-DkOS1hkm.js:47596:25

My config (after adding the alias):

import { sveltekit } from '@sveltejs/kit/vite';
import { defineConfig } from 'vitest/config';
import { paraglide } from '@inlang/paraglide-sveltekit/vite';
import { svelteTesting } from '@testing-library/svelte/vite';
export default defineConfig(({ mode }) => ({
    plugins: [
        paraglide({
            // recommended
            project: './project.inlang',
            outdir: './src/lib/paraglide',
        }),
        sveltekit(),
        svelteTesting(),
    ],
    resolve: {
        conditions: mode === 'test' ? ['browser'] : [],
    },
    test: {
        include: ['src/**/*.{test,spec}.{js,ts}'],
        environment: 'jsdom',
        setupFiles: ['./vitest-setup.ts'],
        alias: {
            '@testing-library/svelte': '@testing-library/svelte/svelte5',
        },
    },
}));

When I remove the alias property from test I am able to run my test suite. Hopefully this is helpful feedback!

mcous commented 6 months ago

@pboling ~Looks like maybe our alias needs to be more specific; it's replacing imports it shouldn't. I'll experiment with this a little and update the README shortly.~ Upon further inspection, I think the problem might lie with a couple updates you should make to your config now that you're using the new svelteTesting plugin.

Both of these jobs the plugin now handles for you. I find that this configuration - as listed in the README - behaves well with Svelte 5:

// vite.config.ts
// ...
export default defineConfig({
  plugins: [sveltekit(), svelteTesting()],
  test: {
    include: ['src/**/*.{test,spec}.{js,ts}'],
    environment: "jsdom",
    alias: {
      "@testing-library/svelte": "@testing-library/svelte/svelte5",
    },
    setupFiles: ["./vitest-setup.ts"],
  },
});
// vitest-setup.ts
// Only needed to configure `@testing-library/jest-dom`
import '@testing-library/jest-dom/vitest`

If this doesn't work, you could try using a more specific alias. In my own projects, this is necessary only if I'm not using the svelteTesting plugin.

{
  // ...
  test: {
    // ...
    alias: [
      {
        find: /^@testing-library\/svelte$/,
        replacement: '@testing-library/svelte/svelte5',
      },
    ],
  // ...
  },
}

(All this being said, something feels off to me here. I think having a separate Svelte 5 import path is probably causing more trouble than it's worth. I don't think the situation will stick around too much longer, though)

pboling commented 6 months ago

@mcous The top suggestions you gave are working! I did not need the more specific alternative alias.

webJose commented 5 months ago

I have a problem testing prop updates. I may be doing it wrong, though.

I am using rerender() to pass updated prop values. Is this the correct way? If it is, then the problem I have is that it seems to cause the component to unmount, which is not expected.

If rerender() is not the right replacement for <component>.$set(), please let me know.

mcous commented 5 months ago

[rerender] seems to cause the component to unmount, which is not expected.

@webJose this was a previous bug with rerender that was fixed in February of this year - see #261. Are you on the latest version of @testing-library/svelte? I'm seeing our own rerender tests passing correctly in CI

https://github.com/testing-library/svelte-testing-library/blob/326fb58aee4648523a88d94a60926e551dc56270/src/__tests__/rerender.test.js#L8-L15

webJose commented 5 months ago

Hello, yes I'm on the latest version. It was actually my bad. It was an error post-test. To be clear, the unmounting was taking place after the test had finished.

But then I have a different problem. I do await component.rerender(...), but the associated $effect() doesn't seem to be running, and the component seems to be working fine. So I wonder if three could be a different problem. I suppose that $effect()'s are working fine in this library?

mcous commented 5 months ago

@webJose do you have a minimal reproduction of this issue? I just tried out a simple $effect test locally and effects seem to be running correctly

test('$effect', async () => {
  const onEffect = vi.fn()
  const { rerender } = render(Comp, { name: 'World', onEffect })

  await rerender({ name: 'Planet' })

  expect(onEffect).toHaveBeenCalledWith('Planet')
})
<script>
  let { name = 'World', onEffect } = $props()

  $effect(() => {
    onEffect(name)
  })
</script>
webJose commented 5 months ago

@mcous Thank you very much for the time you spent trying to help me.

My problem was related to a deeper async tree. Merely awaiting rerender(), for my specific case, was not enough for everything that I was expecting to happen, happen.

I have now obtained the expected test result by adding an await delay(0); to place the rest of the test's code at the end of the NodeJS message pump. For other people's reference, this is the delay() function:

export function delay(ms: number) {
    let rslv: () => void;
    const promise = new Promise<void>((rs, rj) => {
        rslv = rs;
    });
    setTimeout(() => {
        rslv();
    }, ms);
    return promise;
};
Hugos68 commented 5 months ago

Hey, I am building a library with Svelte related utilities and wanted to know wether it's possible to test against SSR output. Currently the render function renders the component as it would on the client. But I would really appreiciated if you could do:

render(App, ..., { ssr: true });

Has this been thought of?

Svelte exposes a render function from svelte/server which can be used I think.

mcous commented 5 months ago

Hi @Hugos68, I think that request would be better suited for its own ticket. But short version: no, to my knowledge this hasn't been something on our radar. This library has always been focused on wrapping Svelte's client-side API to render components into a DOM to interact with and run queries against

mcous commented 5 months ago

I've started work on moving our Svelte 5 support into the non-experimental phase. I've just published @testing-library/svelte@5.2.0-next.1 on our next tag that incorporates Svelte 5 support into the main export; no alias to @testing-library/svelte/svelte5 required

I'll be testing these next versions out before promoting them to our latest tag, and I'd appreciate any help and feedback I could get from anyone else dabbling in Svelte 5!

mcous commented 5 months ago

Followup on my above post: after a few prerelease versions to sort out types and peer dependencies, Svelte 5 support has landed in version 5.2.0 of @testing-library/svelte on our latest tag. Special setup for Svelte 5 should no longer be needed (if you use Vitest)

I'll keep this thread going while Svelte 5 is still itself a prerelease, and as always, please reach out if you run into any issues!

eunukasiko commented 4 months ago

Is $bindable supported?

This test fails with component.value being undefined

    const user = userEvent.setup();
    const { getByRole, component } = render(Textarea);

    await user.type(getByRole('textbox'), 'bar');

    await waitFor(() => expect(component.value).toBe('bar'));

Textarea has value = $bindable(undefined)

mcous commented 4 months ago

@eunukasiko what does you component look like? I don't think Svelte 5 allows accessing prop values directly on the component instance like that.

You likely will have to use a fixture component and a state rune to test binding, as in the Svelte 4 example, which uses a store

One developer to another: consider avoiding bindable props entirely. Using a prop and a callback function to clearly mark data down and data up, respectively, results in more testable and maintainable code in the long run

webJose commented 4 months ago

Hello. I have a problem: It seems that with v5.2.0, it is an all or nothing with Svelte 5. Or am I doing something wrong? My scenario is a Svelte v4 component library that is in the middle of the process of being converted to Svelte v5. As such, there are both v4 and v5 components.

The error I get:

 FAIL  src/lib/Alert/Alert.test.ts > Alert > Should render the contents of the default slot.
Svelte error: lifecycle_function_unavailable
`mount(...)` is not available on the server

Only very few tests are passing. For example, this one passes:

    test('Should apply any additional classes passed through the class property.'), () => {
        // Arrange.
        const cssClass = 'my-custom-class';

        // Act.
        render(Alert, { class: cssClass });

        // Assert.
        const alert = screen.getByRole('alert');
        expect(alert).toBeTruthy();
        expect(alert.classList.contains(cssClass)).toEqual(true);
    };

But this one doesn't:

    test.each(allBootstrapVariants)
        ('Should render an alert of the %s variant.', (variant) => {
            // Act.
            render(Alert, { variant });

            // Assert.
            const alert = screen.getByRole('alert');
            expect(alert).toBeTruthy();
            expect(alert.classList.contains(`alert-${variant}`)).toEqual(true);
        });
mcous commented 4 months ago

@webJose that looks like an SSR vs browser import setup bug. What does your Vite / Vitest setup file look like? Does it match the recommended setup in the readme or setup docs?

The Vite plugin this library exports ensures Svelte's browser code is used during tests. Without the plugin, Svelte's SSR code will be used, instead, which won't work properly

webJose commented 4 months ago

Nooo! There's a Vite plug-in? So in Svelte v4 I've been testing SSR? I'll see about this tomorrow.

mcous commented 4 months ago

So in Svelte v4 I've been testing SSR?

Maybe! It was a really long-standing setup bug: #222. We introduced a Vite plugin to make the setup less error prone.

In Svete 4, the SSR code gets used silently. In Svelte 5, it seems to fail, instead. I hope this is your issue, because the fix is easy! If not, hopefully we can figure out whatever else is going on

lachlancollins commented 4 months ago

Hi there, I've created a separate issue #394 because I didn't want to clutter this feed too much, I would appreciate it if anyone following along could have a look!

webJose commented 3 months ago

How does one test the effect of bind:?

mcous commented 3 months ago

How does one test the effect of bind:?

@webJose you'll probably need a wrapper component of some sort, similar to the Svelte 4 example. See my comment above.

Alternatively, avoid bind and use a callback prop to signal changes. In my experience, unidirectional data flow is a more reliable way to build applications

tony-stark-eth commented 2 months ago

Works great with vite template I've build, thanks!

webJose commented 1 month ago

Recently, the following started to happen in VS Code, in calls to render():

Argument of type 'Component<$$ComponentProps, {}, "">' is not assignable to parameter of type 'ComponentType<SvelteComponent<Record<string, any>, any, any>>'.
  Type 'Component<$$ComponentProps, {}, "">' is not assignable to type 'new (options: ComponentConstructorOptions<Record<string, any>>) => SvelteComponent<Record<string, any>, any, any>'.
    Type 'Component<$$ComponentProps, {}, "">' provides no match for the signature 'new (options: ComponentConstructorOptions<Record<string, any>>): SvelteComponent<Record<string, any>, any, any>'.ts(2345)

It seems that Svelte may have change the typings.

arekrgw commented 1 month ago

Recently, the following started to happen in VS Code, in calls to render():

Argument of type 'Component<$$ComponentProps, {}, "">' is not assignable to parameter of type 'ComponentType<SvelteComponent<Record<string, any>, any, any>>'.
  Type 'Component<$$ComponentProps, {}, "">' is not assignable to type 'new (options: ComponentConstructorOptions<Record<string, any>>) => SvelteComponent<Record<string, any>, any, any>'.
    Type 'Component<$$ComponentProps, {}, "">' provides no match for the signature 'new (options: ComponentConstructorOptions<Record<string, any>>): SvelteComponent<Record<string, any>, any, any>'.ts(2345)

It seems that Svelte may have change the typings.

Got the same thing, wildly appeared after installing one of new svelte@5.0.0-next.259 or 258

mcous commented 1 month ago

@webJose @arekrgw I just pulled down the latest svelte@next and I'm not able to reproduce this issue. If either of you have a second, these answers could help me track it down if we've got a new typing issue to resolve šŸ™

arekrgw commented 1 month ago

@mcous Got the svelte@5.0.0-next.260 now but issue still persist, but only in VSCode, svelte-check does not show any errors. Svelte extension is the latest v109.0.3 from 2 days ago.

Might be worth mentioning that last Friday 2024-09-27 the extension freaked out and showed some stranger errors with generics usage, but it is fixed now, probably new version fixed it.

I guess that's the cost of using prerelease software šŸ¤£

webJose commented 1 month ago

@mcous sure thing: https://github.com/WJSoftware/wjfe-single-spa-svelte/blob/15d2c600ea4f7077f498a76ab958ef909421096f/src/lib/SspaParcel/SspaParcel.test.ts#L21

This // @ts-expect-error line had to be added because of the typing error. Feel free to clone the repo, remove the line (there are other lines in the various calls to render()) and let me know how it goes for you.

My Svelte for VS Code extension info: image

npm run check doesn't detect any errors, but my guess is that it doesn't check .test files.

mcous commented 1 month ago

@webJose thank you, that was exactly what I needed! I'm reproducing the issue in the repo here, too, now, and will get working on a fix

webJose commented 1 month ago

@mcous look at this release of svelte-check. The first bullet point.

mcous commented 1 month ago

Just released ~v5.2.2~ v5.2.3 (please ignore v5.2.2, it had a packaging issue), which should resolve the typing issues when using Svelte 5 and runes. Let me know if any more issues pop up