Open adamdehaven opened 2 years ago
@JessicaSachs @lmiller1990 let me know if I can provide any additional info
Thanks for opening this issue. We should be doing what the Vitest plugin does to enable ESM mocking.
The Vitest code doesn't look too bad. It might be a good solution for now (to help vite-dev-server
) users. Long term, I wonder what's the best option for people who (eventually) want to write e2e tests using native ESM modules, or another dev server?
A dev-server specific mock/stub doesn't seem ideal, although happy to look into this in the short term until we better know how to handle the general problem of ESM module mock/stubs.
We will discuss internally and figure out the logistics/prioritization of this!
Can we include allowing esm stubbing functions in NextJS as well? Is that on the roadmap? @JessicaSachs
That's the only thing preventing us to use the new SWC compiler on NextJS. Currently, we're using the following .babelrc to make stubbing work.
{
"presets": ["next/babel"],
"plugins": ["@babel/plugin-transform-modules-commonjs", "babel-plugin-styled-components"]
}
👋 You want @baus or @ZachJW34 for Next-y or roadmap planning things.
Is there any update for the latest version of Cypress? When trying something similar with a composable I get the error Cannot stub non-existent own property default
rather than the ESModules cannot be stubbed
. When trying to stub out apply
or call
it seems like the function doesn't actually get replaced in the component.
@illegalnumbers as a workaround, I export the composables from another file, then stub.
// composables/index.ts
import useOrg from './useOrg'
export {
useOrg
}
// Component file (usage example)
import composables from '../composables'
const org = composables.useOrg()
// Test spec file
import composables from '../composables'
cy.stub(composables, 'useOrg').returns({ data: [] })
Sorry fam, we did not work on ESM stubbing yet. I'd really like to see this implemented, I hope we can look at it soon.
Vitest supports this but they run everything in a Node.js context. I think the first step is looking at what they do and finding out if we can do the same thing.
Is there any other runners supporting ESM stubs? Jest now has ESM support natively - I wonder how they implement it? I'm thinking the most likely rely on the Node.js module resolution, too - I don't know of any browser runners that support ESM and stubs.
If anyone wants to explore this, I can definitely help out or make some suggestions. Especially if someone can find a reference implementation, I can assist with the "here is where you need to put the code in Cypress".
I'm interested in adding browser support to esmock. it would be awesome if someone would add a cypress test folder alongside esmock's other test folders, and inside the test folder, add a passing test and a broken/failing test that uses esmock in the right place to try and browser-import a module with mock import tree.
Oh nice! Hadn't seen this project, this looks promising.
I think it's time to tackle this, going to get some resources into this in our next sprint. Not sure on the complexity or how best to solve this, but we can at least start investigating.
Hey team! Please add your planning poker estimate with Zenhub @astone123 @marktnoonan @mike-plummer @warrensplayer @ZachJW34
There is a nice repo we can test this on once delivered: https://github.com/muratkeremozcan/tour-of-heroes-react-vite-cypress-ts. Enable the lines with the comment // TODO: wait for https://github.com/cypress-io/cypress/issues/22355
Hey team! Please add your planning poker estimate with Zenhub @astone123 @marktnoonan @mike-plummer @warrensplayer @jordanpowell88
Unfortunately this is a non-trivial problem due to how ES Modules are designed. The ESM spec requires modules to have an immutable namespace, and since Cypress component tests run inside the browser and use the browser's module loader to ensure spec compliance it isn't possible to do something like this:
import MyModule from './module.js'
// `MyModule` is a sealed namespace, can't resassign/add/delete direct members
MyModule.something = 'somethingElse'
Other testing tools like Jest & Vitest work around this in a couple different ways:
Replacing the Node module loader so that modules are mutable This nicely sidesteps the issue for tools that run tests within Node, but also means your code isn't spec-compliant when it runs. This could disguise actual problems in your tests.
Mocking an entire module for the duration of a spec
Cypress APIs are designed to allow ad-hoc stubbing/spying throughout a test (see cy.stub
and cy.spy
), and we would like to maintain the capability. Replacing an entire module for the whole spec is a very different way of structuring tests and can be a bit restrictive depending on your application structure and testing use case.
Rest assured, we are working on a way to support this in a way that doesn't have a massive breaking change and doesn't change the way your code runs. In the interim, there are a couple workarounds that I'll try to outline here:
Export wrappers to sidestep module immutability As stated above, this doesn't work:
/// module.js
export function myFunc() {}
// module.spec.js
import * as MyModule from './module.js'
cy.stub(MyModule, 'myFunc')
However, this does work:
/// module.js
function myFunc() {}
export const MyModule = {
myFunc
}
// module.spec.js
import { MyModule } from './module.js'
cy.stub(MyModule, 'myFunc')
This is because the module namespace is immutable, but the module can export members that are mutable. In this case, the wrapper object MyModule
is mutable, allowing us to stub anything within it. This requires you to structure and use your modules in a particular way and it is only an option for modules under your control.
Use an importmap
If you have modules that you always want mocked out and are using a fairly modern browser you can add an importmap
to your component-index.html
to tell the browser to resolve a custom implementation anytime it is requested. Note that this will impact all uses of the module throughout your test suite, but it is an easy way to replace modules you don't want running in your tests or that you want to behave slightly different in all tests.
Note that the "mock" implementation will itself be loaded by the browser as an ES Module, so it will be subject to the same restrictions as the original. This means you can't use cy.stub
on any namespace member of the "mock" implementation, but you can write any logic you want into the mock to be shared across all of your tests.
Avoid stubs & spies Stubbing and spying can be an anti-pattern in testing depending on how they're used. You might consider whether it's possible to refactor a component to split out behavior you want to stub/spy into a separate component/hook/etc.
Short answer - we aren't sure yet. We're kicking around a few ideas, and if you know of another we'd love to hear about it.
To get the same sort of behavior as Jest and others we could introduce a new API like cy.mock('./module.js', {... })
to mock a module for the entirety of a test. This would be a bit less capable than our existing APIs and could potentially be a breaking change. There are also potential issues around how this could work with things like custom commands since the mocking would have to occur before anything accesses that module. Finally, this sort of API would really just be a wrapper around the importmap
idea above so you already have the capability to do it, just with a bit more configuration.
Another option we're considering is a plugin that rewrites ES Modules as they're served by your dev server so that they're mutable. This is a fairly complex thing to get right, and we aren't 100% sold on this being the right thing to do. One of the major principles of Cypress is that we try to be as standards-compliant as possible so we don't accidentally hide problems in your code. We'd love to hear your thoughts on this approach, and if anyone knows of a tool that already does this (or is interested in writing one) let us know!
We will have an experimental Vite plugin available this month (April 2023) that will let cy.stub
and cy.spy
work with Vite and ESM. We will share that here once it's ready to go. After some initial testing, if not blockers/problems emerge, we can make it part of the core offering. It's happening 💯
@lmiller1990 one place where this is currently a bigger blocker is when trying to stub the composables exported by vue-router
, i.e. useRoute
and useRouter
within components.
Other than wrapping the Vue Router composables as outlined in the example from the original post, is there another way to stub these within a component test?
@adamdehaven To my knowledge you have three options at the moment:
importmap
to have the browser completely replace the vue-router
library with an implementation of your own. This replacement still wouldn't be stubbable, but you could build any implementation you like into it.We are working on an approach that should allow traditional cy.stub
/cy.spy
use with ESM - we're hoping to have an initial solution together in the next couple weeks. It will be very "alpha" and likely take the form of an additional plugin to apply to Vite when running within Cypress; if you'd be interested in helping us test it out that would be awesome
if you'd be interested in helping us test it out that would be awesome
💯 yes
I would also be super interested in helping test this out - I very nearly ended up writing my own Vite plugin to add this kind of support a few months ago, but got pulled into other things and this project ended up in my backlog. Please let me know if there are ways I can help make this successful! 😄
Will keep you updated
@andrew-productiv if you had any learnings or progress or even a strategy on how you'd approach this, please share it! We are finding this quite challenging (many edge cases, integration with Sinon is tricky, etc...)
Tentative PR for this: https://github.com/cypress-io/cypress/pull/26536
See the description for how to get an early release! cc @adamdehaven. It's a plugin, which means you'll need to install it. This will be released under the @cypress
namespace soon, and once we've tested it more and ironed out all the bugs, eventually bundled in the binary and available out of the box.
Other browser-based runners that use ES modules could likely use a similar approach, too, eg in https://github.com/vitest-dev/vitest/issues/3046. The next milestones:
@lmiller1990 this is awesome 🚀 I'll def get this worked into some test flows next week
There are still some bugs we are working on fixing, please share a repro of any you run into. Latest would be @lmiller1990/vite-plugin-cypress-esm
.
This is working pretty well, a bunch of fairly complex React component tests are passing: https://github.com/muratkeremozcan/tour-of-heroes-react-vite-cypress-ts/pull/103
This (as a npm module) will be live soon under @cypress/vite-plugin-cypress-esm
(name TBA). It's still got work, but this will let us get more feedback - the more feedback and testing we can get from real projects, the sooner we can make it part of the core offering.
This is published now: https://www.npmjs.com/package/@cypress/vite-plugin-cypress-esm
I found this is not working for some basic cases: https://github.com/lmiller1990/esm-bug
I will file an issue.
@marktnoonan how would I go about using mocking ESM modules with a NextJS project?
@dwilt you could try: https://www.npmjs.com/package/@cypress/vite-plugin-cypress-esm
Are you actually using ESM (eg "type": "module"
)? Or just the import/export
syntax (depending on your config, that may be transpiled to CJS).
If you have a specific reproduction of something not working, happy to take a look.
@dwilt you could try: npmjs.com/package/@cypress/vite-plugin-cypress-esm
Are you actually using ESM (eg
"type": "module"
)? Or just theimport/export
syntax (depending on your config, that may be transpiled to CJS).If you have a specific reproduction of something not working, happy to take a look.
@lmiller1990 Trying out the plugin returns this error:
We just use the import/export syntax. We don't have "type": "module"
set in our package.json since we get this error:
core:test:cy: (node:24174) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
core:test:cy: (Use `node --trace-warnings ...` to show where the warning was created)
core:test:cy: (node:24174) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
core:test:cy: /Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11
core:test:cy: super({});
core:test:cy: ^
core:test:cy: TypeError: Class constructor OTLPTraceExporter cannot be invoked without 'new'
core:test:cy: at new OTLPTraceExporter (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11:9)
core:test:cy: at Object.<anonymous> (/Users/user/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/require_async_child.js:9:18)
core:test:cy: at Module._compile (node:internal/modules/cjs/loader:1254:14)
core:test:cy: at Module.m._compile (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:857:29)
core:test:cy: at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
core:test:cy: at Object.require.extensions.<computed> [as .js] (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:859:16)
core:test:cy: at Module.load (node:internal/modules/cjs/loader:1117:32)
core:test:cy: at Function.Module._load (node:internal/modules/cjs/loader:958:12)
core:test:cy: at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
core:test:cy: at ModuleJob.run (node:internal/modules/esm/module_job:194:25)```
Will need a minimal reproduction to give any useful advice!
@dwilt you could try: npmjs.com/package/@cypress/vite-plugin-cypress-esm Are you actually using ESM (eg
"type": "module"
)? Or just theimport/export
syntax (depending on your config, that may be transpiled to CJS). If you have a specific reproduction of something not working, happy to take a look.@lmiller1990 Trying out the plugin returns this error:
We just use the import/export syntax. We don't have
"type": "module"
set in our package.json since we get this error:core:test:cy: (node:24174) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time core:test:cy: (Use `node --trace-warnings ...` to show where the warning was created) core:test:cy: (node:24174) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time. core:test:cy: /Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11 core:test:cy: super({}); core:test:cy: ^ core:test:cy: TypeError: Class constructor OTLPTraceExporter cannot be invoked without 'new' core:test:cy: at new OTLPTraceExporter (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11:9) core:test:cy: at Object.<anonymous> (/Users/user/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/require_async_child.js:9:18) core:test:cy: at Module._compile (node:internal/modules/cjs/loader:1254:14) core:test:cy: at Module.m._compile (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:857:29) core:test:cy: at Module._extensions..js (node:internal/modules/cjs/loader:1308:10) core:test:cy: at Object.require.extensions.<computed> [as .js] (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:859:16) core:test:cy: at Module.load (node:internal/modules/cjs/loader:1117:32) core:test:cy: at Function.Module._load (node:internal/modules/cjs/loader:958:12) core:test:cy: at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29) core:test:cy: at ModuleJob.run (node:internal/modules/esm/module_job:194:25)```
@andyhqtran I ran into this the other day and created an issue with a reproduction https://github.com/cypress-io/cypress/issues/28696
Hi @lmiller1990, we've been struggling to make the @cypress/vite-plugin-cypress-esm plugin work without success for component tests in a Vue 3 + Typescript project.
An easy way to reproduce is to just scaffold the example Vue 3 app with: npm create vue@latest
, and then install the plugin.
The error we are getting is this one:
We tried to add Vue to the ignoreModuleList as:
CypressEsm({
ignoreModuleList: ['*vue*']
})
But the problem persists.
Any ideas on how to approach it?
Hmm can you try ignoreImportList
(just a guess) https://github.com/cypress-io/cypress/tree/develop/npm/vite-plugin-cypress-esm#ignoreimportlist
I am guessing you cannot stub something defined at runtime, which could be expose
.
After a lot of time/effort from many people in many projects, there is still no solution to stub/mock for ESM modules. They are sealed and immutable at the implementation level, I can't really imagine if this will ever be fully solvable.
I am not entirely sure what is the actual issue here. It looks like on line 8 of your code it does:
setup(props, {expose: __expose)`
And for some reason, the object it is trying to destructure is null. I can't think why that would be or how it's caused by this plugin.
Please try the ignoreImportList
and let me know what happens. If no luck, I will think / try something out and see if I can make a recommendation.
Hi @lmiller1990 same problem with ignoreImportList
.
Keep in mind that the setup(props, {expose: __expose)
is something Vue 3 automatically generates for any component, so this problem will happen to anybody using Vue 3.
The idea of ignoreImportList
was partly to let modules opt out of it. It looks like it is not getting applied:
Not sure what's next... do you have a minimal example repo so someone can clone and try debug?
Hey @lmiller1990 I just created one for you: https://github.com/mverdaguer/cypress-vue3-esm-stubs-error/tree/main
It's great to see the effort on such an important topic. I think once there is an option to mock modules on real environments all my wishes come true. Unfortunately using the plugin on our React-Typescript-vite setup fails.
[plugin:vite:import-analysis] Failed to parse source for import analysis because the content contains invalid JS syntax. If you are using JSX, make sure to name the file with the .jsx or .tsx extension.
/Users/user/code/predium/node_modules/.vite/predium-client/deps/chunk-INT4XFS7.js:33:44
31 | unstable_ClassNameGenerator
32 | };
33 | //# sourceMappingURL=chunk-INT4XFS7.js.map
| ^
34 |
at formatError (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:44062:46)
at TransformContext.error (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:44058:19)
at TransformContext.transform (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:41784:22)
at async Object.transform (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:44352:30)
at async loadAndTransform (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:55026:29)
at async viteTransformMiddleware (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:64430:32
and a readable error like:
> Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/@fs/Users/christophgriehl/code/predium/apps/client/src/pages/DataCollection/DataCollectionEnergyCertificateDraft.cy.tsx
I have the plugin configured with ignoreModuleList
like so:
CypressEsm({ ignoreModuleList: ['*react*'], }),
The plugin seems to fail on some basic step of our project even though it is quite common and nothing unorthodox happens in our build process therefore it is difficult to attempt a repro case.
Is there anything I can do? What is the state of the plugin currently. Does someone has a working example except the PR where it got merged?
@Crismon96 The plugin is an alpha-stage experiment so instability and errors are to be expected, but I can say for certain that it is being used successfully on several very large complex projects. The plugin relies on fairly basic methods to transform code (regexes rather than an AST) for now, so it's very possible a particular string or magic variable name in a unique third-party dependency or newer babel transform is tripping things up.
Without a reproduction case I can't provide much direct assistance, but I would recommend the following flow:
ignoreModuleList
and ignoreImportList
config flags to exclude that resource from being transformed by the plugin.If you are able to narrow it down and get something working hopefully that may give you enough info to report back and open an issue with a reproduction case. Good luck!
Just adding for future reference, someone pointed out you could experiment with using MSW to intercept a module and return mocked one. From: https://github.com/vitest-dev/vitest/issues/3046#issuecomment-2152511007
I was interested in this issue a while back and thought I'd play around with MSW after @lmiller1990 pointed out the viability of using MSW. I was able to get something to work (with some quirks...) if anyone is interested in checking it out cypress-vite-esm-msw-mocker and giving it a go.
The package allows you to intercept network requests and replace a modules contents with another file on disk or just a string. Works with esm modules and default exports (since the entire file gets replaced). The package is rough around the edges but could be useful!
Is there a solution for webpack for the same issue?
@matthias-deschoenmacker The solution most people have used is modifying Webpack to not generate ESM modules when bundling for your Component Tests. There is a config option that allows you to customize the Webpack config applied to your project. That would cover situations where your code is bundled as ESM - if you're somehow pulling in third-party libs that are prebundled as ESM then you have a more complex problem that is a bit outside of Cypress - you'd either need to stub a higer-level wrapper that is under your control, or restructure the import to be dynamic and use something like msw
to intercept and replace like suggested above. Discord would be the right place to ask for help if you run into issues trying to implement any of these.
Is the consensus here that vitest will not support this then?
Outside of the plugin (with works for a good amount of cases, did you try it?) and various other suggestions that are suggested here there is no ongoing work to enable ESM mocking/stubbing in Cypress / Vite integration.
Does Vitest (eg, the Node.js test runner powered by Vite: https://vitest.dev/) even support this? My understanding is the ESM is designed specifically to not be mutable - the core issue is we are all trying to do something the spec specifically disallows (tinkering with modules).
What would you like?
Utilizing Vite + Vue and Cypress Component Test Runner, how would you stub a composable function since you can't stub the default export?
I can't find a decent example that doesn't utilize Babel, and the only solution we have come up with is exporting an object with methods that can be stubbed, which, to be honest, would be a large refactor.
When stubbing the default export, as shown below, an error is thrown:
ESModules cannot be stubbed
. I know this is a valid error (here's a great write-up for reference); however, we need the ability to stub the default exports of imported modules.Why is this needed?
It's a standard in Vue 3 to move stateful logic into composable functions. Composables typically export a default function (not an object) and devs need the ability to stub the outputs of a composable.
There is currently a workaround; however, it would require refactoring large blocks of code within our (and most) application whereby the composable exports a utility function that is used to set the returned values. This workaround (shown below) is a bit cumbersome and requires a very explicit, non-standard way of writing composable functions.
Here is the same composable provided above, rewritten for the workaround (not ideal):
Other
No response