feat(cypress): support Cypress 10

hinogi commented 2 years ago

What kind of change does this PR introduce? (check at least one)

[ ] Bugfix
[ ] New test runner
[ ] Documentation
[ ] Feature
[ ] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

If you are adding a new test runner, have you...? (check all)

[ ] Created an issue first?
[ ] Registered it in /packages/base/runners.json?
[ ] Added it to /README.md?
[ ] Included one test that runs baseline.spec.vue?
[ ] Added and updated documentation?
[ ] Included a recipe folder with properly building quasar project?

Does this PR introduce a breaking change? (check one)

[ ] Yes
[ ] No

If yes, please describe the impact and migration path for existing applications:

The PR fulfills these requirements:

[ ] It's submitted to the dev branch and not the master branch
[ ] When resolving a specific issue, it's referenced in the PR's title (e.g. fix: #xxx[,#xxx], where "xxx" is the issue number)
[ ] It's been tested on Windows
[ ] It's been tested on Linux
[ ] It's been tested on MacOS
[ ] Any necessary documentation has been added or updated in the docs (for faster update click on "Suggest an edit on GitHub" at bottom of page) or explained in the PR's description.

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

Closes https://github.com/quasarframework/quasar-testing/issues/255

IlCallo commented 2 years ago

Hey there, what's missing on this one to promote it to ready for review? Can you list here any blockers?

hinogi commented 2 years ago

Hey there, what's missing on this one to promote it to ready for review? Can you list here any blockers?

The Dialog would be one. Vite module resolution via path alias is the major one.

Evertvdw commented 2 years ago

I experimented with Vitest and Cypress v10 component testing in one repo. I have it working with a shared vite-config.ts for both testing utilities. Here is an example repo, maybe that helps with the AE: https://github.com/Evertvdw/temp-quasar-test-test

IlCallo commented 2 years ago

No wonders the setup doesn't work, you haven't updated the dependencies accordingly 😅 I'm taking a closer look to the PR, hope I'll be able to make it work before this evening

176ce27 (#259)

It may also be that the existing dependencies are interfering with the new ones, as if I got it right dev-server packages are now exposed by the main Cypress binary

khteh commented 2 years ago

+1

sudocovik commented 2 years ago

I've manually added the Cypress 10 in my project so I assume you might find useful problems I encountered and solved.

First, I tried cloning the hinogi:feature/cypress-10 and then point the @quasar/quasar-app-extension-testing-e2e-cypress to the appropriate folder but the app-installer always used template files from previous version. Anyway, I just removed the dependency and began the journey alone.

I'm using Quasar 2.7.7 with TypeScript and ESLint.

Added Cypress 10 (~~10.4.0~~ 10.6.0) and @cypres/webpack-dev-server (~~2.1.0~~ 2.2.0).
Run cypress open, follow the official migration guide and the guide from this pull request's readme.

Copied the function for getting webpack config from this pull request and adjusted devServer option in cypress.config.ts


import { devServer as webpackDevServer } from '@cypress/webpack-dev-server'
import { quasarWebpackConfig } from 'test/cypress/quasar-dev-server' // this a location of the file I copied from this pull request, replace with your path
import DevServerConfig = Cypress.DevServerConfig

const devServer = async (devServerOptions: DevServerConfig) => webpackDevServer({ ...devServerOptions, // eslint-disable-next-line @typescript-eslint/no-unsafe-assignment,@typescript-eslint/no-unsafe-call webpackConfig: await quasarWebpackConfig(), framework: 'vue' })

4. Added `indexHtmlFile` to `component` option in `cypress.config.ts`. The `component` option:

component: { supportFile: 'test/cypress/support/component.ts', specPattern: 'src/*/.spec.ts', indexHtmlFile: 'test/cypress/support/component-index.html', devServer }

5. Added `'**/*.cy.{js,ts}'` in `.eslintrc.js` in `override` section.
6. Copied `registerCommands()` function from this pull request's repository and changed the import path.
7. Added `cy.mount` command (and replaced `mount()` occurrences in every spec).
At this point, there were some exceptions like Notify plugin not working etc. and after some digging I think the `installQuasarPlugins()` function and other global config options stopped working. They relied on `config` variable from `@vue/test-utils` so instead of modifying the config globally I moved it to `cy.mount()` instead.
The `cy.mount()` is:
```ts
Cypress.Commands.add('mount', (component, options: MountingOptions<unknown> = {}) => {
  options.global = options.global || {}
  options.global.stubs = options.global.stubs || {}
  options.global.components = options.global.components || {}
  options.global.plugins = options.global.plugins || []
  options.global.mocks = options.global.mocks || {}

  options.global.plugins.unshift([Quasar, { plugins: { Notify, Dialog, Cookies } }]) // replace with the plugins you need
  options.global.plugins.push(i18n) // if you need i18n add import { i18n } from 'src/boot/i18n' on top of the file

  return mount(component, options)
})

There are some changes in @vue/test-utils which broke the type inference when using findComponent (and similar commands) so I had to manually cast to unknown and then to VueWrapper<MyComponent>.
[^1] I've noticed my CI job takes forever (it would fail after default timeout in GH Actions - 5 hours) and after investigation the issue is quasar cannot find files it generates in .quasar directory. Because I had these files in local environment given the fact I run quasar dev and cypress open simultaneously it didn't affect my local workflow but affects CI. Looks like quasar does not generate these files when running webpack dev server within Cypress. The workaround is unignore .quasar in .gitignore add commit all the files from .quasar. I'm the only developer working on the project so I consider this a safe workaround but I don't think it's so safe when working in a team.
There are some issues with html-webpack-plugin when using Yarn 1 please see this issue. Fixed it by adding to my package.json:
```
"resolutions": {
"html-webpack-plugin": "^5.5.0"
},
```

Here are some issues that are just annoying but you don't need to workaround them:

[^1] ~~When running component tests quasar logs [Quasar] Running SPA.~~
[^1] Quasar tries to mount the application at #q-app and if you tag the div in component-index.html the warning goes away but then you produce unwanted side effects (I fetch session details on page load in App.vue, don't need those in component tests).
[^1] ~~I'm using Notify plugin and I noticed there is two #q-notify divs. Functionality works as expected, notifications are rendered only in the last div.~~
There is couple of unnecessary divs in the component testing iframe: div[data-cy-root] > div[id="__cy_vue_root"] > div[data-v-app]
In Cypress 10.6.0 if you stub the component, mount it, and use Notify plugin div[#q-notify] gets tagged with [data-v-app] attribute.

Hope you can find this insightful.

[^1]: Fixed in https://github.com/cypress-io/cypress/issues/23224

IlCallo commented 2 years ago

Thanks for the detailed report! As you probably noticed, we're waiting for Cypress team to allow webpackConfig/viteConfig to accept a Promise, as we bumped in the same problems you did (especially the html-webpack-plugin)

I'll try to apply/investigate other problems you mentioned 👍🏻

sudocovik commented 2 years ago

I was browsing Cypress changelog and noticed interesting bugfix in 10.5.0. I've updated to 10.6.0 and noticed a couple of problems are gone. Please see my initial post, crossed out things that got fixed by that bugfix and added one more problem (annoying problem, not show stopper)

hinogi commented 2 years ago

I was browsing Cypress changelog and noticed interesting bugfix in 10.5.0. I've updated to 10.6.0 and noticed a couple of problems are gone. Please see my initial post, crossed out things that got fixed by that bugfix and added one more problem (annoying problem, not show stopper)

I guess you have to look out for the cypress dev server version that is used with the cypress version since there are some big difference in the new dev server implementation, I tried that at the beginning like the root attribute and such.

hinogi commented 1 year ago

@IlCallo @Covik This should now work

IlCallo commented 1 year ago

Webpack is now working correctly, Vite apparently hit 404 on all loaded components, I'm investigating why

hinogi commented 1 year ago

oh i forgot about vite totally

IlCallo commented 1 year ago

Unluckily I can't wrap up the work once more 🥲 The problem seems to be Quasar config for Vite break Cypress assumptions in some way, eg by adding a plugin it don't expect

@hinogi @Covik this seems a promising path if you want to investigate: https://github.com/cypress-io/cypress/issues/22505

yusufkandemir commented 1 year ago

Explanation of the Vite problem: https://github.com/cypress-io/cypress/issues/22505#issuecomment-1277855100 I suggested a workaround for it in this PR: https://github.com/quasarframework/quasar-testing/pull/259#discussion_r994857387 I tested this solution with yarn test:component and yarn:test:component:ci. They both work as expected.

hinogi commented 1 year ago

Explanation of the Vite problem: cypress-io/cypress#22505 (comment) I suggested a workaround for it in this PR: #259 (comment) I tested this solution with yarn test:component and yarn:test:component:ci. They both work as expected.

great find, how did you uncover it?

quasarframework / quasar-testing

feat(cypress): support Cypress 10 #259