quasarframework / quasar-testing

Testing Harness App Extensions for the Quasar Framework 2.0+
https://testing.quasar.dev
MIT License
179 stars 65 forks source link

Dialog, Notify and Emit mock/access is not available in test environment anymore #314

Closed AlphaJuliettOmega closed 1 year ago

AlphaJuliettOmega commented 1 year ago

Visual Studio Code

OS: MacOs Node: 8.9.0 NPM: v16.13.2

Any other software related to your bug: Unknown where the issue comes from.

  "dependencies": {
    "@ckeditor/ckeditor5-vue": "^4.0.1",
    "@quasar/extras": "^1.15.7",
    "@scandltd/vue-injector": "^4.0.2",
    "aws-amplify": "^4.2.4",
    "axios": "^0.21.1",
    "core-js": "^3.16.2",
    "csvtojson": "^2.0.10",
    "cypress-file-upload": "^5.0.8",
    "is-valid-domain": "^0.1.6",
    "quasar": "^2.10.2",
    "util": "^0.12.4",
    "vue": "3",
    "vue-json-pretty": "^2.0.6",
    "vue-router": "4",
    "vuex": "^4.0.2"
  },
  "devDependencies": {
    "@babel/core": "^7.20.5",
    "@babel/eslint-parser": "^7.19.1",
    "@babel/preset-env": "^7.20.2",
    "@quasar/app-webpack": "^3.6.2",
    "@quasar/quasar-app-extension-testing": "^2.0.6",
    "@quasar/quasar-app-extension-testing-unit-jest": "^3.0.0-beta.5",
    "@types/node": "^18.11.11",
    "@typescript-eslint/eslint-plugin": "^5.46.0",
    "@typescript-eslint/parser": "^5.46.0",
    "@vue/test-utils": "^2.2.6",
    "cypress": "^10.7.0",
    "dotenv": "^10.0.0",
    "eslint": "^8.29.0",
    "eslint-config-standard": "^17.0.0",
    "eslint-plugin-import": "^2.26.0",
    "eslint-plugin-jest": "^27.1.6",
    "eslint-plugin-n": "^15.6.0",
    "eslint-plugin-node": "^11.1.0",
    "eslint-plugin-promise": "^6.1.1",
    "eslint-plugin-vue": "^9.8.0",
    "jest": "^29.3.1",
    "jest-serializer-vue": "^3.1.0",
    "pre-commit": "^1.2.2",
    "timekeeper": "^2.2.0",
    "typescript": "^4.9.4"
  }

What did you get as the error?

all tests failing in a suite of 550 after a quasar upgrade -i After much manual work there was un-communicated migration issues related to deps and configs, cleaned up. Now on current versions of everything, yarn test only fails ±40/550 tests.

Dialog example:

    TypeError: Cannot set properties of undefined (setting 'dialog')

      126 |     // eslint-disable-next-line
      127 |     // @ts-ignore
    > 128 |     wrapper.vm.$q.dialog = jest.fn().mockReturnThis()
          |                         ^
      129 |     wrapper.vm.$q.dialog().onOk = jest.fn()

Notify examples:

    TypeError: Cannot set properties of undefined (setting 'notify')

      31 |       }
      32 |     })
    > 33 |     wrapper.vm.$q.notify = jest.fn()
         |                         ^
    TypeError: Cannot read properties of undefined (reading 'notify')

      216 |     resultsTable = wrapper.find('[data-test="resultsTable"]')
      217 |     expect(resultsTable.exists()).toBe(false)
    > 218 |     expect(wrapper.vm.$q.notify).toHaveBeenCalledTimes(1)
          |                          ^

Emit example:

  console.warn
    [Vue warn]: Component emitted event "virtual-scroll" but it is neither declared in the emits option nor as an "onVirtual-scroll" prop.

      68 |     const add = wrapper.find('[data-test="add"]')
      69 |
    > 70 |     await (table as any).vm.$emit('virtual-scroll', { to: 0, ref: { refresh: () => null } })

What were you expecting?

The notify, dialog and $emits properties to still exist on the wrapper.vm.$q object after update. No changes were documented anywhere that I could find that touches on this behaviour so I do not know how to fix/mitigate.

What steps did you take, to get the error?

quasar upgrade -i and a lot of manual work to fix seemingly breaking config and type changes. I did not change anything re. these quasar components that are now undefined in the Jest environment.

AlphaJuliettOmega commented 1 year ago

I tried this too:

import { Dialog, Notify } from 'quasar';

installQuasarPlugin({ plugins: { Dialog, Notify } });

because that helped another user here: https://github.com/quasarframework/quasar-testing/issues/201#issuecomment-1008694171

But it does not make a difference in my tests in a .spec.ts file whether I add the plugins parameter to the installQuasarPlugin function.

The same notify/emit/dialog is undefined still occurs regardless, no change in errors/output.

AlphaJuliettOmega commented 1 year ago

Interesting.

import { QDialog } from 'quasar'
installQuasarPlugin({ components: { QDialog } })

Fixes the Dialog issue, but doesn't work for the Notify.

import { QDialog, Notify } from 'quasar'
installQuasarPlugin({ components: { QDialog, Notify}})

results in:

Type '{ QDialog: ComponentConstructor<QDialog, any, any, any, ComputedOptions, MethodOptions>; QNotify: any; }' is not assignable to type 'QuasarComponents'.
  Object literal may only specify known properties, and 'QNotify' does not exist in type 'QuasarComponents'.ts(2322)
import { QDialog, Notify } from 'quasar'

installQuasarPlugin({ components: { QDialog }, plugins: { Notify } })

Which lints as 'valid' results in Notify still being undefined.

I feel like I'm probably just missing information related to what a Component vs a Plugin is and how these are supposed to be used in Testing now vs how it was possible/automated before.

IlCallo commented 1 year ago

image

Components are all the ones into "Vue Components" in the docs, plugins are the ones into "Quasar Plugins" into the docs Many plugins need to be registered, as they are opt-in Components are usually auto-imported via a loader thanks to Quasar, but Jest won't allow it so you need to manually mount which components you need

I actually noticed you didn't need to import components anymore some months ago, but I couldn't understand what happened. Apparently it was a Quasar bug of some kind and it now reversed to the previous behaviour

Could you open a draft PR adding one or more example tests which are using Dialog or Notify as you need? We'll try to check it out and make it work, so we'll also have examples for future users and a benchmark to run tests against for future releases

Can you pinpoint which version of quasar packages generated the error?

AlphaJuliettOmega commented 1 year ago

@IlCallo the initial comment has examples of usage, probably not ideal but it was the only way that it even worked in the test environment.

I believe the change is related to Quasar 2.10 and the Test Tools going from Alpha to Beta. There's some seemingly contradicting information here that's related:

Dependencies externalization, upgrade and removal
We adapted to the ecosystem shift towards externalizing dependencies, we now manage supported versions of third parties packages via peerDependencies. Run yarn add -D jest @vue/test-utils to install previously bundled dependencies.

Run yarn add -D @quasar/quasar-app-extension-testing-unit-jest@beta eslint-plugin-jest@latest to upgrade dependencies which were already externalized.

Here are some resources which you should read, adapting your tests accordingly:

[Jest 26 to 27](https://jestjs.io/blog/2021/05/25/jest-27)
[Jest 27 to 28](https://jestjs.io/docs/28.x/upgrading-to-jest28)
[Jest 28 to 29](https://jestjs.io/docs/upgrading-to-jest29)
[Vue Test Utils 2.0 migration guide](https://next.vue-test-utils.vuejs.org/migration/)
Here are some resources which you should read only if you still have errors after having applied this guide steps:

[Vue Test Utils releases](https://github.com/vuejs/test-utils/releases?page=2). Users upgrading from this package alpha release should check release changelogs from v2.0.0-rc.11 onwards.
[Vue Jest releases](https://github.com/vuejs/vue-jest/releases). Users upgrading from this package alpha release should check release changelogs from v27.0.0-alpha.2 onwards.
[TS Jest changelog](https://github.com/kulshekhar/ts-jest/blob/HEAD/CHANGELOG.md). Users upgrading from this package alpha release should check release changelogs from v26.5.6 onwards.

I just need Dialog, emit and whatever normal features and functionality are available in Quasar, to have Typings on them and be available in the Jest/Test environment.

wrapper.vm.$q.notify / dialog

Previously gave you a way to see if a Notification or Dialog was called in the Test environment.

I need to find another way to mock that now.

AlphaJuliettOmega commented 1 year ago

Might be related to this older issue, unsure: https://github.com/quasarframework/quasar-testing/issues/221

Or the mounted inside Notify itself, i'm going to try the mocking strategy in this thread next

AlphaJuliettOmega commented 1 year ago

Doesn't work. for one, the example in that thread uses an array parameter to plugins:

installQuasarPlugin({
  plugins: [ Dialog ]
})

but current API wants plugins to be an object:

installQuasarPlugin({
  plugins: { Dialog }
})

And results in the current non-working situation:

   TypeError: Cannot read properties of undefined (reading 'dialog')

      145 |     await flushPromises()
      146 |
    > 147 |     expect(wrapper.vm.$q.dialog().onOk).toHaveBeenCalledTimes(1)

To my understanding right now: $q isn't defined, the Quasar plugin / useQuasar doesn't work in Jest like it used to.

I have not found any documentation or comment re. the plugins declaration/functionality change if there is one.

IlCallo commented 1 year ago

There's some seemingly contradicting information here that's related:

Not sure what's contradictory in that part of the README, could you explain it?

Doesn't work. for one, the example in that thread uses an array parameter to plugins:

That's about localVue, so it shows the old syntax to register Vue2 plugins. Here we're talking about Quasar plugins, which never had an array-like syntax AFAIK


I just tried to reproduce the problems you're experiencing into the test project, upgrading all relevant dependencies one by one to see if one of them is the culprit.

Apparently the problem appears in VTU release v2.2.3, so it isn't related to Quasar or this AE I'd suggest you to check out that release changelog to investigate what's the problem, or just revert to v2.2.2 for now

Once you discovered the problem, you could enhance the Dialog test available here and add a non regression test in case this problem pops up again

AlphaJuliettOmega commented 1 year ago

Not sure what's contradictory in that part of the README, could you explain it?

I should have chosen my words more wisely, I cannot really find contradictory statements. It's just a case of complexity / real use cases being just a hint more complex than the examples so the readme doesn't take me very far. For many tests/mock/stub setups it's mostly experimentation and scouring random forums and old closed issues to get them working instead of there being a super clear process to testing a component/feature.

Here we're talking about Quasar plugins, which never had an array-like syntax AFAIK I apologize for my misunderstanding and appreciate the correction.

Updating to the newer syntax for adding the plugins/components did the trick for Dialog, like so for example:

installQuasarPlugin({
  components: {
    QDialog,
    QCard,
    QCardSection,
    QBtn,
    QTable,
    QIcon,
    QTr,
    QTd,
    QSelect,
    QForm,
    QFile
  },
  plugins: { Dialog, Notify }
})

This fixed all Dialog and Emit test issues I was having, but I did not find a solution for Notify. I've opted for not testing the Notify as the tests already cover all functionality leading to the notification, I trust the Quasar Notify to work, it doesn't make sense to test every possible notification imo.

That can be tested in E2E tests.

Thanks again for the super constructive discussion, I appreciate your time.

IlCallo commented 1 year ago

No problem, thanks for coming back and sharing your solution :)

Unluckily "automated test" in itself is a pretty complex topic and many quirks depend on each project custom setup If you feel we can enhance the README in some way or add some example tests which would have helped you, please share your ideas

Just be aware that we cannot put examples which are too specific, that's a slippery slope that leads to unreadable documentation in the long run