Add ui tests - Githubissues

jupyterlab / extension-examples

JupyterLab Extensions by Examples

BSD 3-Clause "New" or "Revised" License

454 stars 167 forks source link

Add ui tests #171

Closed fcollonval closed 3 years ago

fcollonval commented 3 years ago

Add playwright to test the examples

Remaining examples:
- [x] settings
- [x] state
- [x] toolbar-button
- [x] widget
[x] Tests on CI

This uncovers a bug in style packaging.
This is using playwright test runner (no galata).

There are nice examples of playwright code for:

capturing log console message (hello-world, settings,...),
capturing alert message (settings,...),
emulate drag and drop (move a panel in the main area to create a split panel) (kernel-output & log-messages).
get select current value (state)
wait for a request (state)

log console and alert dialogs are not capture when generating videos from tests. So it seems unappropriate to replace existing preview with those videos although they will allow to be up to date.

What do you think @jtpio @echarles?

jtpio commented 3 years ago

Thanks for starting this!

This is using playwright test runner (no galata).

This could be useful to lay the foundations for a potential rework in Galata discussed in https://github.com/jupyterlab/galata/issues/67

So it seems unappropriate to replace existing preview with those videos although they will allow to be up to date.

Some time a while ago it was also discussed whether to check the screencasts in the repositories because of their size. Probably this will be a similar issue with videos?

fcollonval commented 3 years ago

Probably this will be a similar issue with videos?

Yes. What I had in mind was to create a manual workflow (or one triggered by new JLab release) that would have open a PR to update those videos/screenshots only once in a while (with maybe a conversion to GIF and cutting the first few seconds corresponding to the logo animation for the videos).

fcollonval commented 3 years ago

The failure is on Mac and not related to this.

jtpio commented 3 years ago

Probably this could fix https://github.com/jupyterlab/extension-examples/issues/45?

Or do we still want to have examples of setting up unit tests in this repo?

jtpio commented 3 years ago

Also we might want to make the ui-tests more discoverable, for example by linking to the folder from the example README.md?

So folks interesting in this could learn how to set them up. This could be in the hello world example, or in a dedicated testing example.

fcollonval commented 3 years ago

I would rather create a dedicated example to talk about testing (addressing ui-tests and jest tests). But I want to work on galata to stabilize its usage with playwright before doing so.

I'll merge this one and open a PR to add a section about the tests in the README.