Open jackocnr opened 4 months ago
I know this effort is already well under way, but I had some quick feedback after starting to work on #1816 (side note: sorry for not getting to it as quickly as Iād expected). Things got a little messy when I tried to write the tests for Jest instead of the older Jasmine tests.
For #1816, I thought it was pretty important to get the actual async loading of the utils script working in the tests (itās still a big TODO comment in Jasmine tests). For Jasmine, that was relatively straightforward ā the tests just needed to run on an HTTP server instead of from a file://
URL.
In Jest, however, things are really complicated by the fact that Jestās ESM support is still experimental (even though Node.jsās normal support is production-ready, Jest uses the vm
module, where it is not):
.js
file is CommonJS or ESM (Jest uses its own module loading system instead Nodeās built-in one), which means it tries to load utils.js
as CommonJS and fails. We canāt tell it the whole package is ESM, either, since most of the other files are CommonJS (e.g. intlTelInput.js
, intlTelInputWithUtils.js
, etc.).Iām pretty sure Iāve found workarounds for all these, but they did make things feel a little awkward and required a lot of digging around for docs and experimenting ā I wonder if using a tool that can run things in an actual browser instead of a simulated one like Jest does would be better.
Anyway, I know stated youāve already evaluated some other options, so Iām not going to push for or advocate a different tool (unless you want me to!). I just wanted to get these thoughts written down after trying to port & expand two test suites.
Thanks for this write up.
I wonder if using a tool that can run things in an actual browser instead of a simulated one like Jest does would be better.
I'm absolutely open to suggestions. I was also disappointed by Jest's (lack of) ESM support.
I've had good experiences with Webdriver.io, but it tends to be a little configuration-heavy (I've only used it with Mocha as the unit test framework and not Jasmine, so I can't say if there are any special caveats with that).
A year and a half ago or so, I tried to use Vitest's new browser mode for a project, but it was extremely under-baked. Looking at the docs now, it looks like it's in much better shape. It also comes with TypeScript support out of the box and tools for React and Vue component testing (it looks like those are untested here right now?). Maybe worth trying out. Downside: it looks like it's designed around issuing browser automation commands, so a lot of the test code may have to get wrapped up in commands to execute JS on the page.
Thanks for your thoughts. I was just reading about Vitest's browser mode and how in the docs it soon says btw you also need to install either playwright or webdriverio, and I checked out playwright and see it is backed by Microsoft, and extremely popular, and thought: why not just use that directly? Is that crazy? I know it is just for E2E testing, but honestly that's all I care about - the experience for the user - those are the only tests I bother writing anyway. I never write unit tests. I'm aware we'd have to wrap the plugin initialisation call in an evaluate call, but otherwise I think it would work pretty well - what do you think? (please feel free to speak your mind here - this is not my area of expertise!)
EDIT: perhaps one concern is how long the tests will take to run?
But on the positive side you have amazing test debugging tools like UI Mode.
I checked out playwright and see it is backed by Microsoft, and extremely popular, and thought: why not just use that directly?
Playwright is great, too!
I mainly mentioned Vitest over Playwright because of Vitestās explicit aims to be largely Jest- (and therefore also Jasmine) compatible, so it follows along more similar lines to what you already have. Playwrightās test module is definitely not attempting to do that. That might not matter much to you, though. Otherwise, Vitest browser has a more serious focus on React/Vue/etc. component testing (Playwright has an experimental plugin for that, but it is clearly not a primary focus), although I think that is definitely pretty low on the list of things are critical for you here.
Since I havenāt really done much with Vitestās browser mode, I canāt really give you a much deeper comparison. I definitely wouldnāt recommend against Playwright. It is much more mature, and that counts for a lot. š
I know it is just for E2E testing, but honestly that's all I care about - the experience for the user - those are the only tests I bother writing anyway. I never write unit tests.
FWIW, it seems like there are quite a few tests here that are more unit test-y (they arenāt simulating user interactions) ā but I take your point that some awkwardness around the more unit-testy bits is an OK sacrifice for more accurate testing of user interactions in the places where you do need to test them. After all, this project is a UI component at the end of the day. š
Also, just to make sure itās clear, all three (Webdriver.io, Vitest browser, Playwright) do E2E. None of them are holding you back from that.
perhaps one concern is how long the tests will take to run?
Iād expect all three will have roughly similar performance overhead: browser launches and page loads are usually going to cost more than anything else the framework is doing, and, as far as I know, each framework behaves pretty similarly in that regard. They all can watch and keep browsers open to efficiently re-run things. š¤·
The one exception is that running Webdriver.io in what I think of as āunit testā mode (run the whole test framework inside the browser, rather than tests running in node and automating the browser with commands, the way Playwright works) can obviously be a bit faster because you arenāt loading a whole new page for each test. But, as you noted before, you want to focus more on E2E-style tests, so this isnāt the way youād mostly use Webdriver.io.
Thank you so much for your input, that's really helpful. I think when I have some time, I will begin migrating to Playwright.
It's time to migrate to a modern testing setup! I've settled on Jest/testing-library and I've got a few Jest tests up and running (in /tests), but there are still a lot of old Jasmine tests to migrate over (in /src/spec). I'll keep track of the migration here. I would welcome pull requests to move more over. Thanks! š
Core
Methods
Options
Static