The rules_webtesting workspace currently supports several different languages, but one place it is lacking is JavaScript / TypeScript. Since these are the required languages for working with browsers, many developers would prefer to work within them instead of having their end-to-end tests written in a different language. rules_nodejs already provides a lot of the core Bazel functionality for working with JavaScript / TypeScript, so I think this should be fairly straightforward to support.
Currently if users want to do end-to-end testing in JS/TS, their options mainly include:
This requires setting up a WebDriver server and downloading the relevant browsers and drivers, which can be quite involved and is the main problem rules_webtesting solves.
The first two are probably the most attractive, but both of these ship browsers and their drivers via NPM, so any edit to the package.json file causes them to be redownloaded. Add in the need to run a bare npm install to bring package-lock.json up to date with any changes, and npm ci in certain situations, plus the fact that rules_nodejsWORKSPACE rules also run npm install, means that even minor, unrelated package.json changes can result in 3-4 downloads of the relevant browsers. That can easily exceed 500MB. Even in Silicon Valley this can still take 15+ minutes for me from a trivial package.json edit on my home network (because American ISPs are highly competitive with each other). I'm sure others feel this pain even worse.
rules_webtesting already solves this problem since browsers are versioned via Bazel and not dependent on package.json. We just need a way to author tests in JS/TS.
I propose a new jasmine_node_test_suite() rule, similar to the existing *_web_test_suite() rules, but just using jasmine_node_test(). It would simply set up WebDriver in the same fashion and set the relevant environment variable. We would leave it up to users what library they want to use to connect to that instance (WebDriverIO, WebDriverJs, etc.).
I implemented this strategy in one of my projects recently and I think it worked out pretty well TBH. Most of the work went into re-implementing wrap_web_test_suite() and figuring out how to use WebDriverIO like a simple library (the latter would the user's problem, not rules_webtesting's problem, so nothing we need to worry about there). In particular, it makes editing package.json a much less painful experience.
rules_nodejs also supports a few other testing systems, so I want to call out some of the alternative options:
nodejs_test() - This is more generic than jasmine_node_test() and arguably a better candidate to avoid tying rules_webtesting to any one particular JS/TS testing framework. Unfortunately, it's also less useful and requires manually integrating the testing framework, which is a lot of extra work to ask for a user.
Jest - This appears to be possible in rules_nodejs, but does not have an official implementation. A Jest user would most likely be able to use it with rules_webtesting given a similar amount of effort as was required for my experiment, and I think that's fairly reasonable given its support status in rules_nodejs.
Karma - Tests run in the browser, which doesn't make sense with WebDriver.
I personally think jasmine_web_test_suite() is the best option here, but I'm interested to hear other opinions on this. Arguably we could have a more direct integration with an actual JS/TS implementation of WebDriver, but I don't thinks that's strictly necessary or would improve the developer experience much. I think it's easier and more flexible for users if rules_webtesting just sets up the WebDriver server and lets the user figure out how to connect to it.
If this seems like a good direction to take, I would be happy to take a stab at the implementation. I already did most of the same work for my project, so upstreaming it here would make things simpler on my end and allow others to more easily use the trick.
The
rules_webtestingworkspace currently supports several different languages, but one place it is lacking is JavaScript / TypeScript. Since these are the required languages for working with browsers, many developers would prefer to work within them instead of having their end-to-end tests written in a different language.rules_nodejsalready provides a lot of the core Bazel functionality for working with JavaScript / TypeScript, so I think this should be fairly straightforward to support.Currently if users want to do end-to-end testing in JS/TS, their options mainly include:
cypress_web_test().nodejs_test().nodejs_test().rules_webtestingsolves.The first two are probably the most attractive, but both of these ship browsers and their drivers via NPM, so any edit to the
package.jsonfile causes them to be redownloaded. Add in the need to run a barenpm installto bringpackage-lock.jsonup to date with any changes, andnpm ciin certain situations, plus the fact thatrules_nodejsWORKSPACErules also runnpm install, means that even minor, unrelatedpackage.jsonchanges can result in 3-4 downloads of the relevant browsers. That can easily exceed 500MB. Even in Silicon Valley this can still take 15+ minutes for me from a trivialpackage.jsonedit on my home network (because American ISPs are highly competitive with each other). I'm sure others feel this pain even worse.rules_webtestingalready solves this problem since browsers are versioned via Bazel and not dependent onpackage.json. We just need a way to author tests in JS/TS.I propose a new
jasmine_node_test_suite()rule, similar to the existing*_web_test_suite()rules, but just usingjasmine_node_test(). It would simply set up WebDriver in the same fashion and set the relevant environment variable. We would leave it up to users what library they want to use to connect to that instance (WebDriverIO, WebDriverJs, etc.).I implemented this strategy in one of my projects recently and I think it worked out pretty well TBH. Most of the work went into re-implementing
wrap_web_test_suite()and figuring out how to use WebDriverIO like a simple library (the latter would the user's problem, notrules_webtesting's problem, so nothing we need to worry about there). In particular, it makes editingpackage.jsona much less painful experience.rules_nodejsalso supports a few other testing systems, so I want to call out some of the alternative options:nodejs_test()- This is more generic thanjasmine_node_test()and arguably a better candidate to avoid tyingrules_webtestingto any one particular JS/TS testing framework. Unfortunately, it's also less useful and requires manually integrating the testing framework, which is a lot of extra work to ask for a user.rules_nodejs, but does not have an official implementation. A Jest user would most likely be able to use it withrules_webtestinggiven a similar amount of effort as was required for my experiment, and I think that's fairly reasonable given its support status inrules_nodejs.rules_webtestingis not useful. Also it's support status is currently unclear.I personally think
jasmine_web_test_suite()is the best option here, but I'm interested to hear other opinions on this. Arguably we could have a more direct integration with an actual JS/TS implementation of WebDriver, but I don't thinks that's strictly necessary or would improve the developer experience much. I think it's easier and more flexible for users ifrules_webtestingjust sets up the WebDriver server and lets the user figure out how to connect to it.If this seems like a good direction to take, I would be happy to take a stab at the implementation. I already did most of the same work for my project, so upstreaming it here would make things simpler on my end and allow others to more easily use the trick.