This PR makes it possible to author, and run, RStudio automation / integration tests from within RStudio itself. The intent is to easy to:
Interactively author and test your automation tests from within the IDE;
Make it possible to run an RStudio instance's automation tests non-programmatically, by launching RStudio with --run-automation.
The ultimate goal is to make it easy enough to author automation tests that they can be included as part of pull requests when appropriate, with those tests run on Jenkins as well.
The video below gives an example of me running the automation tests for a RStudio Desktop instance, from within my development RStudio Server instance.
This PR builds upon the Chrome DevTools Protocol, which exposes a websocket API for driving a Chromium instance. This is what gets used under the hood by tools like Selenium; however, we're creating and accessing the websocket by hand ourselves. This is all done through the R websocket package (maintained by us).
For non-interactive use, RStudio can now be launched with the --run-automation flag. This runs RStudio in a "headless" mode, where it will then launch a separate instance of RStudio with the --automation-agent flag, which will then instruct that automation agent to expose some extra hooks to make automation testing easier. (Currently, this just means it'll be easier to list application commands, and execute application commands, from Javascript.)
Automated Tests
An important question... who tests the automated tests?
QA Notes
We should consult with QA whether they'd like to author integration tests here, or continue authoring them in the separate rstudio-ide-automation repository. To further confirm, this PR only deals with the tests themselves; it does not touch any of the existing machinery for running automated tests, which should continue to live in rstudio-ide-automation.
Documentation
I should write a small README discussing the various SessionAutomation tools.
Checklist
[ ] If this PR adds a new feature, or fixes a bug in a previously released version, it includes an entry in NEWS.md
I also think we'll want to bundle the automation tests with the IDE itself, so it can be easily run as-is. (Or, if not, make it easy to check out and run automation tests from a remote source?)
Intent
This PR makes it possible to author, and run, RStudio automation / integration tests from within RStudio itself. The intent is to easy to:
--run-automation
.The ultimate goal is to make it easy enough to author automation tests that they can be included as part of pull requests when appropriate, with those tests run on Jenkins as well.
The video below gives an example of me running the automation tests for a RStudio Desktop instance, from within my development RStudio Server instance.
https://github.com/rstudio/rstudio/assets/1976582/15847cbc-b9cd-4e1a-83a2-4541ce4e2aa5
Approach
This PR builds upon the Chrome DevTools Protocol, which exposes a websocket API for driving a Chromium instance. This is what gets used under the hood by tools like Selenium; however, we're creating and accessing the websocket by hand ourselves. This is all done through the R websocket package (maintained by us).
For non-interactive use, RStudio can now be launched with the
--run-automation
flag. This runs RStudio in a "headless" mode, where it will then launch a separate instance of RStudio with the--automation-agent
flag, which will then instruct that automation agent to expose some extra hooks to make automation testing easier. (Currently, this just means it'll be easier to list application commands, and execute application commands, from Javascript.)Automated Tests
An important question... who tests the automated tests?
QA Notes
We should consult with QA whether they'd like to author integration tests here, or continue authoring them in the separate rstudio-ide-automation repository. To further confirm, this PR only deals with the tests themselves; it does not touch any of the existing machinery for running automated tests, which should continue to live in rstudio-ide-automation.
Documentation
I should write a small README discussing the various SessionAutomation tools.
Checklist
NEWS.md