Implement browser automation tests with pytest-selenium

[simsong]

Currently, we have no way of testing planttracker.js to make sure it works.

[sbarber2]

Here is what appears to be a comprehensive, if somewhat overwhelming, summary of JavaScript testing tools

[simsong]

The way for us to move forward on JavaScript testing is by using pytest-selenium, which integrates with our pytest framework. This python module spins up a selenium browser and then lets you click on things and check the results. Here are some pages I found useful:

• https://pytest-selenium.readthedocs.io/en/latest/user_guide.html
• https://www.lambdatest.com/blog/end-to-end-tutorial-for-pytest-fixtures-with-examples/

It also allows testing with other browsers.

@sbarber2 - do you want to try to code something up?

[sbarber2]

So, Selenium is a browser automation framework. It's API operates that level of HTTP interactions and DOM entity access. It's not a unit testing framework per se. And while it has a JavaScript binding for its browser control API, it's not oriented toward the direct testing of JavaScript code. The Selenium JavaScript API is intended to run in a Node.js instance separate from the browser under test -- a Selenium script written in JavaScript has no access at all to any JavaScript code or runtime entities running in the webapp under test. Selenium is a remote control tool, basically.

That doesn't mean Selenium won't be valuable for Plant-Tracer/webapp testing. It's especially useful for automating so-called end-to-end (E2E) testing using a browser as the front-end, emulating user actions.

But additional layers on top of Selenium itself may be useful, as well as parallel testing frameworks and tools that focus directly on JavaScript functional testing, running either in a browser or "standalone" (that is, testing JavaScript function behavior via Node.js).

pytest-selenium is cool but it doesn't seem to have broad enough adoption that there are any examples or tutorials easily available. Most of the pytest selenium tutorials and examples out there (including the one at the lambdatest link in the first comment in this issue) use the PyPI selenium package directly rather than pytest-selenium -- that is, the tutorials create their own pytest functions and/or fixtures using the selenium python API binding rather than indirecting through the selenium fixture provided by pytest-selenium.

I'm not sure at this point which approach is "better" for Plant-Tracer/webapp, as documentation and examples are scarce and we'll have to learn through experience.

Here is a super-simple pytest test module that shows both ways of doing pytests with selenium:

import pytest
import pytest_selenium

# not needed for using pytest-selenium, just selenium directly
from selenium import webdriver

# test function illustrating the use of pytest-selenium package
# pytest-selenium itself uses the selenium package
def test_example_pytest_selenium(selenium):
    selenium.get('https://www.google.com')
    assert selenium.title == 'Google'

# test function illustraing the use of selenium package directly
def test_example_just_selenium():
    browser_driver = webdriver.Chrome() # the browser type could be externalized

    browser_driver.get('https://www.google.com')
    assert browser_driver.title == 'Google'

This test module is run with a command line like this:

python3 -m pytest --driver Chrome tests/selenium_example_test.py

My plan at the moment for this Issue to expand the example tests above to create a few tests using each paradigm that will be useful examples in the context of Plant-Tracer/webapp.

Perhaps I will break this Issue out into more than one issue, since doing E2E browser-based testing and browser-embedded JavaScript functional testing and standalone JavaScript functional testing are all different things.

Open questions for Plant-Tracer browser-based testing include "Which browsers?" and "How do we do cross-browser testing?" and "Does all this work headless from GitHub Actions-initiated test runs?"

So there's plenty to do here just with Selenium!

[sbarber2]

Modified scope of this Issue to exclude JavaScript functional unit testing; browser automation testing with selenium only -- using Python selenium bindings/apis/packages.

[sbarber2]

JavaScript functional unit testing now in #192

[simsong]

More things to try: Try adding --headless per https://stackoverflow.com/questions/77284244/devtoolsactiveport-file-doesnt-exist-sessionnotcreatedexception-selenuim-python

Also try: chrome_options.add_argument('--remote-debugging-pipe')

4403 - chromedriver - WebDriver for Google Chrome - Monorail bugs.chromium.org

per https://bugs.chromium.org/p/chromedriver/issues/detail?id=4403#c35

Also: https://stackoverflow.com/questions/50642308/webdriverexception-unknown-error-devtoolsactiveport-file-doesnt-exist-while-t?page=1&tab=scoredesc#tab-top