OneZoom / OZtree

OneZoom Tree of Life Explorer
Other
87 stars 18 forks source link

Add tests to check all pages load without errors (in different situations) #809

Open hyanwong opened 5 months ago

hyanwong commented 5 months ago

We did have some (probably poorly written) tests that I wrote that spidered through the OneZoom site to check all the pages were working (e.g. broken pages caused by stuff like https://github.com/OneZoom/OZtree/issues/779). Ideally these would also catch ajax-y type errors like https://github.com/OneZoom/OZtree/issues/794 and https://github.com/OneZoom/OZtree/issues/799, logging-in errors like https://github.com/OneZoom/OZtree/issues/791, and also JS errors like https://github.com/OneZoom/OZtree/issues/801.

@lentinj : do you know the status of this sort of testing? Is there any way to start setting up stuff to run on GH PRs that tests at least some of this (it's hard, I guess?). It worries me that we can release a site in which some of this is broken.

As a slight aside, we should also check that hard-coded OTTs, especially the ones in https://github.com/OneZoom/OZtree/blob/main/views/default/footer_sponsor_items.load, are still retained in the database (see https://github.com/OneZoom/OZtree/pull/812 for a fix when this failed). We could perhaps do this when building a new tree, e.g. with added instructions in https://github.com/OneZoom/tree-build/blob/main/oz_tree_build/README.markdown#upload-data-to-the-server-and-check-it (point 10).

lentinj commented 5 months ago

Chrom(e|ium) headless mode has gotten rid of the faff of webdrivers, and there's various actions for github actions to handle the setup for us, but the fundamental problems that make the tests in tests/functional hard to write and brittle long-term haven't gone away.

We've talked a few times about a minimal tree MySQL dump that's part of the OZtree repo, that could be enough both for new developers to get up to speed without needing persuade a dump out somewhere, and for github actions to be able to configure MySQL and run the tests/unit tests. That would include the controllers/default test, which triggers the controller for each endpoint there.

Once that works we could think about resurrecting the functional tests. If we can make them work locally, github actions shouldn't be too much of a stretch.

Generically loading every page and seeing if there's Javascript errors won't be easy, since a page being "done" doesn't have an obvious answer, beyond something like "wait 3 seconds after the DOM is loaded". But then the tests get very slow, and a github action on every commit is probably the wrong approach.

lentinj commented 5 months ago

check that hard-coded OTTs . . . are still retained in the database

This is somewhat similar to the discussion in #742, although having a custom default startpoint was abandoned in the end, configuring these hardcoded OTTs in tree-build-land does feel like the right place for them.