Set up a CI/CD Pipeline with Automated Testing

[noi5e]

Starting the conversation around setting up a CI/CD pipeline, particularly, automated testing.

Ultimately, the first iteration of NLD Web 2.0 is going to be pretty simple, just like the current site is (a map that displays polygons with links to pages containing more info about individual features).

I think it would be valuable in the long run to set up (since we're anticipating more open source contributions in the future) automated testing to protect the basic functions of the site, which mostly are centered around the front page Mapbox map.

---

Example

Perhaps a basic flow would be something like this:

• Upon pull request:
  • Deploy PR code to a staging server on AWS
  • Run Selenium or other automated browser tests on the front page map

The tests would be of the following nature:

it should display the front page map...
it should zoom to one of 3 specified locations on page load...
it should display polygons for territories...
it should display polygons for languages...
it should display polygons for treaties...
it should show a list of selected features on map click...
it should highlight selected features with a black stroke and change fill color...
it should provide working links to individual features in the selected territories list...
it should display a list of suggested locations on user search entry...
it should zoom to a geocoded location on user input of location...
etc...

The point is that since so much of the site revolves around the front page map—user interaction, how it looks, etc—we need automated browser testing so we can protect the basics, while still allowing for contributors to build out new features.

---

Next Steps and Details

• For now, native-land-web-2.0 is built with Mapbox, so that's what we have to work with.
  • Mapbox GL JS docs have a little piece about browser testing that may or may not be relevant.
• Also, we are using react-map-gl to integrate Mapbox GL JS with React.
  • I believe deck.gl can integrate with our current Mapbox build
  • This would allow us to use some of their testing features
• There are practical questions around AWS I don't know about yet, particularly around cost and can NLD afford to deploy an app instance on every PR?

There is a lot about this particular aspect that I'm learning as I go, but I'm reading around, and planning on tinkering with this soon and seeing what works and makes sense.

@gdomingu @kyleramirez @rudokemper

[rudokemper]

Thanks @noi5e!

I think it could be good to look into non-profit programs with some of the possible services that we could use for automated testing. I personally use Docker in most of my work, and Dockerhub has a very open and generous program for non profits, so I do most of my automated testing (upon PR submission) via Github Actions and Docker. CircleCI and Bitrise could also be worth exploring.

[noi5e]

After doing some research, I'm getting a good sense of how to approach end-to-end testing for our app.

My impression is that testing WebGL maps is difficult, roundabout, and that testing solutions are lacking. There are barriers to simulating or querying the current state of a <canvas> element within a NodeJS testing environment.

These are a few of the approaches that I could find (definitely chime in if you know something that I don't!):

1. Visual regression type testing in Cypress
   • ie. take a screenshot of the front page map in main, and do a visual comparison with pull-request to see if anything has broken
2. Mocking maps in Jest
   • Simulating a <canvas> element with JavaScript
   • I'm guessing the testing possibilities are limited compared to 1
3. deck.gl's testing suite
   • This still seems promising to me
   • I'm going to tinker around with this on a branch to see if this offers anything that 1 and 2 cannot
4. Plain old manual testing, without automation

Our app is pretty simple, so whatever tests we come up with don't need to be too complicated at the end of the day.

If we can get a good combination of 1, 2, and maybe 3, set up, AND cover our bases with 4 and automatic deployments on AWS (or DockerHub, thanks for the suggestion @rudokemper), that should be pretty decent coverage.

I'm starting work on writing Cypress visual regression tests ASAP.

[noi5e]

Also dropping something that @kyleramirez mentioned on Slack:

5. Migrating to self-hosting vector tiles, and having multiple environments. This will help a lot with testing. We can render maps with a tool like LightGun to make screenshots of individually rendered map features, and alert us if we have broken anything. It will also make it so that we can package all those map shapes and assets nicely, make it more portable.

Moving toward open-source and self-hosting sounds great to me, will be a long-term goal though 🤞🏽