Police-Data-Accessibility-Project / data-sources-app

An API and UI for using and maintaining the Data Sources database
MIT License
2 stars 5 forks source link

deprecate `regular_api_checks.py` #323

Closed josh-chamberlain closed 3 months ago

josh-chamberlain commented 3 months ago

Context

We used to have a github action that periodically ran to check the API for basic functionality. Now, we have the health monitoring repo.

These tests were related to Police-Data-Accessibility-Project/data-sources-app#154

Requirements

maxachis commented 3 months ago

Prior to eliminating regular_api_check.py, we may want to evaluate whether migrating some of these checks to the Health Monitoring repo is useful. I have a few reasons for why they may not be, however:

  1. Many/all of these test functionality already tested in the newly added integration and unit tests that run in the dev environment -- unless we've reasons to believe prod will differ substantially from dev, running these again would be redundant.
  2. Some of these tests are POST or PUT requests, which would modify the production database when run -- not something that should be done without careful consideration.
  3. Some carry specific "test flags", which have been deprecated from the code, so these would need to be modified in turn.
  4. Granular testing of production may give us less bang for our limited buck than more generalized, non-intrusive tests which target common points of failure -- any endpoint responding will help us validate that endpoints in general are not down; checking that the server and client are online and responding to basic PINGs or ping-equivalents is also important; checking metrics such as latency of response, CPU/RAM/storage load of servers.

I've created an issue advocating for what to include in health monitoring: Police-Data-Accessibility-Project/health-monitoring#3.

EvilDrPurple commented 3 months ago

@josh-chamberlain @maxachis I went ahead and completed the outlined changes in the linked branch. I can hold off on submitting the PR if you wanted to evaluate if there are any checks worth saving before moving forward

maxachis commented 3 months ago

@josh-chamberlain @maxachis I went ahead and completed the outlined changes in the linked branch. I can hold off on submitting the PR if you wanted to evaluate if there are any checks worth saving before moving forward

@EvilDrPurple From my vantage point, you can go ahead and submit: After reviewing, I only saw one that might need re-adding, and only as an integration test, not as a health check: I've noted that in Police-Data-Accessibility-Project/data-sources-app#329

josh-chamberlain commented 3 months ago

Saving the pagination check is good, but I figured since this wasn't working anyway it'd be better to deprecate and move to the new tests. Thanks y'all! @EvilDrPurple you could feel free to look at Police-Data-Accessibility-Project/data-sources-app#329 if you want to work on tests in the new infrastructure.