Backend/API projects as well as databases have no consistently implemented testing strategies. Currently we cannot tell on a consistent basis:
Whether code deployed on post apis is functioning as intended other then a basic 200 response on a root route
Whether code deployed functions as intended, returns expected values
If changes are made either to projects or changes are made to infrastructure, whether existing code continues to function (regression testing)
As we discuss future changes to infrastructure as well as provide a model to future projects/others in open source community, we should take the time to pay down this fairly deep technical debt.
A few current hurdles to providing a good framework (will flesh out what they mean, trying to notate for now):
No testing/staging environment available in the CI/CD framework - we are running tests on production server. This is a bad pattern for a variety of reasons, what if apis make post/delete calls in future? reliance on what the data "is" when writing tests, vs testing code
No guidelines to developers on what should be tested, how to write tests. Testing in the development containers this year was more of an after thought, started with how I implemented in the examplar. We may not need to swing to total test-driven-development but test-with-development should be more of our culture.
Conflagration of tests into Django. It seems that we moved more from testing Django application code to testing whether the api was able to connect to the database.
A couple starting points:
What's define what levels we need to test, ie django code, database level, integration, regression, load testing, and come up with some test plans, understand the best methods to test for each
Reexamine how we run tests, what low/cost, low/configuration options are out there for testing databases, does travis have an option?
Write some regression tests for existing apis. I'd make this a fairly high priority task if any larger changes are undertaken to the infrastructure or projects.
Backend/API projects as well as databases have no consistently implemented testing strategies. Currently we cannot tell on a consistent basis:
As we discuss future changes to infrastructure as well as provide a model to future projects/others in open source community, we should take the time to pay down this fairly deep technical debt.
A few current hurdles to providing a good framework (will flesh out what they mean, trying to notate for now):
No testing/staging environment available in the CI/CD framework - we are running tests on production server. This is a bad pattern for a variety of reasons, what if apis make post/delete calls in future? reliance on what the data "is" when writing tests, vs testing code
No guidelines to developers on what should be tested, how to write tests. Testing in the development containers this year was more of an after thought, started with how I implemented in the examplar. We may not need to swing to total test-driven-development but test-with-development should be more of our culture.
Conflagration of tests into Django. It seems that we moved more from testing Django application code to testing whether the api was able to connect to the database.
A couple starting points:
What's define what levels we need to test, ie django code, database level, integration, regression, load testing, and come up with some test plans, understand the best methods to test for each
Reexamine how we run tests, what low/cost, low/configuration options are out there for testing databases, does travis have an option?
Write some regression tests for existing apis. I'd make this a fairly high priority task if any larger changes are undertaken to the infrastructure or projects.