Right now the Terratests are a bit of a mess so I would like to refactor them by creating some basic methods that you can easily chain together to test the full setup.
The methods I suggest right now are:
TerraformApplyAndDestroy : input testing.t and terraform.options; calls tf init, tf plan, tf, apply and defers tf destroy
AssertContainersAreRunning: asserts if all containers in an ecs cluster are in the running status
AssertAirflowWebserverIsReachable: Checks if you can access the airflow ui
AirflowLogin: logs in to the airflow ui
AssertAirflowLoggedIn: check if you are logged in to airflow
AirflowAddDag: uploads a new dag to s3
AssertAirflowDagExists: checks if a dag is visible in the UI
AirflowRunDag: runs a dag
AssertAirflowDagHasSucceeded: checks if a certain dag has successfully run via the UI
All these methods (except for TerraformApplyAndDestroy) make sure that airflow is deployed in a usable state and that you for example didn't mess up something in the configuration of airflow.
An example test would look like this (pseudo code):
func TestAirflowWithRBAC(){
options = {
"airflow_authentication": "rbac"
}
TerraformApplyAndDestroy(t, options)
AssertContainersAreRunning("my-test-cluster", ["airflow-webserver", "airflow-scheduler"])
AssertAirflowWebserverIsReachable("http://airflow.test")
AirflowLogin("admin", "admin", "http://airflow.test") // only needed when rbac is enabled
AssertAirflowLoggedIn("http://airflow.test") // only needed when rbac is enabled
AirflowAddDag("path/to/dag.py", "s3-bucket-name", "s3/path/key/dag.py", "http://airflow.test")
AssertAirflowDagExists("name_of_dag", "http://airflow.test")
AirflowRunDag("name_of_dag", "http://airflow.test")
AssertAirflowDagHasSucceeded("name_of_dag", "http://airflow.test")
}
Right now the Terratests are a bit of a mess so I would like to refactor them by creating some basic methods that you can easily chain together to test the full setup.
The methods I suggest right now are:
TerraformApplyAndDestroy
: input testing.t and terraform.options; calls tf init, tf plan, tf, apply and defers tf destroyAssertContainersAreRunning
: asserts if all containers in an ecs cluster are in the running statusAssertAirflowWebserverIsReachable
: Checks if you can access the airflow uiAirflowLogin
: logs in to the airflow uiAssertAirflowLoggedIn
: check if you are logged in to airflowAirflowAddDag
: uploads a new dag to s3AssertAirflowDagExists
: checks if a dag is visible in the UIAirflowRunDag
: runs a dagAssertAirflowDagHasSucceeded
: checks if a certain dag has successfully run via the UIAll these methods (except for
TerraformApplyAndDestroy
) make sure that airflow is deployed in a usable state and that you for example didn't mess up something in the configuration of airflow.An example test would look like this (pseudo code):