search

statuspal / terraform-provider-statuspal

Mozilla Public License 2.0
3 stars 3 forks source link

Run acceptance test against real infrastructure #26

Open henrybarreto opened 4 days ago

henrybarreto commented 4 days ago

Terraform's recommends running acceptance tests against real infrastructure, but you are running into mock server.

What are your thoughts on implementing this change?

FabianDaniel00 commented 4 days ago

That's strange for me because we never test against the real infrastructure or server. For example we need to create an account just for testing. What happens if it fails to delete the created resource? We have the API in a separate project, if we change that the tests will fail until we deploy the API changes to production.

Let me know if I'm wrong.

henrybarreto commented 4 days ago

That's strange for me because we never test against the real infrastructure or server. For example we need to create an account just for testing. What happens if it fails to delete the created resource? We have the API in a separate project, if we change that the tests will fail until we deploy the API changes to production.

Let me know if I'm wrong.

You aren't wrong.

As you know, there are many types of tests: unit, integration, acceptance, etc., each one with their purpose.

If I'm not mistaken, Terraform recommends acceptance tests to run against real structure to validate the provider in the real world, but it doesn't exclude unit tests, that are mind to run locally.

https://developer.hashicorp.com/terraform/plugin/testing/acceptance-tests/testcase#testcase-reference-api

IsUnitTest allows a test to run regardless of the TF_ACC environment variable. This should be used with care - only for fast tests on local resources (e.g. remote state with a local backend) but can be used to increase confidence in correct operation of Terraform without waiting for a full acceptance test run.

I purpose we run the TestAcc on the real infra, specially on GitHub actions, and unit test locally, as you do now.

henrybarreto commented 4 days ago

What happens if it fails to delete the created resource?

In my view, this is a total normal situation in the development cycle, and we should be able to delete it manually if something went wrong.