Cache TerraformTest methods

[marshall7m]

The changes introduced incorporate PyTest fixtures that cache tftest method outputs. The outputs can be used for future pytest session invocation without users having to rerun potentially time-intensive Terraform commands. The fixtures originated from a repo I created called pytest-terra-fixt although I thought that this may be useful to add to the tftest package.

Fixtures

All fixtures below can be parametrized via (@pytest.mark.parametrize())

terra:

• Scope: Session
• Parameters [Required]:
  • binary: Path to binary (must end with terraform or `terragrunt)
  • tfdir: Absolute or relative path to basedir
  • basedir: Base directory for tfdir (defaults to cwd)
  • env: Environment variables to pass to the command
• Returns: Either the tftest.TerragruntTest or tftest.TerraformTest class depending on the binary parameter

terra_setup

• Scope: Session
• Parameters [Optional]: A: Dictionary of keyword arguments to passed to the tftest.TerraformTest.setup() method B: Dictionary of terra directories and their respective keyword arguments to pass to the method mentioned above
• Returns: The terra fixture's associated setup output

terra_plan

• Scope: Session
• Parameters [Optional]: A: Dictionary of keyword arguments to passed to the tftest.TerraformTest.plan() method B: Dictionary of terra directories and their respective keyword arguments to pass to the method mentioned above
• Returns: The terra fixture's associated plan output

terra_apply

• Scope: Session
• Parameters [Optional]: A: Dictionary of keyword arguments to passed to the tftest.TerraformTest.apply() method B: Dictionary of terra directories and their respective keyword arguments to pass to the method mentioned above
• Returns: The terra fixture's associated apply output

terra_output

• Scope: Session
• Parameters [Optional]: A: Dictionary of keyword arguments to passed to the tftest.TerraformTest.output() method B: Dictionary of terra directories and their respective keyword arguments to pass to the method mentioned above
• Returns: The terra fixture's associated output

Examples

conftest.py

import pytest

def pytest_generate_tests(metafunc):
    if "terra" in metafunc.fixturenames:
        metafunc.parametrize(
            "terra",
            [
                pytest.param(
                    {
                        "binary": "terraform",
                        "env": {
                           "TF_VAR_foo": "bar"
                        },
                        "tfdir": "fixtures",
                    },
                )
            ],
            indirect=True,
            scope="session",
        )

test_tf.py

import pytest

@pytest.mark.usefixtures("terra_setup", "terra_apply")
class TestModule:

   @pytest.mark.parametrize("terra_plan", [{"extra_files": ["plan.tfvars"]}])
   def test_plan(self, terra, terra_plan):
      assert terra_plan["bar"] == "zoo"

   def test_out(self, terra, terra_output):
      assert terra_output["doo"] == "foo"

Use Cases

The following are short and long versions of the possible use case

Brief use case

• Faster setup time for testing terraform modules that don't change between testing invocations
• Give users the confidence to write tests without worrying about errors within their test code causing them to have to wait for their Terraform setup logic to run every time

Long use case Terraform modules can take a long time to spin up. If you're developing tests locally for your modules and the teardown logic or tftest setup method includes the cleanup_on_exit=True argument, then subsequent test invocations will have to run the entire Terraform deployment flow again in order for your tests to run. This may be ideal if the Terraform module source code changes in which you want the refreshed outputs to be used for testing assertions. Although if you want a fast feedback loop for writing assertions and the module code hasn't changed, then the tftest method fixtures may come in handy given its caching abilities. An alternative is just to comment out any teardown logic or set cleanup_on_exit=False, although tftest will still have to call the requested Terraform command even though its associated output hasn't changed. This process is still slower compared to using the local caching feature.

Notes

I'm more than happy to change any fixture names or general styling/formatting to be more coherent with the tftest package.

[ludoo]

Hey Marshall, thanks for this. Could you briefly describe a concrete use case? If we add this, we should add some text to the README explaining typical usage.

[marshall7m]

Hey Marshall, thanks for this. Could you briefly describe a concrete use case? If we add this, we should add some text to the README explaining typical usage.

Hi @ludoo, thanks for the recommendation. I made the appropriate changes to the PR description under the Use Cases section

[ludoo]

Marshall, sorry for the delay on this. Would you mind adding a section to the top-level README that explains the fixtures, and give a quick usage example? Then this is good to go, even though I'm still struggling to understand it in full (my problem, not yours). :)

[marshall7m]

Marshall, sorry for the delay on this. Would you mind adding a section to the top-level README that explains the fixtures, and give a quick usage example? Then this is good to go, even though I'm still struggling to understand it in full (my problem, not yours). :)

@ludoo no worries, I'm glad it wasn't merged given it gave me the time to figure out how to simplify and improve the PR. I made some big changes including removing the fixtures entirely and adding the caching feature internally to the TerraformTest class.

This removes the need to use the fixtures in order to benefit from the caching feature. I also realized that the fixtures are pretty general and it's likely that users are going to want to customize the logic within the tftest fixtures anyways.

I added a description and a usage snippet to the README. Let me know if there's anything else I should add.

The idea is kind of confusing but I'm hoping the changes I just introduced should simplify it. Is there a particular aspect of it that you need more clarification about? I'd be happy to answer any questions you may have.

[ludoo]

I can never figure out who can actually see the merge button, if you cannot just let me know.