Project still active?

[luke-vidler]

We have considerable interest in this project, is it still being maintained? Specifically we are trying to Unit Test (using Kitchen-Terraform and Jenkins) TF on an OpenStack provider.

[sworisbreathing]

I've recently stumbled across this project as well, and wondering the same thing. The gem seems to have disappeared from rubygems, so the instructions on the readme are no longer valid. So I'm guessing not.

[bsnape]

Hey @luke-vidler and @sworisbreathing, apologies for the slow response!

This is something I hacked about with a few years ago. Since then it's seemed to have got a lot of interest despite not working on it!

I'm still using Terraform on an almost daily basis so this project is definitely still of interest to me.

Looking back, I wasn't sure about the direction of this which is why it gathered dust. In essence it is a glorified console scraper which leaves it very susceptible to minor changes in the Terraform console output. Of course that can be overcome but it felt a little brittle.

More significantly, I'm not certain of the testing role rspec-terraform should play. I originally thought it could be more about unit tests / sanity checks but I don't think that gives you much over and above a plan. Terraform has improved a lot since I started this and logical errors are more often caught at the plan phase instead of the apply phase. It would be wrong to place extra logic inside rspec-terraform that Terraform itself should be validating.

I think the only remaining option is to physically spin up real infrastructure and verify your expected configuration against it using tests (maybe some autogenerated ones would be nice too). I think this is fairly straightforward in theory (but not necessarily in practice). I've used vagrant, docker, rspec-puppet, test-kitchen, molecule etc. extensively so it feels a little strange to not be able to spin things up locally. I suppose that's just something we have to deal with.

In conclusion I might start looking at this again over the quiet Christmas period... but it would be nice to get your thoughts on the above.

[sworisbreathing]

hi @bsnape

Thanks for the response. I found rspec-terraform from this blog post. My plan was to use it as a unit testing tool, as opposed to an integration testing tool (think chefspec vs test-kitchen/inspec/serverspec). Personally I still see some value in sanity-checking the plan output in an automated way. terraform plan can tell you if you have errors in your templates but there's still a gap between pure validity checking/linting and testing against provisioned infrastructure, which this library could fill.

For example, you want to ensure that the IP ranges used for ingress/egress rules are configured through input variables, so you can reliably deploy to dev/test/prod environments that may have different configurations. Ideally you wouldn't yet be making any changes to the infrastructure, but your test execution could:

1. run terraform plan without any arguments (except for overriding the state backend to use a local, empty, test-specific state backend to ensure you don't clobber remote state)
2. run terraform plan again but override the input variables for the IP ranges (again, empty/local/test-scoped backend)
3. compare the results to see if the new IP ranges you've supplied are in the plan.

That's the sort of use case I'm looking for something like rspec-terraform to fit.

As for the brittleness of terraform plan output between versions - it's a known issue, and the official word is that hashicorp wants to do it, but they're not going to commit to stabilizing it just yet, because they feel the plan structure isn't sufficiently stable yet. So unfortunately that does mean more work for maintaining rspec-terraform, in terms of keeping up to date with changes the plan output format.

There is a node package available for dumping the terraform plan to json (obviously the developer has to keep this up to date with new terraform versions that change the plan output) but I'm not aware of an equivalent ruby gem (at least, not one that's actively maintained).

[blijblijblij]

Hi @bsnape, just stumbled across this project today while figuring out how to to make sure CI is capturing more obvious issues on terraform plans, I also would be a very happy camper if rspec-terraform received some tlc.

[bitdivine]

Regarding parsing plans, I have had much joy with tfjson that parses the plan directly, rather than the stringified output. For the latest version of terraform some tweaks are needed but they are documented in the corresponding issue. https://github.com/palantir/tfjson

[luke-vidler]

Looking at the same application as @sworisbreathing Unit Testing the resource blocks, and then running the tests in CI/CD. It would effectively give us TDD based TF projects, we have a lot of code and it is unstable. Unit Testing the resources would at least make refactoring safer and faster. Kitchen Terraform is a good framework for TDD style TF.

[js-timbirkett]

😢