Scaffold Provider (just like swagger codegen / go-swagger)

[m-v-k]

First of all big kudos on this provider! It's a great addition to the Terraform community.

Have you considered extending this tool, so that rather using it directly as a provider, it actually creates (scaffold) a custom terraform provider?

We have a microservice architecture with lots of Management Api's that we use internally to manage all our (customer) resources with Terraform/GitOps.

Currently we use go-swagger to generate the clients, and then put these inside mastercard's terraform rest provider. It's that last part that makes for a lot of boilerplate code and hours spend.

Your solution seems to take away that boilerplate, but i'm a bit hasitant to use it, because;

• The remote swagger definition is the source of truth, which is outside of terraform (state) control. So identical code and/or tf state now no longer results in identical result. (not idempotent).
• Missing intellisense and overal developer experience.
• (Maybe i am wrong here?) i can't do terraform validate
• In Terraform 0.13, custom providers have had major updates. They can now be registered in (public) registries, and have to be initialized explicitly. This is in conflict with a dynamic provider.

Please correct me if i am wrong about these points, i'm only human ;).

And i would very much like to know what others think about a custom provider code generator.

[dikhan]

Hi @m-v-k ,

Thanks for the feedback! That's a great idea and something that I thought of too when creating this provider. This provider was originally created to fulfill mainly the following use case:

As a service provider
I want to have a terraform plugin that uses my API's OpenAPI document (swagger file) as the source of truth
So that I can spend my time developing/evolving the service and not have to deal with supporting integrations with the terraform plugin any time I release a new version of my API.

While I'd find the terraform generator (scaffold) a pretty handy tool and a great addition to the swagger tooling family, it would be hard to extend this provider to follow that model and in my opinion it would be better to have that solution in a different repo considering the nature of what it's optimizing for. Having said that, I have some ideas on how this repo could be extended to support Terraform 0.13 Automatic Installation of Third-Party Providers like:

• Enabling a variable to be set with the OpenAPI URL at compile time so the binary produced would no longer require the OpenAPI URL as input since it'd be already baked in and could be shipped independently pointing at the desired API.
• Using this repo as a framework/library that other custom providers could import (like how it's done in the main.go) and have their own lifecycle.

The above combined with the ability to automatically render the Terraform docs for the custom provider using the OpenAPI Terraform docs auto-generator tool could be a way to go. I am still brainstorming on this and would be happy to hear more ideas that you may have.

Please see below my two cents from your observations:

The remote swagger definition is the source of truth, which is outside of terraform (state) control. So identical code and/or tf state now no longer results in identical result. (not idempotent).

I would like to understand more about this point, would you mind sharing fleshing that out (perhaps with an example)? Terraform performs a refresh command before applying any operation (create, update, etc) which is used to reconcile the state Terraform knows about (via its state file) with the real-world infrastructure. This ensures that if the API has introduced anything new that the current state is not aware of (eg: new property added to a resource model), the state will be updated with the updates. This should be transparent for the user though and should not get any diffs. This is a similar behaviour for terraform providers that are released along with API releases (eg: aws terraform provider) with the difference that those require a new release of the plugin for the users to benefit from the new things offered by the API.

Missing intellisense and overal developer experience.

As far as custom terraform providers is concerned, I am not very familiar with what kind of support there is for intellisense. I'd be curious to learn more about it though if you could share more details on it.

(Maybe i am wrong here?) i can't do terraform validate

What error are you getting? I just tried with one of the examples and seemed to work fine:

➜  terraform-provider-openapi git:(master) ✗ PROVIDER_NAME="goa" make install
[INFO] Building terraform-provider-openapi binary
[INFO] Creating /Users/.../.terraform.d/plugins if it does not exist
[INFO] Installing terraform-provider-goa binary in -> /Users/.../.terraform.d/plugins/terraform-provider-goa
➜  swaggercodegen git:(master) ✗ pwd
/Users/.../src/github.com/terraform-provider-openapi/examples/swaggercodegen
➜  goa git:(master) ✗ terraform validate
Success! The configuration is valid.

In Terraform 0.13, custom providers have had major updates. They can now be registered in (public) registries, and have to be initialized explicitely. This is in conflict with a dynamic provider.

Shared my thoughts on this above.

Hope this answered your questions or at least some of them, looking forward to hearing your thoughts.

Thanks, Dani

[m-v-k]

Thank you very much for the elaborate answer!

• your suggestions for making 'tf 0.13 registry ready' are good, i mean baking the url in, will solve some pain points i described. using the provider as an import will work also, but will still make me write more code than i want (yes, i am really lazy! hehe).
• I don't know about auto generating provider docs, i do this only for modules at the moment terraform-docs
• for the 'source of truth': my concern is exectly what you describe with aws provider example. but maybe my conclusion is different; I consider all provider behavior changes a risk (within the same version). some minor api changes might not brake a terraform project, but it's 'fingers crossed'. major changes would definitely break it, of course a api publisher could version it's api, but i can't always control that. So what aws does in your example would exaclty be what i suggest, scaffold the provider with everything baked in. I can then choose to ignore api/provider updates, keeping my projects stable, and choose when i want to upgrade/test new versions. i can control that.
• intellisense: i use the Hashcorp Terraform extension in vscode, you need to init first, and maybe configure some module rootfolders (it's in the docs). works very nicely.
• terraform validate is not working : hmmm, i think something wrong on my side indeed, don't have time to investigate now but let's assume it's my bad....

Anyway, thanks again for your response. I will 'park' this idea for now and keep an eye open to the tf community and this great project. If i have more information or see options to contribute i'll be back ;)