search

GalleyBytes / terraform-operator

A Kubernetes CRD to handle terraform operations
http://tf.galleybytes.com
Apache License 2.0
364 stars 47 forks source link

Sources are ambiguous and hard to understand #63

Closed isaaguilar closed 2 years ago

isaaguilar commented 2 years ago

The spec.sources[] array is hard to understand. Is is used to download tfvars? Or templates? Or other .tf files that get run as part of my module?

Rhetorical question... but technically it can be any of those and it is hard to understand the behaviour there. What will probably change soon is that source extras will be changed to options, and the options that can be used (that I can think of) is type which can be either tfvars or tpl.

So a source may look like:

...
spec:
  sources:
  - address: https://github.com/isaaguilar/private-config//simple-aws-modules/test.tfvars
    type: tfvars

Another option may be path, which would be defined as the relative path from the main module to download the file into. 

...
spec:
  sources:
  - address: https://github.com/isaaguilar/private-config//simple-aws-modules/test.json.tpl
    type: tpl
    path: templates/test.json.tpl # or maybe just templates/ to keep the same name of the file

By making these changes, sources becomes less ambiguous and easier to reason about in their execution when the terraform runners get executed.

Also, sources is kind of an odd name for downloaded resources. Another name perhaps?

isaaguilar commented 2 years ago

Along the lines of downloading resources to be used by the terraform runners, the TFO controller should be relieved of the task of doing the downloads. The onus of downloads should be put on the setup runner. This does multiple things for the project:

  1. Speeds up seeing the first pod get launched because TFO isn't waiting to try the download
  2. Users have a single place to look if their workflow didn't get set up correctly. User's no longer have to check TFO controller logs for download related tasks.

One downside losing the ability to do a SOCKs5 proxy which the TFO Controller used to fetch http(s) under sources. This just means research into how to accomplish this via sshtunnels or other proxy methods. Typically, getting terraform modules to accept proxies has been a pain point.

The benefits of User experience outweighs the downsides of firewall workarounds. Proxy needs more research nonetheless.