Discussion: Make l0-setup pure terraform (or more terraform-like)

[zpatrick]

Open discussion:

Layer0 Setup is pretty bloated. If we were able to convert it to just be pure terraform files, it would remove a significant burden from us. However, I think there are some critical features we shouldn't remove from l0-setup - so perhaps a stripped-down version of l0-setup that only performed the critical features is a better option.

Significant problems with pure terraform l0-setup:

• Would no longer be able to support installing layer0 inside an existing vpc
• Would no longer be able to backup/restore terraform.tfstate, terraform.tfvars, and dockercfg files to/from S3. Terraform does have a remote state configuration option, but this only backup/restores terraform.tfstate.
• The endpoint command would no longer exist
• Managing different Layer0s would need to be a manual process (copy terraform files over to ~/layer0/instances, cd into that file to run any commands, etc)

Proposed Compromise: Stripped down version of l0-setup with overarching goal to be compatible with terraform: e.g. you don't ever need l0-setup to manage layer0 - it just makes it easier.

• Drop support for existing vpcs (no longer template on top of the terraform files)
• l0-setup backup/restore copies the entire directory at ~/layer0/instance/ to/from s3. This way, custom layer0 configuration get's synced properly (for example: users can add custom security groups into their Layer0).

Commands:

• init - copies terraform files over to ~/layer0/instances/, asks for required variables (splits up responsibility of apply)
• apply - stripped-down version of what it is today. Just runs terraform apply followed by a backup.
• backup/restore - copies the entire directory at ~/layer0/instance/ instead of just the current 3 files. This way, custom layer0 configuration get's synced properly (for example: users can add custom security groups into their Layer0).
• destroy - no change
• endpoint - no change

[diemonster]

I like everything except for dropping support for existing VPCs, mostly because I don't think we should force a VPC for every new Layer0 instance. That being said, if we give a clear path on how to install Layer0 into an existing VPC that requires terraform -var usage and/or editing a file directly, that seems like a decent tradeoff.

[zpatrick]

One thing to note - we could also just manage/release 2 different terraform modules - one for layer0 inside an existing vpc and one for creating the vpc

[diemonster]

More issues that would support revisiting l0-setup in general: https://github.com/quintilesims/layer0/issues/135 https://github.com/quintilesims/layer0/issues/133 https://github.com/quintilesims/layer0/issues/130

[tribaljack]

Losing the --vpc isn't a big deal, i don't think, it'd be best if we can provide a VPC peering, we can end up with consistent networking inside l0 installs that can talk to each other, best of both worlds?

[diemonster]

some nice improvements in terraform v0.9 that might make this more appealing: https://www.hashicorp.com/blog/terraform-0-9/#state-locking

specifically 'state locking' and interruptible providers

[zpatrick]

VPC could use conditionals: https://www.hashicorp.com/blog/terraform-0-8/#conditional-values

[ignatzami]

Honestly, I'd consider switching to CFTs. It reduces the number of tools that need to be supported, and would be a huge benefit to people learning the system via the console.

Add to this the fact that AWS provides CFTs for nearly everything out of the box I don't see a need, or benefit, to reinventing the wheel.

[tlake]

While we're listing out specific actionable items, the CLI help for the -s flag is confusing. The current output of

-s, --syntax="bash"    Show commands using the specified syntax (bash, powershell, cmd)

makes the user suspect that a naked -s flag automatically defaults to --syntax="bash", when in reality, you need to specify the shell in all cases.

Something like

-s="SYNTAX", --syntax="SYNTAX"    Show commands using the specified SYNTAX (bash, powershell, cmd)

might be more helpful.

[diemonster]

I think we discussed adding validation into the new l0-setup, but if not, here's a reference: https://github.com/quintilesims/layer0/issues/228