search

terraform-google-modules / terraform-google-startup-scripts

Provides a library of useful startup scripts to embed in VMs
https://registry.terraform.io/modules/terraform-google-modules/startup-scripts/google
Apache License 2.0
73 stars 36 forks source link

Add stdlib::init_gsutil_crcmod_el #11

Closed jeffmccune closed 5 years ago

jeffmccune commented 5 years ago

Without this patch startup-scripts needing to pull resources from GCS may have difficulty because gsutil is a dependency.

This patch aims to provide the gsutil dependency for Enterprise Linux operating systems. EL6 and EL7 have been tested.

jeffmccune commented 5 years ago

Putting this up for comment. I'm not sure we want to include this. It was very helpful in the field, but the use case needed only EL6 and EL7 systems. This also starts down a slippery slope of platform support. What about Debian and Ubuntu?

Ultimately, it seems like gsutil may be better off managed and installed in the image build phase, not the startup-script phase.

Leaving this PR here for posterity if only to copy and paste elsewhere. My suggestion is we close it without merging it.

morgante commented 5 years ago

Is it possible to make these scripts more pluggable? It'd be preferable if I could mix and match which scripts I want included.

One simple way to do that might be to make the library into a template file?

jeffmccune commented 5 years ago

Is it possible to make these scripts more pluggable? It'd be preferable if I could mix and match which scripts I want included.

One simple way to do that might be to make the library into a template file?

I like this idea. It bumps up against some design considerations I had in my mind but never wrote down:

The simple use case should remain as simple as possible. Currently, this is a simple matter of passing the file content to the startup-script metadata attribute in Terraform. Converting the main script to a template would increase complexity, so should be avoided if there are good alternatives.

@morgante What if you could set STDLIB_PLUGINS=foo,bar,baz using the config loading behavior in #10? If set, the expected behavior is instance/attributes/startup-script-foo, instance/attributes/startup-script-bar and instance/attributes/startup-script-baz are all sourced into the bash context prior to sourcing instance/attributes/startup-script-custom?

I think it makes sense to deal with static assets like scripts in images rather than downloading them at each startup.

@aaron-lane Another design principle of this Terraform module is that scripts end up not being static. Startup scripts tend to evolve and change frequently over the course of a cloud migration project. In an effort to minimize the operational impact of updating a startup script, they should exist outside of the image itself. Rolling an image tends to be a disruptive process in many situations, whereas a startup script may be re-executed at runtime without bouncing an instance.

My comment about baking the behavior into the image was specifically about gsutil. I think it makes sense to bake software dependencies into the image, especially software which isn't likely to be updated frequently.

If we need to move the startup-scripts behavior out of the metadata service, I think the next best alternative is a GCS bucket. This would preserve the design goal of updating the startup scripts and re-executing them from a running instance.

jeffmccune commented 5 years ago

Will close this PR once we close on the design discussions.

morgante commented 5 years ago

The simple use case should remain as simple as possible. Currently, this is a simple matter of passing the file content to the startup-script metadata attribute in Terraform. Converting the main script to a template would increase complexity, so should be avoided if there are good alternatives.

I don't think making the main script into a template substantially increases complexity.

To be clear, I'm referring to a Terraform template. The templating would be entirely abstracted for module users so they still just pass the returned file content to the startup-script metadata attribute. We can just start to introduce variables which modify that returned file content.

jeffmccune commented 5 years ago

@morgante Thanks for taking the time to clarify.

I don't think making the main script into a template substantially increases complexity.

I was thinking about this as the user having to manage the template and variable binding. Now I understand you're meaning to preserve the "API" of the content output as-is.

I really like the idea of handling this internally and will explore it in a separate ticket.