support templatefile function (by way of content field) in shell provisioner

StephenWithPH commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request. If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

It would be useful to hand a script to the shell provisioner whose contents were rendered by the templatefile function. This is more elegant than composing the file provisioner (to render the file and place it on the target machine) and the shell provisioner (to run the emplaced file) or coercing substitution via environment variables with the shell provisioner.

Note: this is heavily inspired by the file provisioner's content field.

Potential configuration

  provisioner "shell" {
    execute_command = " echo '${var.user_password}' | sudo --stdin '{{ .Path }}'"
    content = templatefile("${path.root}/some-script.sh.tpl", {
      SOME_VAR = var.some_var,
    })
  }

azr commented 2 years ago

Hello there, thanks for opening, this should already work using the inline field of the shell provisioner (it's an array of strings):

provisioner "shell" {
    inline = [templatefile("${path.root}/some-script.sh.tpl", {
      SOME_VAR = var.some_var,
    }),
    "echo hello there"]
}

Closing this one, let us know if something's up ! 🙂

StephenWithPH commented 2 years ago

@azr thanks!

I was thrown off by the line too long error that comes out when I mistakenly fed the output of templatefile to the shell provisioner's script field. I didn't consider the inline field because it didn't seem to fit my use case (described below).

I think there's still a little bit of grossness in the ergonomics of:

provisioner "shell" {
    inline = [templatefile("${path.root}/some-script.sh.tpl", {
      SOME_VAR = var.some_var,
    }),
    "echo hello there"]
}

In this use case, some-script.sh.tpl is presumably a fully-formed script file that might already have the shebang at the top (say, #!/usr/bin/env bash). Since it's a well-formed script, I can do things to it like running shellcheck, sourcing it, etc.

My bash script is breaking when Packer runs since inline defaults to adding a #!/bin/sh -e, so bashisms are breaking (d)ash on the target. I know I could coerce the outcome I need, but it starts to pull me away from a "normal-looking" shell script that just happens to get a little bit of template interpolation from Packer.

The development evolution flowed like so:

I need to run my boostrapping script inside the target:

provisioner "shell" {
execute_command = " echo '${var.user_password}' | sudo --stdin '{{ .Path }}'"
script          = "${path.root}/some-script.sh"
}

I need some variable from Packer's scope to be interpolated into the bootstrapping script:

provisioner "shell" {
execute_command = " echo '${var.user_password}' | sudo --stdin '{{ .Path }}'"
script = templatefile("${path.root}/some-script.sh.tpl", {
SOME_VAR = var.some_var,
})
}

I think there's utility in combining the "atomicity" of script with the light interpolation provided by templatefile in the context of the shell provisioner.

Having said all this, I'll defer to your judgment if you think this is some ergonomic sugar worth adding. Thanks for your work on a very useful tool!

azr commented 2 years ago

Oh, I see. The inline_shebang is important here, so that users can just write a short command. Because an inline command is written to a file, then executed. But with the introduction of template files this might no longer be the case. Since inline could also come from a file with a shebang.

If you want the inline shebang to come from your file I see a few good options here:

we modify the code so that when an inline command starts with #!, we don't add that shebang.
modify the code so that when inline_shebang is set to none we don't write that shebang. This sounds okay because presumably none is never going to be a valid inline_shebang.
add a new no_inline_shebang field options that will do the same thing.

My favorite option is the first one. Let me know what you think. Would you have the time to work on that one ? I think the modification would be here:

https://github.com/hashicorp/packer/blob/479a5ca813af1b6609ba5c69960785ae40c08b09/provisioner/shell/provisioner.go#L202-L204

Something like: if the characters of the first command are #!, skip adding that part.

StephenWithPH commented 2 years ago

Let me know what you think.

Assuming only additions, I think the ideal api would look like...

Exactly one of the following is required:

inline (array of strings) - This is an array of commands to execute. The commands are concatenated by newlines and turned into a single file, so they are all executed within the same context.
script (string) - The path to a script to upload and execute in the machine.
scripts (array of strings) - An array of scripts to execute. The scripts will be uploaded and executed in the order specified.
content (string) - This is the content to write to the machine and execute. It will be chmodded to 0755 and executed as-is. Any dependencies necessary on the target machine for this file to execute successfully must already be satisfied.

... so content would be a new way to provide something for the shell provisioner to execute.

It's implementation would look very similar to... https://github.com/hashicorp/packer/blob/cf121d899e14c50bbd4388a4bde8fd3b1abc4412/provisioner/file/provisioner.go#L136-L147

... but p.config.Scripts = append(p.config.Scripts, file.Name()) in https://github.com/hashicorp/packer/blob/cf121d899e14c50bbd4388a4bde8fd3b1abc4412/provisioner/shell/provisioner.go#L76

From there on, I believe this would behave just as if the user had specified script in that the file's path is known and its contents are complete.

Is this a palatable approach?

azr commented 2 years ago

The scrip.content option sounds excellent to me ! Well found name ! I also like the consistency with the file provisioner.

sohaib94 commented 4 months ago

Is this still an open issue? Would be happy to contribute to this if no one's working on it

hashicorp / packer