scottwinkler / terraform-provider-shell

Terraform provider for executing shell commands and saving output to state file
Mozilla Public License 2.0
279 stars 61 forks source link

Option to show stdout or stderr in Terraform output #17

Closed jemc closed 4 years ago

jemc commented 4 years ago

As part of my lifecycle scripts I'd like to have the ability to print messages that will actually be shown to the user running Terraform.

For example, when running kubectl apply on a set of manifests I'd like to be able to show the individual resources getting created / being unchanged / etc.

Currently, I can inspect the output of the lifecycle commands using TF_LOG=debug, but that also prints a bunch of other verbose messages from Terraform and from the plugin that I dont' want to see.

I can understand that some people want the output of their commands to be hidden when not in debug mode, but I'd appreciate a way to present curated output. Maybe the provider could be modified to always print the contents of ether stdout or stderr (while hiding the other)? Or if you want to keep backwards compatibility, maybe you could open a new file descriptor for writing to the Terraform output (4?)


Somewhat separately, but somewhat related (let me know if you'd prefer to discuss in a different ticket), when a lifecycle command fails, I only see the nonzero exit status printed from terraform, but it would be nice to also see further information for troubleshooting purposes, like the stdout/stderr of the program, and possibly also the state it received as input.

scottwinkler commented 4 years ago

I usually use TF_LOG=debug to be able to get error messages and this is consistent with the behavior of all other Terraform providers. I understand, however, that this provider is unique in that in lets you incorporate custom shell scripts. I think it may be useful to add a configuration variable to the provider to output messages regardless. I will consider adding this as a feature, and leave this ticket open until then. In the meantime, if you want output values without having debug on, then simply return it as part of the json payload and set it as a Terraform output value.

For the nonzero status value, I am not sure why you would be experiencing that problem. I believe I fixed that in a previous release. In the utility.go file, here is the code that runs the shell script and outputs results:

    log.Printf("[DEBUG] shell script command old state: \"%v\"", state)

    // Output what we're about to run
    log.Printf("[DEBUG] shell script going to execute: %s %s \"%s\"", shell, flag, command)

    // Run the command to completion
    err = cmd.Run()
    pw.Close()
    log.Printf("[DEBUG] Command execution completed. Reading from output pipe: >&3")

    //read back diff output from pipe
    buffer := new(bytes.Buffer)
    for {
        tmpdata := make([]byte, maxBufSize)
        bytecount, _ := pr.Read(tmpdata)
        if bytecount == 0 {
            break
        }
        buffer.Write(tmpdata)
    }
    log.Printf("[DEBUG] shell script command stdout: \"%s\"", stdout.String())
    log.Printf("[DEBUG] shell script command stderr: \"%s\"", stderr.String())
    log.Printf("[DEBUG] shell script command output: \"%s\"", buffer.String())

    if err != nil {
        return nil, fmt.Errorf("Error running command: '%v'", err)
    }

as you can see, it not only outputs the previous state, but the commands it is about to run, and the stdout, stderr and (3) output. Only after outputting these does it check if there was an error and then fail. So you should be getting output messages if you have debug on.

adudek commented 4 years ago

Another debug workaround that i found useful is to update lifecycle command with stderror redirect for example: create = "create.sh 2>dbg.txt >&3"

works every time.

jemc commented 4 years ago

So you should be getting output messages if you have debug on.

Yes, in either case I get output messages if I have debug on.

I just meant that it would be nice to be able to print a more specific reason for the failure, even without debug on, which ties it into the earlier argument.

ppff commented 4 years ago

+1

This is an important feature imo. The behavior should be akin to local-exec by transferring the script's stdout and stderr to terraform's stdout. See here: https://github.com/hashicorp/terraform/blob/d4ac68423c4998279f33404db46809d27a5c2362/builtin/provisioners/local-exec/resource_provisioner.go#L50

I will propose a PR if I can find the time for it.

jemc commented 4 years ago

Yeah, I guess that's why I found the curent behaviour surprising - local-exec prints the output, so I was expecting this provider to do that too.

scottwinkler commented 4 years ago

@ppff if you want to make a PR, have at it. I am pretty busy right now so I otherwise wouldn't get around to it for a while.

lukasmrtvy commented 4 years ago

@ppff Will You create a PR ?

ppff commented 4 years ago

I have cloned the project and plan on working on the following issues https://github.com/ppff/terraform-provider-shell/issues

I just don't know when exactly.

scottwinkler commented 4 years ago

closing issue due to pr #27 . wasn't able to get the plugin to return logs at different levels. I think Terraform has deprecated the use of any log level other than DEBUG for providers. If someone finds a way around this I would be happy to know