Allow resources to report more detailed progress from CRUD

[radeksimko]

This was mostly triggered by https://github.com/terraform-providers/terraform-provider-aws/issues/3236 but I pondered with this idea for a long time in some other contexts.

Many resources, especially in AWS involve some kind of retry mechanism, for mainly two reasons:

1. eventual consistency (retry on error)
2. asynchronous API for stateful resource (e.g. RDS)

and it's not immediately obvious to the common user that we perform such retries - or it's not obvious what's happening behind the scenes.

Current state

We keep the UI very brief (probably intentionally, at least by default) whilst we retry:

aws_dynamodb_table.example: Still creating... (10s elapsed)
aws_dynamodb_table.example: Still creating... (20s elapsed)
aws_dynamodb_table.example: Still creating... (30s elapsed)

Proposal

Optionally we could expose some details about the progress.

State

(typically from resource.StateConf)

aws_db_instance.example: Still creating... (10s elapsed)
aws_db_instance.example: Current status: creating
aws_db_instance.example: Still creating... (20s elapsed)
aws_db_instance.example: Still creating... (30s elapsed)
aws_db_instance.example: Current status: backing-up

Event log

e.g. CloudFormation Stack

aws_cloudformation_stack.example: Still creating... (10s elapsed)
aws_cloudformation_stack.example: CREATE_IN_PROGRESS: SiteBucket (AWS::S3::Bucket)
aws_cloudformation_stack.example: CREATE_COMPLETE: SiteBucket (AWS::S3::Bucket)
aws_cloudformation_stack.example: Still creating... (20s elapsed)
aws_cloudformation_stack.example: Still creating... (30s elapsed)
aws_cloudformation_stack.example: UPDATE_IN_PROGRESS: MyCertificate AWS::CertificateManager::Certificate

Error retries

e.g. Lambda

aws_lambda_function.example: Still creating... (10s elapsed)
aws_lambda_function.example: Retrying on InvalidParameterValueException: The provided execution role does not have permissions to call SendMessage on SQS
aws_lambda_function.example: Still creating... (20s elapsed)
aws_lambda_function.example: Retrying on InvalidParameterValueException: The provided execution role does not have permissions to call SendMessage on SQS
aws_lambda_function.example: Still creating... (30s elapsed)

---

This would be certainly worth a full-blown RFC, but I'm creating this issue so we can link to it from provider repos.

[sheerun]

It would be sweet. I could use it for my provider for pulling docker images (it would output a progress for each pulled layer). I'd just skip "Still creating..." if resource tells it can handle progress by itself.

[pcj]

Any update on this? New terraform user here, but if I wanted to write a Web-UI that reported progress events about what terraform is doing, how would I approach that, forking terraform itself? I don't see a progress API. Thx

[apparentlymart]

Hi @pcj! We've not been able to start any further detailed design work on this yet since we've been focused on the configuration language improvements for v0.12, but we do intend to resolve this in a later release when we can shift our focus to changes in the command line interface.

With that said, I think you're asking for something slightly different than what this issue was originally talking about. Radek was talking about giving providers themselves a way to mark progress during long-running operations, while I think you're asking about a similar thing at the interface between Terraform Core and the program that is running it. Both of these are likely to be solved separately but they are related in that any progress milestones reported from the provider to core would need to be echoed out as one of many different events that Terraform Core itself would then report to its caller. Both of these things will happen, but are likely to happen separately because they are in different subsystems.

We'll post some more notes here once we have some updates on the first part of the problem. I don't think we have an open issue for the "machine-readable output from Terraform Core" thing, but from our end we're considering that part of some general reorganization of how backends work that is planned for a future release, since the current internal API for backends is what is blocking this from being possible today. We'll open one or more issues about this once we've decomposed the problem a little more and can write a more reasonable description of specific tasks to be done.

[Didainius]

Would be interested to see this in action as I think it can bring value to users - plugin authors have the chance to give better hints and/or progress updates.

[vbauzys]

I vote for adding the ability to allow providers to push messages to stdout.

[apparentlymart]

We do not intend to allow providers to write directly to stdout, because that is incompatible with Terraform running in the machine-readable JSON mode. Instead, any output from providers will need to be designed with a suitable structure that it can be translated to both a terminal-based UI and to other UI contexts such as Terraform Cloud's web UI, by interpreting the JSON data and translating it into a suitable presentation for the context.

Ultimately the Terraform CLI layer must retain full control over what gets read from and written to the terminal, because many operations are happening concurrently and it's Terraform's responsibility to ensure that these concurrent operations can occur without interfering with one another and in a way that doesn't block future improvements to Terraform.

[justinmchase]

It would be nice to just accept a progress input which took in a file path which you'd then write progress too, then people could just pick how they want the progress to be displayed.

terraform apply -p /dev/stdout