jason-johnson / azure-pipelines-tasks-terraform

Azure Pipelines extension for Terraform
MIT License
122 stars 51 forks source link

Terraform command 'plan' failed with exit code 'null' #412

Closed tlfzhylj closed 4 months ago

tlfzhylj commented 5 months ago

Describe the bug In one of my pipelines I get an error during terraform plan saying: ##[error]Terraform command 'plan' failed with exit code 'null'. ##[error]TypeError: Cannot read property 'toString' of null

To Reproduce Can't understand what is actual does this, so I can't reproduce in another repo.

Expected behavior Terraform plan goes through without any errors.

Pipeline Logs

Logs with debug mode:

##[debug]Exit code null received from tool '/azp/_work/_tool/terraform/1.7.3/x64/terraform'
##[debug]STDIO streams have closed for tool '/azp/_work/_tool/terraform/1.7.3/x64/terraform'
##[debug]allowTelemetryCollection=false
##[error]Terraform command 'plan' failed with exit code 'null'.
##[debug]Processed: ##vso[task.issue type=error;]Terraform command 'plan' failed with exit code 'null'.
##[debug]set TERRAFORM_PLAN_HAS_CHANGES=false
##[debug]Processed: ##vso[task.setvariable variable=TERRAFORM_PLAN_HAS_CHANGES;isOutput=false;issecret=false;]false
##[debug]allowTelemetryCollection=false
##[debug]command=plan
##[debug]command=plan
##[debug]secureVarsFile=undefined
##[debug]command=plan
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]command=plan
##[debug]environmentServiceName=***
##[debug]command=plan
##[debug]providerAzureRmSubscriptionId=undefined
##[debug]command=plan
##[debug]environmentServiceName=***
##[debug]runAzLogin=false
##[debug]command=plan
##[debug]publishPlanResults=undefined
##[debug]command=plan
##[debug]command=plan
##[debug]command=plan
##[debug]command=plan
##[debug]command=plan
##[debug]command=plan
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local
##[debug]backendType=local

Agent Configuration

Additional context

The Azure pipeline task is looking like this.

          - task: TerraformCLI@1
            displayName: "Run > terraform plan"
            inputs:
              command: "plan"
              environmentServiceName: "$(service_connection_name)"
              #publishPlanResults: "PlanResults"
              commandOptions: "-out=$(Build.ArtifactStagingDirectory)/$(Build.Buildnumber).tfplan -var azuredevops_personal_access_token=$(azuredevops_personal_access_token)"
              workingDirectory: $(working_directory)
              allowTelemetryCollection: false

I tried to comment out the publishPlanResults option, and the pipeline went through as expected. At next run I get the same error again.

tlfzhylj commented 5 months ago

When upgrading to azurerm 3.90 from 3.50 it looks like it actual gives me an error I can read, and not just throw exit code null. The error message says: The plugin.(*GRPCProvider).ConfigureProvider request was cancelled

See issue mentioned above.

tlfzhylj commented 4 months ago

I experienced the same error in another pipeline which had an older Terraform version.

To get the error message The plugin.(*GRPCProvider).ConfigureProvider request was cancelled I also had to update Terraform version from 1.5 to 1.7.

tlfzhylj commented 4 months ago

I figured out the actual problem:

In my Terraform configuration I configure the azurerm provider several times to go against several Azure subscriptions. If I configured the azurerm provider six times against six different Azure subscriptions or more I got the error.

When configuring the provider so many times it looks like it uses more memory than on a normal run with one configured azurerm provider. My self-hosted Azure DevOps agent had 1 GB of memory. When increasing the memory to 2 GB everything works as expected.

So, the problem was actually that the Azure DevOps Agent ran out of memory.

I will probably hit the same error if configuring around 12 azurerm providers, after increasing the memory to 2 GB.

jason-johnson commented 4 months ago

I’m glad you solved it. Thanks for letting us know.