The job's timeout value should be set to null, allowing it to run longer than 10080 minutes as per the updated configuration.
Actual Behavior
The timeout value of the job remains set to the previous value of 10 years despite the terraform apply command.
Relevant Error/Panic Output Snippet
I tried changing the value of timeout to null from AWS Glue console, and then hit terraform plan with timeout parameter removed from the aws_glue_job terraform configuration. Then the plan produces the below changes -
resource "aws_glue_job" "streaming_extraction" {
for_each = var.tables
name = "streaming_extraction-${each.key}"
role_arn = data.aws_iam_role.glue_job_role.arn
command {
name = "gluestreaming"
python_version = "3"
script_location = "s3://${data.aws_s3_bucket_object.script.id}"
}
maintenance_window = "Sun:04"
#timeout = null # You can try by setting this to null as well
}
Steps to Reproduce
Create an aws_glue_job resource of streaming type, with a timeout value set to 10080. (5256000 as timeout may not work)
Attempt to update the aws_glue_job resource by removing the timeout parameter or set its value to null and adding a maintenance window.
Run terraform plan and observe that the timeout value change is not recognized.
Run terraform apply and observe that the timeout value remains unchanged.
Note:
If the timeout value is set to another non-zero value, then terraform plan does recognize the change.
Debug Output
No response
Panic Output
No response
Important Factoids
Initially, the jobs were created successfully with a timeout value set to 5256000 (10 years). The jobs were setup about an year ago.
When attempted to update the worker type of the job, the operation failed due to the timeout value being greater than 10080 minutes.
Error: error updating Glue Job (my_job_name): InvalidInputException: Timeout should be less than or equal to 10080 minutes , leave the timeout value as null to run it longer than 10080 minutes.
Updating the job by removing the timeout value and setting up a maintenance window does not reflect the intended changes in the timeout value, when the configuration is applied.
The issue appears to be that terraform plan does not recognize the timeout value being set to null, and thus terraform apply does not update it.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
Volunteering to Work on This Issue
If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.
Terraform Core Version
1.3.9
AWS Provider Version
5.58.0
Affected Resource(s)
Resource:
aws_glue_job
, Type:gluestreaming
Expected Behavior
The job's timeout value should be set to null, allowing it to run longer than 10080 minutes as per the updated configuration.
Actual Behavior
The timeout value of the job remains set to the previous value of 10 years despite the
terraform apply
command.Relevant Error/Panic Output Snippet
I tried changing the value of timeout to null from AWS Glue console, and then hit
terraform plan
withtimeout
parameter removed from the aws_glue_job terraform configuration. Then the plan produces the below changes -Terraform Configuration Files
Steps to Reproduce
aws_glue_job
resource of streaming type, with a timeout value set to 10080. (5256000 as timeout may not work)aws_glue_job
resource by removing thetimeout
parameter or set its value tonull
and adding a maintenance window.Note: If the timeout value is set to another non-zero value, then
terraform plan
does recognize the change.Debug Output
No response
Panic Output
No response
Important Factoids
Initially, the jobs were created successfully with a timeout value set to 5256000 (10 years). The jobs were setup about an year ago.
When attempted to update the worker type of the job, the operation failed due to the timeout value being greater than 10080 minutes.
Error: error updating Glue Job (my_job_name): InvalidInputException: Timeout should be less than or equal to 10080 minutes , leave the timeout value as null to run it longer than 10080 minutes.
Updating the job by removing the timeout value and setting up a maintenance window does not reflect the intended changes in the timeout value, when the configuration is applied.
The issue appears to be that
terraform plan
does not recognize the timeout value being set to null, and thusterraform apply
does not update it.References
For streaming jobs that run indefinitely, leave the value blank, which is the default value for streaming jobs.
Would you like to implement a fix?
None