hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.61k stars 9k forks source link

[Bug]: Sagemaker target, unable to create pipeline target with no parameters (API error) #33526

Open alagden-cmc opened 9 months ago

alagden-cmc commented 9 months ago

Terraform Core Version

1.2.x

AWS Provider Version

5.17.0

Affected Resource(s)

aws-scheduler-schedule (probably affected cloudwatch events_targets?)

Expected Behavior

Should be able to create a schedule with a SageMaker pipeline as a target , but an empty parameter list. (When created through AWS console, the parameter list is shown as [] upon review in the defined schedule)

Actual Behavior

API error during terraform apply. (plan passes ok)

Relevant Error/Panic Output Snippet

Error: creating Amazon EventBridge Scheduler Schedule (schedule-name): operation error Scheduler: CreateSchedule, https response error StatusCode: 400, RequestID: 5e819b5b-5097-45d5-8c8a-f047a070bbc6, ValidationException: Parameters not valid due to the following reason: Parameters SageMakerPipelineParameters must be specified.

Terraform Configuration Files

resource "aws_scheduler_schedule" "this" {
  name                = "schedule-name"
  schedule_expression = "cron(0 0 ? * Sun *)"
  state               = "DISABLED"

  flexible_time_window {
    mode = "OFF"
  }

  target {
    arn      = <arn of sagemaker pipeline>
    role_arn = <sagemaker execution role>
    sagemaker_pipeline_parameters {
    }
  }
}

Steps to Reproduce

A simple pipeline target but with no parameters.

Debug Output

No response

Panic Output

No response

Important Factoids

Works fine if i add a dummy name/value parameter.

References

https://github.com/hashicorp/terraform-provider-aws/issues/21374 https://github.com/hashicorp/terraform-provider-aws/pull/32882

Would you like to implement a fix?

No

github-actions[bot] commented 9 months ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

justinretzolk commented 8 months ago

Hey @alagden-cmc 👋 Thank you for taking the time to raise this! Pardon my ignorance around this particular resource, but I modified your sample configuration slightly, removing the empty sagemaker_pipeline_parameters block, and it passed terraform validate:

variable "arn" {}
variable "role_arn" {}

resource "aws_scheduler_schedule" "this" {
  name                = "schedule-name"
  schedule_expression = "cron(0 0 ? * Sun *)"
  state               = "DISABLED"

  flexible_time_window {
    mode = "OFF"
  }

  target {
    arn      = var.arn
    role_arn = var.role_arn
  }
}

Does making that change work for you, or am I possibly missing some context?

alagden-cmc commented 8 months ago

Hi Justin, the issue only shows itself when doing the terraform apply. i had previously tried some variations and removing the block as described i think. I will check and attempt a test again, but It appeared the AWS API requires the empty list somehow. It always passed validate and plan, just couldnt apply. I've seen something similar when manually editing a AWS scheduler entry in the AWS console with a Sagemaker pipeline target.. If you specify the pipeline target with no parameters or remove any existing pipeline parameters then view the pipeline target again, you can see the parameters defined as '[]'. I think the API is expecting the same. I'll try to add a screenshot of how that looks below. Screenshot 2023-11-09 at 17 33 17

alagden-cmc commented 8 months ago

Just to confirm, i double checked and everything passes but still fails on the terraform apply when the sagemaker_pipeline_parameters is absent. Error is still: Error: creating Amazon EventBridge Scheduler Schedule (schedule-pipeline-test): operation error Scheduler: CreateSchedule, https response error StatusCode: 400, RequestID: d8c95826-1fd2-4845-bad6-2ba13d299c14, ValidationException: Parameters not valid due to the following reason: Parameters SageMakerPipelineParameters must be specified.

mritterfigma commented 2 months ago

Note that the ability to specify empty parameters is important because otherwise, you cannot schedule a pipeline that has no parameters. I tried putting in placeholder parameters, but then the pipeline fails to execute:

    "errorMessage": "Encountered errors when validating the provided pipeline parameters. The following provided parameters are not present in the pipeline definition: [Parameter(name=placeholder, value=placeholder)]. ",