awscc_dms_data_provider does not create

kstephensontm commented 6 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Affected Resource(s)

awscc_dms_data_provider

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "awscc_dms_data_provider" "dms_data_provider_source" {
  data_provider_name = "sc-sql-server"
  engine = "sqlserver"
  settings = jsonencode({
    MicrosoftSqlServerSettings = {
      DatabaseName = "mssql"
      Port         = 1433
      ServerName   = "mssql-default.blah.us-west-2.rds.amazonaws.com"
      SslMode      = "none"
    }
  })
}

Debug Output

Panic Output

Expected Behavior

The resource awscc_dms_data_provider should have created the data provider. I was able to create the data provider manually and import without any issues which tells me this is a bug. It will not let me create a new data provider.

Actual Behavior

Just errors out with a GeneralServiceException.

│ Error: AWS SDK Go Service Operation Incomplete │ │ with awscc_dms_data_provider.dms_data_provider_source, │ on dms-schema-conversion.tf line 55, in resource "awscc_dms_data_provider" "dms_data_provider_source": │ 55: resource "awscc_dms_data_provider" "dms_data_provider_source" { │ │ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state │ transitioned to FAILED. StatusMessage: Error occurred during operation 'AWS::DMS::CreateDataProvider'.. │ ErrorCode: GeneralServiceException

Steps to Reproduce

terraform apply

Important Factoids

None

References

https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/dms_data_provider

quixoticmonk commented 6 months ago

Thank you for opening the issue, @kstephensontm . Can you check with the settings based on the schema below ?

resource "awscc_dms_data_provider" "dms_data_provider_source" {
  data_provider_name = "sc-sql-server"
  engine = "sqlserver"
  settings = {
    microsoft_sql_server_settings = {
      database_name = "mssql"
      port         = 1433
      server_name   = "mssql-default.blah.us-west-2.rds.amazonaws.com"
      ssl_mode      = "none"
    }
  }
}

kstephensontm commented 6 months ago

It gives me this error....

│ Error: Incorrect attribute value type │ │ on dms-schema-conversion.tf line 29, in resource "awscc_dms_data_provider" "dms_data_provider_source": │ 29: settings = { │ 30: microsoft_sql_server_settings = { │ 31: database_name = "mssql" │ 32: port = 1433 │ 33: server_name = "mssql-default.blah.us-west-2.rds.amazonaws.com" │ 34: ssl_mode = "none" │ 35: } │ 36: } │ │ Inappropriate value for attribute "settings": string required.

quixoticmonk commented 6 months ago

What version of the awscc provider are you using ? I am not able to reproduce your scenario.

Plan: 1 to add, 0 to change, 0 to destroy.
awscc_dms_data_provider.dms_data_provider_source: Creating...
awscc_dms_data_provider.dms_data_provider_source: Still creating... [10s elapsed]
awscc_dms_data_provider.dms_data_provider_source: Creation complete after 11s [id=arn:aws:dms:us-east-1:XXXX:data-provider:QB3OSOECCBDZPFZP2VCKWX2MVU]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

kstephensontm commented 6 months ago

the latest version ...

awscc = {
  source  = "hashicorp/awscc"
  version = "0.74.0"
}

quixoticmonk commented 6 months ago

That must be it. The version has been bumped and from the attribute reference I see the difference in schema. 0.75

[settings](https://registry.terraform.io/providers/hashicorp/awscc/0.75.0/docs/resources/dms_data_provider#settings) 

(Attributes) The property identifies the exact type of settings for the data provider. (see [below for nested schema](https://registry.terraform.io/providers/hashicorp/awscc/0.75.0/docs/resources/dms_data_provider#nestedatt--settings))

0.74

[settings](https://registry.terraform.io/providers/hashicorp/awscc/0.74.0/docs/resources/dms_data_provider#settings) 

(String) The property identifies the exact type of settings for the data provider.

kstephensontm commented 6 months ago

Yes, that worked. Wow, they just published it yesterday after I tried all of this. Thanks for your help.

kstephensontm commented 6 months ago

hi @quixoticmonk ... i'm creating a provider for postgresql aurora and it's saying it's not supported but i'm able to create it manually with no issues. any ideas on why it's throwing this error?

│ Error: AWS SDK Go Service Operation Incomplete │ │ with awscc_dms_data_provider.dms_data_provider_target, │ on dms-schema-conversion.tf line 39, in resource "awscc_dms_data_provider" "dms_data_provider_target": │ 39: resource "awscc_dms_data_provider" "dms_data_provider_target" { │ │ Waiting for Cloud Control API service CreateResource operation completion returned: waiter │ state transitioned to FAILED. StatusMessage: DMS doesn't support aurora_postgresql database │ engine. Choose another database engine and try again. (Service: DatabaseMigration, Status │ Code: 400, Request ID: a5033be9-0e1c-457a-bd9a-e751af7e224f). ErrorCode: InvalidRequest

resource "awscc_dms_data_provider" "dms_data_provider_target" { data_provider_name = "sc-postgresql2" engine = "aurora_postgresql" settings = { postgre_sql_settings = { database_name = "pssql" port = 5432 server_name = "postgres-default-1.blah.us-west-2.rds.amazonaws.com" ssl_mode = "none" } } }

kstephensontm commented 6 months ago

also ... i have my sqlserver server name set with blah and it works. as soon as i change it to the actual endpoint i get the following error.

server_name = "mssql-default.blah.us-west-2.rds.amazonaws.com"

│ Error: AWS SDK Go Service Operation Incomplete │ │ with awscc_dms_data_provider.dms_data_provider_source, │ on dms-schema-conversion.tf line 26, in resource "awscc_dms_data_provider" "dms_data_provider_source": │ 26: resource "awscc_dms_data_provider" "dms_data_provider_source" { │ │ Waiting for Cloud Control API service UpdateResource operation completion returned: waiter │ state transitioned to FAILED. StatusMessage: Error occurred during operation │ 'AWS::DMS::UpdateDataProvider'.. ErrorCode: GeneralServiceException

kstephensontm commented 6 months ago

just realized that the error of changing the sqlserver to the actual endpoint errored because that endpoint is already in use. Would be nice to have the error state that though. I'm still having issues with the engine being set to aurora_postgresql. this says it's not supported.

quixoticmonk commented 6 months ago

I can reproduce the engine reference for aurora_postgreqsl.
@kstephensontm Do you mind opening another issue for the GeneralServiceException on the endpoint change?

│ Error: AWS SDK Go Service Operation Incomplete
│
│   with awscc_dms_data_provider.dms_data_provider_source,
│   on main.tf line 1, in resource "awscc_dms_data_provider" "dms_data_provider_source":
│    1: resource "awscc_dms_data_provider" "dms_data_provider_source" {
│
│ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state
│ transitioned to FAILED. StatusMessage: DMS doesn't support aurora_postgresql database engine.
│ Choose another database engine and try again. (Service: DatabaseMigration, Status Code: 400,

@wellsiau-aws This might be another upstream one.

I do get the valid list from the api when I try with an incorrect engine type.

 Error: Invalid Attribute Value Match
│
│   with awscc_dms_data_provider.dms_data_provider_source,
│   on main.tf line 3, in resource "awscc_dms_data_provider" "dms_data_provider_source":
│    3:   engine = "aurora-postgresql"
│
│ Attribute engine value must be one of: ["postgres" "mysql" "oracle" "sqlserver" "aurora"
│ "aurora_postgresql"], got: "aurora-postgresql"

kstephensontm commented 6 months ago

created issue https://github.com/hashicorp/terraform-provider-awscc/issues/1636

wellsiau-aws commented 6 months ago

@kstephensontm @quixoticmonk , thanks for reporting this issue. I will respond about the engine selection problem on #1636

with that said, are there still outstanding problem that we need to cover on this particular issue?

gzurowski commented 6 months ago

@wellsiau-aws I am getting similar issues when trying to create a data provider for a Postgres database using the following code:

resource "awscc_dms_data_provider" "source" {
  data_provider_name = "my-data-provider-source"
  engine             = "postgres"
  settings = {
    postgre_sql_settings = {
      server_name   = var.source_database_hostname
      port          = 5432
      database_name = "mydatabase1"
      ssl_mode      = "require"
    }
  }
}

The error looks as follows:

module.database-migraton.awscc_dms_data_provider.source: Creating...
╷
│ Error: AWS SDK Go Service Operation Incomplete
│
│   with module.database-migraton.awscc_dms_data_provider.source,
│   on ../../../modules/database-migration/main-homogeneous.tf line 1, in resource "awscc_dms_data_provider" "source":
│    1: resource "awscc_dms_data_provider" "source" {
│
│ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: Error occurred during operation
│ 'AWS::DMS::CreateDataProvider'.. ErrorCode: GeneralServiceException

quixoticmonk commented 6 months ago

@gzurowski I was able to create the resource without any issues on my end for postgres. What version of tf cli, provider are you on ?

resource "awscc_dms_data_provider" "dms_data_provider_target" {
  data_provider_name = "sc-postgresql2"
  engine             = "postgres"
  settings = {
    postgre_sql_settings = {
      database_name = "pssql"
      port          = 5432
      server_name   = "postgres-default-1.blah.us-west-2.rds.amazonaws.com"
      ssl_mode      = "require" # Tried both none and require and resource was created without any issues
    }
  }
}

kstephensontm commented 6 months ago

postgres works for me, it's aurora_postgresql that does not work.

kstephensontm commented 6 months ago

i have this issue open for it. https://github.com/hashicorp/terraform-provider-awscc/issues/1636

gzurowski commented 6 months ago

@gzurowski I was able to create the resource without any issues on my end for postgres. What version of tf cli, provider are you on ?

I am using the following version (and the creation of the data provider fails consistently):

Terraform v1.7.5
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.47.0
+ provider registry.terraform.io/hashicorp/awscc v0.75.0

andreshans commented 3 weeks ago

does anyone know how to fix this one?

hashicorp / terraform-provider-awscc