awscc_appflow_flow does not recognize empty list of string in source_fields when doing map_all operation

yzouyang commented 1 year ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Affected Resource(s)

awscc_appflow_flow

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "awscc_appflow_flow" "test-facebook-ads-flow-terraform" {
  flow_name = "test-facebook-flow-terraform"
  destination_flow_config_list = [{
    connector_type = "S3"
    destination_connector_properties = {
      s3 = {
        bucket_name   = "aws-appflow-resource"
        bucket_prefix = "facebook-test-terraform"
        s3_output_format_config = {
          aggregation_config = {
            aggregation_type = "SingleFile"
          }
          file_type = "CSV"
          prefix_config = {
            prefix_format = "DAY"
            prefix_type   = "PATH"
          }
        }
      }
    }
  }]
  source_flow_config = {
    api_version            = "v15.0"
    connector_type         = "CustomConnector"
    connector_profile_name = "appflow-facebook-ad-connection"
    source_connector_properties = {
      custom_connector = {
        entity_name = "act_masked/ads"
      }
    }
  }
  trigger_config = {
    trigger_type = "OnDemand"
  }
  tasks = [{
    connector_operator = {
      custom_connector = "NO_OP"
    }
    source_fields = []
    task_type     = "Map_all"
    task_properties = [
      {
        key   = "EXCLUDE_SOURCE_FIELDS_LIST"
        value = "[]"
      }
    ]
    write_operation_type = "INSERT"
  }]
}

Debug Output

ValidationException: Model validation failed (#/Tasks/0: required key [SourceFields] not found)

Panic Output

Expected Behavior

On-demand flow created

Actual Behavior

Creation failed

Steps to Reproduce

Set up a test connection profile "appflow-facebook-ad-connection" for Facebook Ads source connector with access to the 'ads' endpoint for a particular Facebook account (masked)
terraform apply

Important Factoids

References

0000

bpicolo commented 9 months ago

This may stem from https://github.com/hashicorp/terraform-provider-awscc/issues/700 ?

Some AWS resource creation APIs have non-optional list fields. Empty list is a valid input, but null or a missing JSON key are not. These don't seem to function with terraform-provider-awscc today when an empty list is passed as input, because it's being converted to null. (Or rather, the key doesn't seem to be present in the API call at all from what I see setting TF_LOG=debug)

rofreytag commented 4 months ago

Truly frustrating. I moved to awscc to fix a bunch of issues with appflow, as they are not supported well with aws provider, but ran into this exact issue. Wonder if other APIs are affected by this?

hashicorp / terraform-provider-awscc