[airbyte_connection] [bug] MongoDB stream change produces error as primary key is pre-defined

[ivan-sukhomlyn]

After schema updating, the order of MongoDB stream representation in Terraform state changes, and it can not be rewritten due to the already defined primary key, even if it's not defined in the source code.

For example, HCL

resource "airbyte_connection" "mongodb_bigquery" {
...
  configurations = {
    streams = [
      {
        name      = test_collection_1
        sync_mode = "incremental_deduped_history"
      },
      {
        name      = test_collection_2
        sync_mode = "incremental_deduped_history"
      }
    ]
  }

After schema updating, I got the following terraform plan:

  # airbyte_connection.mongodb_bigquery will be updated in-place
  ~ resource "airbyte_connection" "mongodb_bigquery" {
      ~ configurations                       = {
          ~ streams = [
              ~ {
                  ~ name         = "test_collection_2" -> "test_collection_1"
                    # (3 unchanged attributes hidden)
                },
              ~ {
                  ~ name         = "test_collection_1" -> "test_collection_2"
                    # (3 unchanged attributes hidden)
                },
            ]
        }
...
    }

terraform apply:

{"type":"https://reference.airbyte.com/reference/errors","detail":"The body of the request contains an invalid connection configuration. **Primary key for stream: test_collection_1 is already pre-defined.** Please do NOT include a primary key configuration for this stream.","title":"bad-request","status":400}

[ivan-sukhomlyn]

The same situation is for the notion source

[ivan-sukhomlyn]

can be omitted by the following definition

resource "airbyte_connection" "mongodb_bigquery" {
...
  configurations = {
    streams = [
        name        = "test_collection_2"
        sync_mode   = "incremental_deduped_history"
        primary_key = null
      }
    ]
  }
}

[ivan-sukhomlyn]

Fixed by https://github.com/airbytehq/terraform-provider-airbyte/pull/128

[ivan-sukhomlyn]

The error is due to the already pre-defined primary key still being present.

╷
│ Error: failure to invoke API
│ 
│   with airbyte_connection.mongodb_bigquery
│ 
│ unknown status code returned: Status 400
│ {"type":"https://reference.airbyte.com/reference/errors","detail":"The body of the request contains an invalid connection configuration. Primary key for stream:
│ { some_stream } is already pre-defined. Please do NOT include a primary key configuration for this stream.","title":"bad-request","status":400}