airbytehq / terraform-provider-airbyte

Terraform Provider for Airbyte API
https://api.airbyte.com/
MIT License
45 stars 21 forks source link

Connection schedule cannot be configured #63

Open willi-mueller opened 10 months ago

willi-mueller commented 10 months ago

I expect the following code to create a connection with a CRON schedule. However, it creates a disabled connection with manual schedule:

resource "airbyte_connection" "facebook_to_bq" {
  configurations = {
    schedule = {
      schedule_type   = "cron"
      cron_expression = "0 45 6 ? * MON,TUE,WED,THU,FRI *"
    }
    streams = [
      { name = "ads_insights" }
    ]
  }
  name                                 = "Facebook_Marketing-BigQuery"
  source_id                            = airbyte_source_facebook_marketing.facebook_marketing.source_id
  destination_id                       = airbyte_destination_bigquery.bigquery.destination_id
  prefix                               = "facebook_"
  non_breaking_schema_updates_behavior = "propagate_columns"
  status                               = "inactive"
}

The connection in the Airbyte UI:

Screenshot 2023-12-04 at 15 26 21

The terraform state:

"connection_id": "eb042d1d-8346-4309-bb6a-797090b5fe57",
            "data_residency": "auto",
            "destination_id": "ae63cf7c-fc9f-4607-b87c-15837417d0f0",
            "name": "Facebook_Marketing-BigQuery",
            "namespace_definition": "destination",
            "namespace_format": null,
            "non_breaking_schema_updates_behavior": "propagate_columns",
            "prefix": "facebook_",
            "schedule": {
              "basic_timing": null,
              "cron_expression": null,
              "schedule_type": "manual"
            },
            "source_id": "1095cdf7-5b11-47b8-8b8e-56b7fd7ec7e3",
            "status": "inactive",

Changing the CRON schedule to a different pattern does not change the schedule either. The Terraform state is unchanged.

Version: OSS Airbyte 0.50.34

ThomasRooney commented 10 months ago

Try adding “ UTC” to your cron expression as a suffix? The API is a little more particular about the format of the expression.

willi-mueller commented 10 months ago

Try adding “ UTC” to your cron expression as a suffix? The API is a little more particular about the format of the expression.

Thank you, @ThomasRooney, I overlooked that. I found one reference: https://github.com/airbytehq/terraform-provider-airbyte/blob/585436d1ba555088a194de89662330ea5e4c63b4/examples/pokeapi/example.tf#L87C4-L87C41

Can I contribute to clarify this in the API documentation as well as the docs of this provider?

codingtony-candid commented 9 months ago

Interestingly "UTC" works but "US/Eastern" does not, even if you can choose it from the UI and it's the value shown in a terraform plan after changing a connection manually

mkamysz commented 9 months ago

Adding to what @codingtony-candid mentioned, it is indeed quite weird behaviour that the module only appears to allow UTC as a timezone value, while you are able to change it manually to a different timezone in the UI.

Any chance to get proper support for all timezones that are available in the UI?

theashishbhatt commented 7 months ago

The issue seems to be with Airbyte API itself not just tf module. I am seeing similar problem airbyte api calls attempting to change the schedule type.