fivetran / terraform-provider-fivetran

Terraform Provider for Fivetran
https://fivetran.com
Apache License 2.0
41 stars 22 forks source link

fivetran_connector crashes on hubspot, slack, zoom, intercom, adyen, notion #282

Closed petero2024 closed 3 months ago

petero2024 commented 3 months ago

Describe the bug fivetran_connector crashes on the following services: hubspot, slack, zoom, intercom, adyen, notion

also, there seem to be no documentation for intercom and adyen

To Reproduce

resource "fivetran_connector" "hubspot_connector" { group_id = var.group_id service = "hubspot" }

Expected behavior A clear and concise description of what you expected to happen.

provision an incomplete connector or give an error why that's not possible. But i'm sure it's possible, on other connectors it is sufficient to add the required parameters, group_id and service name

Logs & Output Provide terraform CLI output on plan | apply. If it is possible, please, refer to this guide to enable debug output.

Stack trace from the terraform-provider-fivetran_v1.1.19.exe plugin:

panic: runtime error: invalid memory address or nil pointer dereference [signal 0xc0000005 code=0x0 addr=0x20 pc=0xe4623f]

goroutine 31 [running]: github.com/fivetran/terraform-provider-fivetran/fivetran/framework/core/model.getDestinatonSchemaForConfig({0x14a9bd8?, 0xc00089a298?}, {0x0, 0x0}, {0x0, 0x0}, {0x0?, 0x0?}) github.com/fivetran/terraform-provider-fivetran/fivetran/framework/core/model/connector.go:257 +0x35f github.com/fivetran/terraform-provider-fivetran/fivetran/framework/core/model.(ConnectorResourceModel).GetDestinatonSchemaForConfig(0xc000118700) github.com/fivetran/terraform-provider-fivetran/fivetran/framework/core/model/connector.go:167 +0x530 github.com/fivetran/terraform-provider-fivetran/fivetran/framework/resources.(connector).Create(0xc0001100b8, {0x14a6360, 0xc00099dda0}, {{{{0x14ab5d8, 0xc0009f4c30}, {0x11fe5c0, 0xc0009e6240}}, {0x14ad2c0, 0xc000138640}}, {{{0x14ab5d8, ...}, ...}, ...}, ...}, ...) github.com/fivetran/terraform-provider-fivetran/fivetran/framework/resources/connector.go:117 +0x556 github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(Server).CreateResource(0xc000240420, {0x14a6360, 0xc00099dda0}, 0xc00089b2c0, 0xc00089b260) github.com/hashicorp/terraform-plugin-framework@v1.4.2/internal/fwserver/server_createresource.go:101 +0x5b8 github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(Server).ApplyResourceChange(0x14a6360?, {0x14a6360, 0xc00099dda0}, 0xc0001399f0, 0xc00089b430) github.com/hashicorp/terraform-plugin-framework@v1.4.2/internal/fwserver/server_applyresourcechange.go:57 +0x4a8 github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(Server).ApplyResourceChange(0xc000240420, {0x14a6360?, 0xc00099dc50?}, 0xc0001399a0) github.com/hashicorp/terraform-plugin-framework@v1.4.2/internal/proto6server/server_applyresourcechange.go:55 +0x27b github.com/hashicorp/terraform-plugin-mux/tf6muxserver.(muxServer).ApplyResourceChange(0x14a62b8?, {0x14a6360?, 0xc00099d920?}, 0xc0001399a0) github.com/hashicorp/terraform-plugin-mux@v0.12.0/tf6muxserver/mux_server_ApplyResourceChange.go:36 +0x1b5 github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(server).ApplyResourceChange(0xc0004a8140, {0x14a6360?, 0xc00099cf30?}, 0xc0001ae2a0) github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov6/tf6server/server.go:857 +0x574 github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ApplyResourceChange_Handler({0x12e0060?, 0xc0004a8140}, {0x14a6360, 0xc00099cf30}, 0xc0001ae230, 0x0) github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:467 +0x170 google.golang.org/grpc.(Server).processUnaryRPC(0xc0001b2000, {0x14abb60, 0xc0001044e0}, 0xc000bc2000, 0xc0001b4510, 0x1acb698, 0x0) google.golang.org/grpc@v1.57.0/server.go:1360 +0xe23 google.golang.org/grpc.(Server).handleStream(0xc0001b2000, {0x14abb60, 0xc0001044e0}, 0xc000bc2000, 0x0) google.golang.org/grpc@v1.57.0/server.go:1737 +0xa2f google.golang.org/grpc.(Server).serveStreams.func1.1() google.golang.org/grpc@v1.57.0/server.go:982 +0x98 created by google.golang.org/grpc.(*Server).serveStreams.func1 google.golang.org/grpc@v1.57.0/server.go:980 +0x18c

Error: The terraform-provider-fivetran_v1.1.19.exe plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue.

Plugin version: Please specify Fivetran Terraform Provider version you are using.

terraform { required_providers { fivetran = { source = "fivetran/fivetran" version = ">= 1.1.17" } } }

Additional context Add any other context about the problem here.

I tried with all sorts of permutations like adding auth/config section, destintaion_schema etc. However, even with the basics it crashed for hubspot, slack, zoom, intercom, adyen, notion

also, there seem to be no documentation for intercom and adyen

beevital commented 3 months ago

Hey @petero2024, thank you for the feedback, will dig into it right now. Looks like we missed that destination_schema should be mandatory field itself in resource schema.

petero2024 commented 3 months ago

these connectors do not work even if destination schema was provided.

I tried with all sorts of permutations like adding auth/config section, destintaion_schema etc. However, even with the basics it crashed for hubspot, slack, zoom, intercom, adyen, notion

also, there seem to be no documentation for intercom and adyen

beevital commented 3 months ago

Yeah, I know, I'm just noticing that provider allows to omit this field, but shouldn't

beevital commented 3 months ago

@petero2024 I've just checked with destination_schema:

resource "fivetran_connector" "hubspot_connector" {
  group_id = data.fivetran_group.testing.id
  service = "hubspot"
  destination_schema {
    name = "hubspot_tftest"
  }
}

Everything works fine on v1.1.19

beevital commented 3 months ago

Reproduced only absence for adyen and issue with omited destination_schema field. Everything else works. Will prepare fix for the issues I've reproduced.

petero2024 commented 3 months ago

hubspot and intercom seem working now. slack, zoom, adyen, and notion are still crashing regardless destination_schema supplied

beevital commented 3 months ago

slack, zoom, adyen, and notion are still crashing regardless destination_schema supplied

I've checked all of them: no issues except adyen - it is missing in schema, so terraform doesn't know about it yet.

beevital commented 3 months ago

There's another issue:

│ When applying changes to fivetran_connector.hubspot_connector, provider
│ "provider[\"registry.terraform.io/fivetran/fivetran\"]" produced an unexpected new value:
│ .config: was absent, but now present.

It will be fixed with https://github.com/fivetran/terraform-provider-fivetran/pull/283, next release planned for tomorrow.

beevital commented 3 months ago

And issue with

panic: Unknown serviceslack

goroutine 82 [running]:
github.com/fivetran/terraform-provider-fivetran/fivetran/common.GetFieldsForService(...)
...

Will be fixed with updated Open API schema, also with next release tomorrow)

beevital commented 3 months ago

I wasn't able to reproduce issue because I've being testing on latest main build, not on 1.1.19

petero2024 commented 3 months ago

sorry, one more thing Google Search console seem to be missing from the docs too. Does it have a fivetran_connector?

I now managed to create incomplete connectors with adding the destination schema for the following: hubspot, intercom, slack, zoom, notion

beevital commented 3 months ago

Yes, it should be google_search_console, and it's included in resource schema already.

beevital commented 3 months ago

I'll update schema and docs tomorrow to make it consistent with latest API version.

beevital commented 3 months ago

@petero2024 I've released new v.1.1.20, tested hubspot, intercom, slack, zoom, notion - succesfully created. As for adyen - it's not yet supported in Fivetran API.

beevital commented 3 months ago

Tested on 1.1.20 - no issues.