MaterializeInc / pulumi-fivetran

A Pulumi provider for the Fivetran ETL platform.
Apache License 2.0
6 stars 1 forks source link

daily_sync_time does not get updated in Fivetran #3

Open solomonshorser opened 1 year ago

solomonshorser commented 1 year ago

When I change the daily_sync_time for a connector using this pulumi provider, it does not get updated in Fivetran. Running pulumi up will continue to show diffs on this property, and if I look at the Fivetran UI, it is not updated there either.

If I try to update a connector's daily_sync_time with Terraform, the update takes and the change is visible in Fivetran.

benesch commented 1 year ago

I'm afraid I won't have time to look at this, but happy to entertain a PR if you manage to track down what is going wrong.

On Wed, May 31, 2023 at 9:16 AM SolomonShorser @.***> wrote:

When I change the daily_sync_time for a connector using this pulumi provider, it does not get updated in Fivetran. Running pulumi up will continue to show diffs on this property, and if I look at the Fivetran UI, it is not updated there either.

If I try to update a connector's daily_sync_time with Terraform, the update takes and the change is visible in Fivetran.

— Reply to this email directly, view it on GitHub https://github.com/MaterializeInc/pulumi-fivetran/issues/3, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGXSIDFAJG3YY4EKI36A6DXI5ADZANCNFSM6AAAAAAYVOQ2EY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

solomonshorser commented 1 year ago

From what I can see, fivetran_connector_schedule (https://github.com/fivetran/terraform-provider-fivetran/blob/3c225786a50fe1679955500ebab79c3de85d7c56/fivetran/provider.go#L27) is a new object that will manage schedule info for a connector - I think this is missing in the pulumi-fivetran code. I added "fivetran_connector_schedule": {Tok: makeResource(mainMod, "ConnectorSchedule")}, to provider/provider.go (in the Resources map) and it now builds without errors, though it still doesn't seem to have an effect on the actual schedule time in Fivetran when I use it, but I think there are few more pieces missing. I'm not familiar with golang, or how these Pulumi plugins work, but if I can get things working, I'll put it into a PR.

solomonshorser commented 1 year ago

@benesch I had started working on this, but then I saw you released v0.1.12 and I realise you beat me to it! I tried the new version out, using the new ConnectorSchedule class, but I get a weird error:

     Type                          Name                                                                                Plan       Info
     pulumi:pulumi:Stack           my_stack
 ~   ├─ pulumi:providers:fivetran  provider                                                                            update     [diff: ~pluginDownloadURL,version]
     └─ pulumi:providers:fivetran  default_0_1_12_https_/github.com/benesch/pulumi-fivetran/releases/download/v0.1.12             1 error

Diagnostics:
  pulumi:providers:fivetran (default_0_1_12_https_/github.com/benesch/pulumi-fivetran/releases/download/v0.1.12):
    error: rpc error: code = InvalidArgument desc = required configuration keys were missing

The code looks like this:

        connector = Connector(resource_name=target_name,
                              destination_schema=destination_schema,
                              opts=opts,
                              group_id=self.group_id,
                              service="s3",
                              run_setup_tests="true",
                              config=connector_config,
                              trust_certificates=False,
                              )

        schedule = ConnectorSchedule(resource_name= self.db_name.upper() + "_s3_schedule",
                                    args=ConnectorScheduleArgs(
                                            connector_id=connector.id,
                                            daily_sync_time=self.s3_sync_time,
                                            paused=self.start_paused,
                                            pause_after_trial=self.pause_after_trial,
                                            schedule_type="AUTO",
                                            sync_frequency=self.sync_frequency,
                                        ),
                                     opts=ResourceOptions(depends_on=[connector]))

Any ideas about what code = InvalidArgument desc = required configuration keys were missing might mean?

benesch commented 1 year ago

Afraid not, sorry! Maybe the new version of the provider requires an argument that you're not passing?

solomonshorser commented 1 year ago

I thought I set everything that there was to set. It's frustrating that pulumi won't say which required keys are missing.