PrefectHQ / terraform-provider-prefect

Terraform Provider for Prefect Cloud
https://registry.terraform.io/providers/PrefectHQ/prefect/latest/docs
Apache License 2.0
29 stars 13 forks source link

Add Docker Compose file for local testing #187

Closed mitchnielsen closed 4 weeks ago

mitchnielsen commented 1 month ago

Summary

Adds a local Docker Compose file to start an instance of Prefect that we can reach with terraform commands.

Closes https://github.com/PrefectHQ/terraform-provider-prefect/issues/186

Testing

Click to expand ``` $ terraform plan ╷ │ Warning: Provider development overrides are in effect │ │ The following provider development overrides are set in the CLI configuration: │ - prefecthq/prefect in /Users/mitch/go/bin │ │ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with │ published releases. ╵ Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # prefect_work_pool.example will be created + resource "prefect_work_pool" "example" { + base_job_template = jsonencode({}) + created = (known after apply) + default_queue_id = (known after apply) + id = (known after apply) + name = "my-work-pool" + paused = true + type = "kubernetes" + updated = (known after apply) } Plan: 1 to add, 0 to change, 0 to destroy. ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now. ~/code/github.com/prefecthq/terraform-provider-prefect/add-testing-in-docker/examples/provider-install-verification on  add-testing-in-docker *?! $ terraform apply ╷ │ Warning: Provider development overrides are in effect │ │ The following provider development overrides are set in the CLI configuration: │ - prefecthq/prefect in /Users/mitch/go/bin │ │ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with │ published releases. ╵ Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # prefect_work_pool.example will be created + resource "prefect_work_pool" "example" { + base_job_template = jsonencode({}) + created = (known after apply) + default_queue_id = (known after apply) + id = (known after apply) + name = "my-work-pool" + paused = true + type = "kubernetes" + updated = (known after apply) } Plan: 1 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes prefect_work_pool.example: Creating... prefect_work_pool.example: Creation complete after 0s [id=c088c80f-9612-4912-b03b-d5bea4facac7] Apply complete! Resources: 1 added, 0 changed, 0 destroyed. ``` image
mitchnielsen commented 4 weeks ago

@parkedwards - looking at the PREFECT_API_URL in the acceptance tests CI file, I suspect we could replace the value (which I think points to dev or staging) and instead spin up a Prefect server instance using this new Docker Compose setup. That'd give us fresh environments to test against in CI and might help with transient errors related to environment pollution.

Does that sound right/fair? If so I'll open an issue and we can look into it more in a separate PR.

parkedwards commented 4 weeks ago

@parkedwards - looking at the PREFECT_API_URL in the acceptance tests CI file, I suspect we could replace the value (which I think points to dev or staging) and instead spin up a Prefect server instance using this new Docker Compose setup. That'd give us fresh environments to test against in CI and might help with transient errors related to environment pollution.

@mitchnielsen that sounds right - could be something like http://localhost:4200 https://github.com/PrefectHQ/terraform-provider-prefect/blob/main/internal/provider/provider.go#L116-L126

just a caveat - one wrinkle is that our OSS server isn't an exact match/port of our SaaS (which the current acceptance tests are mostly oriented towards supporting) - the OSS server has a subset of features and endpoints. so there may be some extra potholes we hit as we try to marry the tests<>this local prefect server instance. but i like where this is going

mitchnielsen commented 4 weeks ago

just a caveat - one wrinkle is that our OSS server isn't an exact match/port of our SaaS (which the current acceptance tests are mostly oriented towards supporting) - the OSS server has a subset of features and endpoints. so there may be some extra potholes we hit as we try to marry the tests<>this local prefect server instance. but i like where this is going

Great point. I added this note in https://github.com/PrefectHQ/terraform-provider-prefect/issues/192.