`prefect_work_pool` creation on Prefect Server

arthur-leclerc commented 10 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform Version

Terraform v1.5.5
on darwin_arm64

Affected Resource(s)

prefect_work_pool

Terraform Configuration Files

terraform {
  required_version = "1.5.5"

  required_providers {
    prefect = {
      source  = "prefecthq/prefect"
      version = "0.0.0-alpha8"
    }
  }
}

provider "prefect" {
  endpoint = "https://<redacted>"
  api_key  = "<redacted>"
}

resource "prefect_work_pool" "default" {
  name              = "default"
  type              = "kubernetes"
  paused            = false
}

Debug Output

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Warning: Prefect API Key
│
│   with provider["registry.terraform.io/prefecthq/prefect"],
│   on providers.tf line 66, in provider "prefect":
│   66: provider "prefect" {
│
│ The Prefect API Key is set, however, the Endpoint is set to a Prefect server installation. Potential resolutions: set the endpoint attribute or PREFECT_API_URL environment variable to a Prefect Cloud endpoint, unset the PREFECT_API_KEY environment variable, or remove the api_key attribute.
╵
╷
│ Error: Error creating work pool client
│
│   with prefect_work_pool.default,
│   on main.tf line 1, in resource "prefect_work_pool" "default":
│    1: resource "prefect_work_pool" "default" {
│
│ Could not create work pool client, unexpected error: both accountID and workspaceID must be set: accountID is "00000000-0000-0000-0000-000000000000" and workspaceID is "00000000-0000-0000-0000-000000000000". This is a bug in the provider, please report this to the maintainers.
╵
exit status 1

Panic Output

Expected Behavior

Be able to create a work pool on Prefect Server.

Actual Behavior

Since version v0.0.0-alpha8, it is no longer possible to create a work pool when we point the provider to a running Prefect Server.

We were particularly happy in my company to see the creation of a Terraform provider compatible with Prefect Server, allowing us to easily configure Prefect, but I am now wondering if you have the objective of continuing to support different targets.

Are these things something you don't want to support with this Terraform provider? Is it primarily designed with the Cloud version of Prefect as a target?

Additionally, as you can see in the TF sample code, we were pleased to see that it was possible to set a Bearer token via the api_key variable in the provider block allowing us to take advantage of the authentication in place on our infrastructure thanks to Authentik, resolving the authentification problem, Prefect not having one.

Is this also something you have in mind to allow the provider to be configured with custom authentication systems like Basic/Bearer token, or is it just some kind of allowed hack with the current implementation?

Steps to Reproduce

terraform apply

Important Factoids

References

0000

parkedwards commented 10 months ago

hey @arthur-leclerc - thanks for raising this issue. when creating the provider initialization logic + validations, we had Cloud in mind when checking for the presence of an API Key; so we did not initially consider your use case of using a separate auth provider for a self-managed Server deployment.

That being said, as you've identified - we're simply passing this API Key to an http client, and we're open to lifting this validation check out of the provider to support your use case

parkedwards commented 10 months ago

hey @arthur-leclerc - i just pushed a fix for this and cut a release to the provider registry. when it finishes publishing, would you give this a try with v0.1.1

https://github.com/PrefectHQ/terraform-provider-prefect/releases/tag/v0.1.1

arthur-leclerc commented 10 months ago

It works perfectly, thank you!

PrefectHQ / terraform-provider-prefect