PrefectHQ / terraform-provider-prefect

Terraform Provider for Prefect Cloud
https://registry.terraform.io/providers/PrefectHQ/prefect/latest/docs
Apache License 2.0
29 stars 13 forks source link

feat: add prefect_worker_metadata datasource, for base job configs #106

Closed parkedwards closed 8 months ago

parkedwards commented 8 months ago

resolves https://github.com/PrefectHQ/terraform-provider-prefect/issues/35

When creating new Work Pool, the API expects a fairly complex JSON string for the base_job_template - this is the jinja-based template string that powers the Job itself

Most advanced users will likely manage a bespoke JSON file in their Terraform repo, and can load that in via file(). However, this PR introduces a new prefect_worker_metadata datasource, which calls the GET /collections/views/aggregate-worker-metadata endpoint. This endpoint returns ~7 base job configuration strings for each of the common worker types

This datasource can be used as a convenience resource for practitioners, so that they can reference a key off the resource and pass it directly into a work pool resource during creation

resource "prefect_work_pool" "example" {
  name              = "test-k8s-pool"
  type              = "kubernetes"
  workspace_id      = data.prefect_workspace.prd.id
  paused            = false
  base_job_template = file("./base-job-template.json")
}

# or

data "prefect_worker_metadata" "d" {}

resource "prefect_work_pool" "kubernetes" {
  name              = "test-k8s-pool"
  type              = "kubernetes"
  workspace_id      = data.prefect_workspace.prd.id
  paused            = false
  base_job_template = data.prefect_worker_metadata.d.base_job_configs.kubernetes
}