search

PrefectHQ / prefect-operator

A Kubernetes operator for managing Prefect servers and work pools
7 stars 0 forks source link

feat(work-pool): add support for external prefect servers #64

Closed parkedwards closed 1 week ago

parkedwards commented 1 week ago

resolves https://linear.app/prefect/issue/PLA-219/support-for-prefectworkpool-pointing-to-external-prefect-servers-and

this PR adds ~3 new fields:

  1. spec.server.remoteApiUrl => to be used if the apiKey is set
  2. spec.apiKey.value => raw, plaintext API Key
  3. spec.apiKey.valueFrom => reference-able secret (via k8s Secret) - uses the corev1.EnvVarSource struct as the type

if either .apiKey attribute is set, we'll use the .remoteApiUrl for PREFECT_API_URL

if either .apiKey attribute is set, we'll set a PREFECT_API_KEY env var

parkedwards commented 1 week ago

thanks for such thorough testing steps and documenting them! you're right - i think we'll need to either (1) document what valid API urls look like (eg. with Cloud, they'll need to include the account/workspace IDs) or (2) expose those as optional values that we'll use to construct the API url for them

parkedwards commented 1 week ago

that said, would someone use the Operator to deploy a Work Pool if they're using Prefect Cloud

I think they would. for some, I imagine this operator being an alternative to the prefect-helm charts that folks are already using to provision their worker(s) that run jobs against Cloud

parkedwards commented 1 week ago

@mitchnielsen i just pushed up this change to explicitly expose an accountid and workspaceid - TBD on if we keep this (vs. just documenting and expecting the user to bring their fully formed URL based on the installation type)

https://github.com/PrefectHQ/prefect-operator/pull/64/commits/d9239829065a276b4bd58c1883c91147aa828416

i'll give it a more thorough test in the morning