PrefectHQ / prefect

Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines
https://prefect.io
Apache License 2.0
15.29k stars 1.5k forks source link

Env vars set from a Prefect secret in prefect.yaml are showing up under the deployment configuration in the UI #14088

Closed tomasmv closed 3 hours ago

tomasmv commented 1 week ago

First check

Bug summary

I'm setting an environment variable in my prefect.yaml via a series of definitions:

definitions:
  work_pools:
    work_pool: &work_pool
      name: my-ecs-pool
      work_queue_name:
      job_variables: &work_pool_job_variables
        env: &work_pool_env_variables
          REDSHIFT_PW: "{{ prefect.blocks.secret.password }}"

When I deploy my flow and check under the Configuration tab for the deployment, I see the raw value of my secret.

Reproduction

definitions:
  work_pools:
    work_pool: &work_pool
      name: my-ecs-pool
      work_queue_name:
      job_variables: &work_pool_job_variables
        env: &work_pool_env_variables
          REDSHIFT_PW: "{{ prefect.blocks.secret.password }}"

deployments:
  - name: test
    version:
    tags:
      - test
    description:
    entrypoint: flows/flow_elt.py:elt_flow
    work_pool: *work_pool_stg

Error

No response

Versions

2.16.8

Additional context

No response

zhen0 commented 3 hours ago

Thanks for the issue @tomasmv - this should be closed by Fix bug where block documents are resolved in job_variables before saving server-side #14156