Lazy templating for deployments in prefect.yaml

Describe the current behavior

Currently when using prefect file for deploying deployments to Prefect Server, all template variables ({{ }}) are evaluated at deploy time. This behavior does not allow evaluating prefect.block or prefect.variable at deployment runtime and thus, any changes to the data in the references does not take effect after deployment. Any other variables like {{ event.resource... }} passed to deployment trigger parameters also does not work.

Describe the proposed behavior

First solution could be to implement some logic to allow lazy evaluation/parsing of template variables, that under some conditions would be either evaluated later in the deploy process or at deployment runtime. This logic could prove to be tricky and not intuitive for the users.
Probably better solution might be to implement an option to explicitly mark templating variable to be evaluated at deploy, or at deployment runtime (in Flow.validate_parameters forexample).

For blocks, it would require to update block evaluation to pass only reference dict object instead of resolved data object.

If there is a will to approve this enhancement, i volunteer to work on it as this is something that would really help me move from deployment definition in python. If there is a need for better design definition for the acceptance, let me know.

Edit: For Block we could implement .reference property that would return reference dict object with ID that could be used in current solution.

Example Use

deployments:
- name: MyDeployment
  description: null
  entrypoint: "my_module:my_flow"
  parameters:
    blocks: 
      - "{{ prefect.blocks.my-block-type.my-block1|reference }}"  # example
      - "{{ prefect.blocks.my-block-type.my-block1:runtime }}"  # example
      - "{{ prefect.blocks.my-block-type.my-block1|runtime }}"  # example
      - "{{ lazy:prefect.blocks.my-block-type.my-block1 }}"  # example
      - "{{ lazy|prefect.blocks.my-block-type.my-block1 }}"  # example
      - "{{ ref|prefect.blocks.my-block-type.my-block1 }}"  # example
      - "{{ ref:prefect.blocks.my-block-type.my-block1 }}"  # example
      - "{{ prefect.blocks.my-block-type.my-block1.reference }}"  # example with reference property for blocks

Additional context

No response

We faced a similar issue with Prefect’s templating resolving variables at deploy time. It limited our ability to inject dynamic values like flow_run.expected_start_time for scheduled flows at runtime.

An old discussion in Prefect forum was suggesting to basically modify the flows implementation to support special values resolved at runtime, which is tedious since we have many flows, and it cannot be easily generalized.

To work around this, we created a wrapper flow that uses Jinja2 templating with custom placeholders (e.g., {{! ... }}), which are evaluated at flow execution. After resolving the placeholders in the parameters, the wrapper triggers the real downstream flow as a subflow, passing the dynamically generated parameters.

This approach might help others until a more native solution is implemented.

PrefectHQ / prefect