PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
15.8k stars 1.55k forks source link

Allow using `flow` and `flow_run` properties in infra names #14646

Open rsundqvist opened 1 month ago

rsundqvist commented 1 month ago

First check

Describe the current behavior

User-chosen names are static. If not given, defaults to flow_run.name.

Describe the proposed behavior

Allow using any flow or flow_run parameter to set the infra name.

Example Use

Specifying a name.

from prefect_docker.worker import DockerWorkerJobConfiguration
DockerWorkerJobConfiguration(name="worker-1/{flow.name}/{flow_run.name}")

The main advantage is that it's easier to tell at a glance what's running, e.g. with docker stats:

CONTAINER ID   NAME                                  CPU %     MEM USAGE / LIMIT     MEM %
b46329f94fbf   worker-1_small-flow_phi310-wezn       34.69%    2.828GiB / 50.94GiB   5.55%
b3e8c1db8da5   worker-1_big-flow_mu5-argana-conflux  769.65%   3.128GiB / 50.94GiB   6.14%

Additional context

This seems pretty easy to implement, see https://github.com/PrefectHQ/prefect/compare/main...rsundqvist:prefect:main

aaazzam commented 1 month ago

Thanks @rsundqvist! This feels like a great first issue for the community, so I've marked it as such!

Nikeshhh commented 2 days ago

@aaazzam Hi, can i work on this issue?