Hello @nat-n
I have a quick question and appreciate your thoughts. I am using the example below to describe my current issue.
I am currently using poe to invoke pipeline runs. My pipelines use click for parameterization and I am trying to combine switch command in poe with dynamic parameters (so far no luck).
Let's assume I have the following files in my pipeline namespace:
# p1.py
import click
@click.command()
@click.option(
"--foo",
type=str,
)
@click.option(
"--bar",
type=str,
)
def main(foo: str, bar: str) -> int:
s = f"pipeline ran with {foo=} and {bar=} ..."
print(s)
return 0
if __name__ == "__main__":
raise SystemExit(main())
and
# p2.py
import click
@click.command()
@click.option(
"--baz",
type=str,
)
def main(baz: str) -> int:
s = f"pipeline ran with {baz=} ..."
print(s)
return 0
if __name__ == "__main__":
raise SystemExit(main())
So, normally if I wanna invoke the above pipelines in terminal, I would do:
# p1 invoke
poetry run python -m pipelines.p1 --foo="FOO" --bar="BAR"
# p2 invoke
poetry run python -m pipelines.p2 --baz="BAZ"
Previously, I was using script command in poe, but in this case I am not sure how can I have one general poe run command that covers all different params cases (something like **params)
I am looking to figure out a way to propagate the params into the switch commands; for example
# pyproject.toml
[tool.poe.tasks.run]
control.expr = "pipeline"
args = ["pipeline", "foo", "bar"]
[[tool.poe.tasks.run.switch]]
case = "p1"
# how to pass the args from terminal here ?
cmd = "python -m pipelines.p1"
So passing poe run --pipeline="p1" --foo="FOO" --bar="BAR" would be like
Poe <= pipeline
Poe => python -m pipelines.p1
pipeline ran with foo=None and bar=None ...
Now, I dont know how can I have a switch command that can handle both p1 and p2 and ... pn params hypothetically.
Hello @nat-n I have a quick question and appreciate your thoughts. I am using the example below to describe my current issue.
I am currently using
poe
to invoke pipeline runs. My pipelines useclick
for parameterization and I am trying to combineswitch
command inpoe
with dynamic parameters (so far no luck).Let's assume I have the following files in my
pipeline
namespace:and
So, normally if I wanna invoke the above pipelines in terminal, I would do:
Previously, I was using
script
command inpoe
, but in this case I am not sure how can I have one generalpoe run
command that covers all differentparams
cases (something like**params
)I am looking to figure out a way to propagate the
params
into the switch commands; for exampleSo passing
poe run --pipeline="p1" --foo="FOO" --bar="BAR"
would be likeNow, I dont know how can I have a
switch
command that can handle bothp1
andp2
and ...pn
params hypothetically.Appreciate your thoughts ....