nat-n / poethepoet

A task runner that works well with poetry.
https://poethepoet.natn.io/
MIT License
1.4k stars 58 forks source link

Combination of `click` and `switch` command in `poe` #153

Closed amirhessam88 closed 1 year ago

amirhessam88 commented 1 year ago

Hello @nat-n I have a quick question and appreciate your thoughts. I am using the example below to describe my current issue.

I am currently using poe to invoke pipeline runs. My pipelines use click for parameterization and I am trying to combine switch command in poe with dynamic parameters (so far no luck).

Let's assume I have the following files in my pipeline namespace:


.
├── src
 │   └── pipelines
 │       ├── p1.py
 │       └── p2.py
└── pyproject.toml
# p1.py

import click

@click.command()
@click.option(
    "--foo",
    type=str,
)
@click.option(
    "--bar",
    type=str,
)
def main(foo: str, bar: str) -> int:
    s = f"pipeline ran with {foo=} and {bar=} ..."
    print(s)

    return 0

if __name__ == "__main__":
    raise SystemExit(main())

and

# p2.py

import click

@click.command()
@click.option(
    "--baz",
    type=str,
)
def main(baz: str) -> int:
    s = f"pipeline ran with {baz=} ..."
    print(s)

    return 0

if __name__ == "__main__":
    raise SystemExit(main())

So, normally if I wanna invoke the above pipelines in terminal, I would do:

# p1 invoke
poetry run python -m pipelines.p1 --foo="FOO" --bar="BAR"

# p2 invoke
poetry run python -m pipelines.p2 --baz="BAZ"

Previously, I was using script command in poe, but in this case I am not sure how can I have one general poe run command that covers all different params cases (something like **params)

I am looking to figure out a way to propagate the params into the switch commands; for example

# pyproject.toml

[tool.poe.tasks.run]
control.expr = "pipeline"
args = ["pipeline", "foo", "bar"]

[[tool.poe.tasks.run.switch]]
case = "p1"
# how to pass the args from terminal here ?
cmd = "python -m pipelines.p1"

So passing poe run --pipeline="p1" --foo="FOO" --bar="BAR" would be like

Poe <= pipeline
Poe => python -m pipelines.p1
pipeline ran with foo=None and bar=None ...

Now, I dont know how can I have a switch command that can handle both p1 and p2 and ... pn params hypothetically.

Appreciate your thoughts ....