gwforg / gwf

A flexible, pragmatic workflow tool.
https://gwf.app/
GNU General Public License v3.0
31 stars 12 forks source link

Scheduler resubmits completed jobs (gwf v1.8.5) #400

Closed micknudsen closed 1 year ago

micknudsen commented 1 year ago

Here is a very simple workflow with two targets, foo and bar, where bar depends on foo:

from gwf import AnonymousTarget, Workflow

gwf = Workflow()

def foo() -> AnonymousTarget:

    inputs = {}
    outputs = {"foo": "foo.txt"}

    options = {"cores": 1,
               "memory": "1g",
               "walltime": "01:00:00"}

    spec = f"""

    touch {outputs["foo"]}

    """

    return AnonymousTarget(inputs=inputs, outputs=outputs, options=options, spec=spec)

def bar(foo: str) -> AnonymousTarget:

    inputs = {"foo": foo}
    outputs = {"bar": "bar.txt"}

    options = {"cores": 1,
               "memory": "1g",
               "walltime": "01:00:00"}

    spec = f"""

    touch {outputs["bar"]}

    """

    return AnonymousTarget(inputs=inputs, outputs=outputs, options=options, spec=spec)

foo_target = gwf.target_from_template(
    name="foo",
    template=foo()
)

gwf.target_from_template(
    name="bar",
    template=bar(foo=foo_target.outputs["foo"])
)

This looks fine:

$ gwf status
⨯ foo      0.00%    an output file is missing
⨯ bar      0.00%    a dependency was scheduled

Let's go ahead and submit the workflow:

$ gwf run
Submitting target foo (an output file is missing)
Submitting target bar (a dependency was scheduled)

and see how it went:

$ gwf status
✓ foo    100.00%    not scheduled because it is a source
✓ bar    100.00%    is up-to-date

Nice! But know this happens:

$ gwf run
Submitting target bar (is up-to-date)

Submitting a target with the reason that it is up-to-date? 🤔

dansondergaard commented 1 year ago

Fixed in recent commit, will be part of the next release.