Closed nsheff closed 3 weeks ago
This may actually be a bug with pipestat, rather than looper. I'm guessing this coincides with a switch to using pipestat for status checks.
I can't reproduce this using a modified hello_looper example for both the basic and the pipestat approaches (which look for their flags in slightly different ways).
However, looking at the basic, non-pipestat example, I can see where the function fetch_sample_flags
might have issues if you had a flag from a different sample in the results folder, because of this logic:
https://github.com/pepkit/looper/blob/1468956dde66abf5b853c80eaeaee2d411bfad64/looper/utils.py#L93-L98
Appears it is only concerned with .flag
and the pipeline_name
. The sample name doesn't matter.
was this fixed by the pipestat update referenced above?
I don't believe so. The pipestat code above was broken for filebackend and is not used for getting sample statuses.
Should be solved with the above commit.
I'm trying to submit 6 jobs with looper. I've never submitted any before, it's a brand new project. I noticed one of them says:
This is bizarre because it's a brand new project! It has never been submitted before.
I realize this sample shares a prefix with another sample: one is named
pairs_swap_maintain_coords
, which the pipeline runs, and then the next sample is namedpair_swap
-- which the pipeline incorrectly says is already completed.I'm guessing there's a regex that's looking for
{sample_name}*_completed.flag
-- if that's the case, it would actually register the first one as completed for the second one, and then never submit that job.