No job status flags in results folder

aaron-gu commented 3 years ago

Job status flags are not being generated for submitted jobs. Tested with looper v1.3.0 on hello looper project.

 $ looper check project/project_config.yaml
Looper version: 1.3.0
Command: check
COMPLETED: 0
FAILED: 0
PARTIAL: 0
RUNNING: 0
WAITING: 0

donaldcampbelljr commented 1 year ago

Right now, pypiper and pipestat can create status flags (pypiper actually uses pipestat to do this). However, if you are using looper independently, it does not create them. Looper can only check for created flags.

nsheff commented 1 year ago

Right, looper should only check flags -- not create them, right?

Well...unless we add a "submitted" flag...

donaldcampbelljr commented 1 year ago

If Looper is configured to use pipestat, it will default to checking for flags in the the results folder supplied via the project configuration under pipestat -> results_file_attribute

looper:
  all:
    output_dir: $HOME/hello_looper_results
  output_dir: $HOME/hello_looper_results
  pipestat:
    project:
      results_file_attribute: ~/GITHUB/looper/master/looper/hello_looper-master/results/results.yaml
      config_attribute: ../pipeline/drc_pipestat_config.yaml
      namespace_attribute: name_from_Project_config
    sample:
      results_file_attribute: ~/GITHUB/looper/master/looper/hello_looper-master/results/results2.yaml 
      config_attribute: ../pipeline/drc_pipestat_config.yaml
      namespace_attribute: name_from_Project_config

looper check ./hello_looper-master/project/project_config_pipestat_example.yaml
Looper version: 1.4.0-dev

     'count_lines' pipeline status summary     
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃   Status   ┃     Jobs count/total jobs      ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│   failed   │              1/5               │
└────────────┴────────────────────────────────┘

pepkit / looper

No job status flags in results folder #326