This is problematic for using pipestat in the looper_config file which is necessary for looper report and looper link.
This is because we can currently only choose one pipestat results file in the looper config.
Spawning separate stats files is default pypiper behavior that can be overridden using the pipestat_results_file parameter.
This allows for specifiying a single results file for the pipeline output:
This works well until the pipeline attempts to retrieve a stat via pm.get_stat. When it attempts to retrieve a result from a file that contains more than one samples, it errors.
Missing stat 'Raw_reads'
Traceback (most recent call last):
File "/home/drc/pepatac_tutorial//tools/pepatac/pipelines/pepatac.py", line 2784, in <module>
sys.exit(main())
File "/home/drc/pepatac_tutorial//tools/pepatac/pipelines/pepatac.py", line 1117, in main
pm.run([cmd, cmd2], rmdup_bam, follow=check_alignment_genome)
File "/home/drc/GITHUB/pepatac/pepatac/venv/lib/python3.10/site-packages/pypiper/manager.py", line 1093, in run
call_follow()
File "/home/drc/GITHUB/pepatac/pepatac/venv/lib/python3.10/site-packages/pypiper/manager.py", line 947, in call_follow
follow()
File "/home/drc/pepatac_tutorial//tools/pepatac/pipelines/pepatac.py", line 1106, in check_alignment_genome
rr = float(pm.get_stat("Raw_reads"))
TypeError: float() argument must be a string or a real number, not 'NoneType'
I believe the solution is to have pypiper instead use pipestat's retrieve_one. Perhaps get_stat can be a wrapper for this.
Actually, I was able to modify get_stat such that is does not only check the first sample in the results file vs the pipestat manager's record_identifier: 2de4e842d6ba555cc1f18473403c04085dc586b2
This is problematic for using pipestat in the looper_config file which is necessary for
looper report
andlooper link
. This is because we can currently only choose one pipestat results file in the looper config.Spawning separate stats files is default pypiper behavior that can be overridden using the
pipestat_results_file
parameter.This allows for specifiying a single results file for the pipeline output:
This works well until the pipeline attempts to retrieve a stat via
pm.get_stat
. When it attempts to retrieve a result from a file that contains more than one samples, it errors.I believe the solution is to have pypiper instead use pipestat's
retrieve_one
. Perhapsget_stat
can be a wrapper for this.Originally posted by @donaldcampbelljr in https://github.com/databio/pepatac/issues/257#issuecomment-1819897535