Traceback error when running looper for tutorial data

[beginnertobioinformatics]

Hello,

I was going through the extended tutorial for PEPATAC to test out the pipeline with the tutorial data. However, when I get to the last step and try running looper run examples/tutorial/tutorial_refgenie.yaml I receive the following error:

Traceback (most recent call last):
  File "/home/usr/PEPATACTut_venv/bin/looper", line 8, in <module>
    sys.exit(main())
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/looper/looper.py", line 1115, in main
    run(args, rerun=(args.command == "rerun"), **compute_kwargs)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/looper/looper.py", line 501, in __call__
    curr_pl_fails = cndtr.add_sample(sample, rerun=rerun)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/looper/conductor.py", line 474, in add_sample
    self.submit()
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/looper/conductor.py", line 514, in submit
    script = self.write_script(self._pool, self._curr_size)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/looper/conductor.py", line 734, in write_script
    namespaces = _exec_pre_submit(pl_iface, namespaces)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/looper/conductor.py", line 848, in _exec_pre_submit
    _update_namespaces(namespaces, func(namespaces))
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/refgenconf/populator.py", line 45, in looper_refgenie_populate
    rgc = refgenconf.RefGenConf(rgc_path)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/refgenconf/refgenconf.py", line 96, in __init__
    super(RefGenConf, self).__init__(
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/yacman/yacman.py", line 125, in __init__
    file_contents = load_yaml(filepath)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/yacman/yacman.py", line 438, in load_yaml
    return read_yaml_file(filepath)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/yacman/yacman.py", line 418, in read_yaml_file
    with open(filepath, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '$REFGENIE'
Exception ignored in: <function YacAttMap.__del__ at 0x7f5078ecaef0>
Traceback (most recent call last):
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/yacman/yacman.py", line 159, in __del__
    if hasattr(self[IK], FILEPATH_KEY) and not getattr(self[IK], RO_KEY, True):
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/attmap/pathex_attmap.py", line 59, in __getitem__
    v = super(PathExAttMap, self).__getitem__(item)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/attmap/ordattmap.py", line 48, in __getitem__
    return AttMap.__getitem__(self, item)
  File "/home/usr/PEPATACTut_venv/lib/python3.10/site-packages/attmap/attmap.py", line 32, in __getitem__
    return self.__dict__[item]
KeyError: '__internal'

I first thought the error might have to do with the version of looper I had (1.4.0), but when I tried downgrading it, it didn't fix the issue. Also, for the other FileNotFound error for $REFGENIE, I did run the export command (export REFGENIE=/home/usr/PEPATACTut_venv/pepatac_tutorial/tools/refgenie_config.yaml) and confirmed that it had successfully run with echo $REFGENIE, but I still get this error. Are these errors related? Or tied to the fact that I'm running these commands in a virtual environment?

I'm very new to running command-line tools, python, etc., so any help or insight from the community is massively appreciated! Thank you!

[beginnertobioinformatics]

Hello, I tried running PEPATAC in a conda environment and got the same error initially, but when I downgraded looper to 1.3.1, it successfully passed the dry run but now this error appears--

Using default schema: /home/usr/pepatac/pipelines/pipestat_output_schema.yaml
Traceback (most recent call last):
  File "/home/usr/pepatac/pipelines/pepatac.py", line 2779, in <module>
    sys.exit(main())
  File "/home/usr/pepatac/pipelines/pepatac.py", line 683, in main
    report_message(pm, assets_file, message)
  File "/home/usr/pepatac/pipelines/pepatac.py", line 202, in report_message
    pm._safe_write_to_file(report_file, message)
AttributeError: 'PipelineManager' object has no attribute '_safe_write_to_file'

### Pipeline failed at:  (07-12 23:24:55) elapsed: 0.0 _TIME_

Total time: 0:00:01
Failure reason: Pipeline failure. See details above.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/usr/miniconda3/envs/pepatac/lib/python3.9/site-packages/pypiper/manager.py", line 2156, in _exit_handler
    self.fail_pipeline(Exception("Pipeline failure. See details above."))
  File "/home/usr/miniconda3/envs/pepatac/lib/python3.9/site-packages/pypiper/manager.py", line 2001, in fail_pipeline
    raise exc
Exception: Pipeline failure. See details above.

I'm thinking this is due to a package version I have that perhaps needs to be downgraded (assuming pypiper?), but I'm not sure what/what version?

[beginnertobioinformatics]

It turned out I had to downgrade piper to 0.12.3, the minimum version listed on the requirements text file. I still run into errors further downstream, but since I now don't think they're relevant to this original posting, I'll close this issue.

[donaldcampbelljr]

This is related to Pypiper (piper). Version v0.13.0 deprecated using _safe_write_to_file in favor of pipestat.report

[nsheff]

it would be nice if pypiper v0.13.0 would retain backwards compatibility, so that existing pipelines still work, while preferring the change to pipestat.report. What do you think about doing a bugfix release, pypiper 0.13.1, to restore backwards compatibility?