pepkit / peppy

Project metadata manager for PEPs in Python
https://pep.databio.org/peppy
BSD 2-Clause "Simplified" License
37 stars 13 forks source link

Duplicated error message upon project load #483

Open nsheff opened 1 year ago

nsheff commented 1 year ago

Using the example pep in the dnameth pipeline, I see:

looper check
No project config defined (peppy), using: {'config_file': 'test_config.yaml', 'output_dir': '.', 'sample_pipeline_interfaces': None, 'project_pipeline_interfaces': None}. Read from dotfile (/home/nsheff/code/dnameth_pipelines/examples/.looper.yaml).
Config file does not have version key. Defaulting to 2.1.0
Looper version: 1.5.2-dev
Command: check
Config file does not have version key. Defaulting to 2.1.0
Not all environment variables were populated in derived attribute source: $CODE/microtest/data/{filename}
Not all environment variables were populated in derived attribute source: $CODE/microtest/data/{filename}
Not all environment variables were populated in derived attribute source: $CODE/microtest/data/{filename}
Not all environment variables were populated in derived attribute source: $CODE/microtest/data/{filename}
Traceback (most recent call last):
  File "/home/nsheff/.local/bin/looper", line 8, in <module>
    sys.exit(main())
  File "/home/nsheff/.local/lib/python3.8/site-packages/looper/looper.py", line 1125, in main
    p = Project(
  File "/home/nsheff/.local/lib/python3.8/site-packages/looper/project.py", line 118, in __init__
    setattr(self[EXTRA_KEY], attr_name, kwargs[attr_name])
  File "/home/nsheff/.local/lib/python3.8/site-packages/peppy/project.py", line 1365, in __getitem__
    return self._project_data[item]
KeyError: '_cli_extra'

Why does it say: Config file does not have version key. Defaulting to 2.1.0 two times?

is looper loading the PEP twice?

nsheff commented 1 year ago

Ok, I don't think it's loading twice, I just think there are two error message for some reason -- also, one is not running through logmuse.

looper --verbosity 5 check
No project config defined (peppy), using: {'config_file': 'test_config.yaml', 'output_dir': '.', 'sample_pipeline_interfaces': None, 'project_pipeline_interfaces': None}. Read from dotfile (/home/nsheff/code/dnameth_pipelines/examples/.looper.yaml).
Config file does not have version key. Defaulting to 2.1.0
DEBU 11:41:39 | root:est:310 > Configured logger 'root' using logmuse v0.2.7 
INFO 11:41:39 | root:looper:1092 > Looper version: 1.5.2-dev
Command: check 
DEBU 11:41:39 | peppy:project:110 > Creating Project from file test_config.yaml 
DEBU 11:41:39 | peppy.utils:utils:148 > Creating a Project from a YAML file: test_config.yaml 
DEBU 11:41:39 | peppy:project:335 > Raw (test_config.yaml) config data: {'sample_table': 'microtest_annotation.csv', 'looper': {'output_dir': '$HOME/dnameth'}, 'sample_modifiers': {'append': {'pipeline_interfaces': '../piface_rrbs.yaml'}, 'imply': [{'if': {'organism': 'human'}, 'then': {'genome': 'hg38'}}, {'if': {'organism': 'mouse'}, 'then': {'genome': 'mm10'}}], 'derive': {'attributes': ['read1', 'read2'], 'sources': {'test_data_R1': 'examples/test_data/{sample_name}_R1.fastq.gz', 'test_data_R2': 'examples/test_data/{sample_name}_R2.fastq.gz', 'microtest': '$CODE/microtest/data/{filename}'}}}} 
DEBU 11:41:39 | peppy:project:368 > Adding attributes: sample_table, looper, sample_modifiers 
WARN 11:41:39 | peppy:project:1281 > Config file does not have version key. Defaulting to 2.1.0 
...

see how the first of the message lacks any logmuse styling?

nsheff commented 1 year ago

Also, it should specify which config (that's a peppy thing)

donaldcampbelljr commented 8 months ago

Could you clarify the exact issue? It appears that the original title is not the actual issue.

Is it ensuring all information is displayed in the terminal via logmuse?

Running with newest version:

(venv) drc@databio:~/GITHUB/hello_looper/hello_looper$ looper --verbosity 5 check --looper-config .looper_pipestat.yaml
DEBU 14:10:08 | root:est:310 > Configured logger 'root' using logmuse v0.2.7 
INFO 14:10:08 | root:cli_looper:614 > Looper version: 1.6.0
Command: check 
DEBU 14:10:08 | peppy:project:113 > Creating Project from file /home/drc/GITHUB/hello_looper/hello_looper/./project/project_config.yaml 
DEBU 14:10:08 | peppy.utils:utils:172 > Creating a Project from a YAML file: /home/drc/GITHUB/hello_looper/hello_looper/./project/project_config.yaml 

Etc.
saanikat commented 6 months ago

After discussion, it was discovered that this issue lies in peppy.

khoroshevskyi commented 2 months ago

This issue should be solved together with #465