pepkit / looper

A job submitter for Portable Encapsulated Projects
http://looper.databio.org
BSD 2-Clause "Simplified" License
20 stars 7 forks source link

RecursionError #320

Closed nsheff closed 1 year ago

nsheff commented 3 years ago

Not sure if this is related to:

When on the cluster, I get this error. The same command works on my local computer on the same file....

looper run asset_pep/refgenie_build_cfg.yaml -p local --amend getfiles -d
Traceback (most recent call last):
  File "/home/ns5bc/.local/bin/looper", line 8, in <module>
    sys.exit(main())
  File "/home/ns5bc/.local/lib/python3.8/site-packages/looper/looper.py", line 932, in main
    args = enrich_args_via_cfg(args, aux_parser)
  File "/home/ns5bc/.local/lib/python3.8/site-packages/looper/utils.py", line 226, in enrich_args_via_cfg
    _get_subcommand_args(parser_args) \
  File "/home/ns5bc/.local/lib/python3.8/site-packages/looper/utils.py", line 260, in _get_subcommand_args
    cfg = peppyProject(parser_args.config_file,
  File "/home/ns5bc/.local/lib/python3.8/site-packages/peppy/project.py", line 59, in __init__
    self.parse_config_file(cfg, amendments)
  File "/home/ns5bc/.local/lib/python3.8/site-packages/peppy/project.py", line 129, in parse_config_file
    config = load_yaml(cfg_path) 
  File "/home/ns5bc/.local/lib/python3.8/site-packages/peppy/utils.py", line 132, in load_yaml
    return read_yaml_file(filepath)
  File "/home/ns5bc/.local/lib/python3.8/site-packages/peppy/utils.py", line 119, in read_yaml_file
    data = oyaml.safe_load(f)
  File "/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/__init__.py", line 162, in safe_load
    return load(stream, SafeLoader)
  File "/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/home/ns5bc/.local/lib/python3.8/site-packages/peppy/utils.py", line 148, in my_construct_mapping
    data = self.construct_mapping_org(node, deep)
  File "/home/ns5bc/.local/lib/python3.8/site-packages/yacman/yacman.py", line 31, in my_construct_mapping
    data = self.construct_mapping_org(node, deep)
  File "/home/ns5bc/.local/lib/python3.8/site-packages/yacman/yacman.py", line 31, in my_construct_mapping
    data = self.construct_mapping_org(node, deep)
  File "/home/ns5bc/.local/lib/python3.8/site-packages/yacman/yacman.py", line 31, in my_construct_mapping
    data = self.construct_mapping_org(node, deep)
  [Previous line repeated 982 more times]
RecursionError: maximum recursion depth exceeded

Relevant versions appear to match. Both are Python 3.8.5.

on rivanna:

 pip3 freeze | grep "yacman\|looper\|attmap"
attmap==0.13.0
looper @ file:///sfs/qumulo/qhome/ns5bc/code/looper
yacman==0.8.1

looper --version
looper 1.3.1-dev

local:

pip3 freeze | grep "yacman\|looper\|attmap"
attmap==0.13.0
looper==1.3.1.dev0
loopercli==0.12.6
yacman==0.8.1

having trouble tracking this down; why can't looper run on rivanna...

nsheff commented 3 years ago

Wow this is strange.

This works:python3 -c "import peppy; p = peppy.Project('asset_pep/refgenie_build_cfg.yaml')"

import peppy
p = peppy.Project("asset_pep/refgenie_build_cfg.yaml")

This does not work: python3 -c "import looper; import peppy; p = peppy.Project('asset_pep/refgenie_build_cfg.yaml')"

import looper
import peppy
p = peppy.Project("asset_pep/refgenie_build_cfg.yaml")
   ...: import looper
   ...: import peppy
   ...: p = peppy.Project("asset_pep/refgenie_build_cfg.yaml")
---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
<ipython-input-1-58741ae6196d> in <module>
      1 import looper
      2 import peppy
----> 3 p = peppy.Project("asset_pep/refgenie_build_cfg.yaml")

~/.local/lib/python3.8/site-packages/peppy/project.py in __init__(self, cfg, amendments, sample_table_index, subsample_table_index, defer_samples_creation)
     57         if isinstance(cfg, str):
     58             self[CONFIG_FILE_KEY] = cfg
---> 59             self.parse_config_file(cfg, amendments)
     60         else:
     61             self[CONFIG_FILE_KEY] = None

~/.local/lib/python3.8/site-packages/peppy/project.py in parse_config_file(self, cfg_path, amendments)
    127         if not os.path.exists(cfg_path) and not is_url(cfg_path):
    128             raise OSError(f"Project config file path does not exist: {cfg_path}")
--> 129         config = load_yaml(cfg_path)
    130         assert isinstance(
    131             config, Mapping

~/.local/lib/python3.8/site-packages/peppy/utils.py in load_yaml(filepath)
    130         return oyaml.safe_load(text)
    131     else:
--> 132         return read_yaml_file(filepath)
    133 
    134 

~/.local/lib/python3.8/site-packages/peppy/utils.py in read_yaml_file(filepath)
    117         filepath = os.path.abspath(filepath)
    118         with open(filepath, "r") as f:
--> 119             data = oyaml.safe_load(f)
    120         return data
    121 

/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/__init__.py in safe_load(stream)
    160     to be safe for untrusted input.
    161     """
--> 162     return load(stream, SafeLoader)
    163 
    164 def safe_load_all(stream):

/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/__init__.py in load(stream, Loader)
    112     loader = Loader(stream)
    113     try:
--> 114         return loader.get_single_data()
    115     finally:
    116         loader.dispose()

/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/constructor.py in get_single_data(self)
     49         node = self.get_single_node()
     50         if node is not None:
---> 51             return self.construct_document(node)
     52         return None
     53 

/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/constructor.py in construct_document(self, node)
     58             self.state_generators = []
     59             for generator in state_generators:
---> 60                 for dummy in generator:
     61                     pass
     62         self.constructed_objects = {}

/apps/software/standard/core/anaconda/2020.11-py3.8/lib/python3.8/site-packages/yaml/constructor.py in construct_yaml_map(self, node)
    411         data = {}
    412         yield data
--> 413         value = self.construct_mapping(node)
    414         data.update(value)
    415 

~/.local/lib/python3.8/site-packages/peppy/utils.py in my_construct_mapping(self, node, deep)
    146 # ordered by default.
    147 def my_construct_mapping(self, node, deep=False):
--> 148     data = self.construct_mapping_org(node, deep)
    149     return {
    150         (str(key) if isinstance(key, float) or isinstance(key, int) else key): data[key]

~/.local/lib/python3.8/site-packages/yacman/yacman.py in my_construct_mapping(self, node, deep)
     29 # ordered by default.
     30 def my_construct_mapping(self, node, deep=False):
---> 31     data = self.construct_mapping_org(node, deep)
     32     return {
     33         (str(key) if isinstance(key, float) or isinstance(key, int) else key): data[key]

... last 1 frames repeated, from the frame below ...

~/.local/lib/python3.8/site-packages/yacman/yacman.py in my_construct_mapping(self, node, deep)
     29 # ordered by default.
     30 def my_construct_mapping(self, node, deep=False):
---> 31     data = self.construct_mapping_org(node, deep)
     32     return {
     33         (str(key) if isinstance(key, float) or isinstance(key, int) else key): data[key]

RecursionError: maximum recursion depth exceeded
nsheff commented 3 years ago

It's these lines:

https://github.com/databio/yacman/blob/5be7237652f219f27f6361cb2379107f4a677858/yacman/yacman.py#L18-L35

Commenting out those lines, the error disappears.

nsheff commented 3 years ago

Found the culprit! It's these lines:

https://github.com/pepkit/peppy/blame/3f8a3ff604f0f5070b06c5dc9bb2daea1c2ae770/peppy/utils.py#L147-L166

@stolarczyk you duplicated this into peppy. That's now happening twice.

So with dev peppy, this breaks because the original mapping function gets patched in yacman, then patched again in peppy, leading to infinite recursion. so that makes peppy incompatible with yacman

nsheff commented 3 years ago

I added some code that restricts this patching to happen only once and this solves the issue for me.

But maybe we should think of a better solution to this problem as it's coming up now in multiple places. duplicating that code is probably not the best way forward...