kundajelab / atac_dnase_pipelines

ATAC-seq and DNase-seq processing pipeline
BSD 3-Clause "New" or "Revised" License
161 stars 81 forks source link

NotADirectoryError -- looks to have misparsed a .py as a dir #71

Closed jet456 closed 7 years ago

jet456 commented 7 years ago

Hi!

We updated our pipeline and are getting an unusual error. The pipeline won't run all the way through.

Thanks--

Traceback (most recent call last):
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 95, in updatecache
bash-4.1$     stat = os.stat(fullname)
NotADirectoryError: [Errno 20] Not a directory: '/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_v
endor/pyparsing.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/bin/conda", line 6, in <module>
    sys.exit(conda.cli.main())
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/main.py", line 154, in main
    activate.main()
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 130, in main
    print(get_activate_path(shelldict))
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 109, in get_activate_path
    binpath = binpath_from_arg(sys.argv[3], shelldict=shelldict)
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 66, in binpath_from_arg
    prefix = prefix_from_arg(arg, shelldict=shelldict)
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 48, in prefix_from_arg
    from conda.base.context import context, locate_prefix_by_name
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/base/context.py", line 17, in <module>
    from .._vendor.auxlib.path import expand
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/_vendor/auxlib/path.py", line 8, in <module>
    import pkg_resources
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 72, in <module>
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/packaging/requirements.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/extern/__init__.py", line 43, in load_module 
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 4615, in <module>
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1257, in setParseAction
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1039, in _trim_arity
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1023, in extract_stack
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 207, in extract_stack
    stack = StackSummary.extract(walk_stack(f), limit=limit)
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 358, in extract
....
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/base/context.py", line 17, in <module>
    from .._vendor.auxlib.path import expand
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/_vendor/auxlib/path.py", line 8, in <module>
    import pkg_resources
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 72, in <module>
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/packaging/requirements.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/extern/__init__.py", line 43, in load_module 
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 4615, in <module>
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1257, in setParseAction
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1039, in _trim_arity
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1023, in extract_stack
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 207, in extract_stack
    stack = StackSummary.extract(walk_stack(f), limit=limit)
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 358, in extract
    f.line
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 282, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 16, in getline
    lines = getlines(filename, module_globals)
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 47, in getlines
    return updatecache(filename, module_globals)
  File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 103, in updatecache
    data = cache[filename][0]()
leepc12 commented 7 years ago

Did you try to run pipelines on your computer or on a cluster?

Can you try a clean installation for miniconda and its dependencies?

1) remove your local miniconda from $PATH in bash init. scripts (~/.bash_rc or ~/.bash_profile) and then re-login.

2) install miniconda following the instruction on the pipeline github repo ( https://github.com/kundajelab/atac_dnase_pipelines#conda).

3) install dependencies on new miniconda ( https://github.com/kundajelab/atac_dnase_pipelines#dependencies).

4) try again

Thanks,

Jin

On Thu, Oct 5, 2017 at 4:22 AM, jet456 notifications@github.com wrote:

Hi!

We updated our pipeline and are getting an unusual error. The pipeline won't run all the way through.

Thanks--

Traceback (most recent call last): File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 95, in updatecache bash-4.1$ stat = os.stat(fullname) NotADirectoryError: [Errno 20] Not a directory: '/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_v endor/pyparsing.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/bin/conda", line 6, in sys.exit(conda.cli.main()) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/main.py", line 154, in main activate.main() File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 130, in main print(get_activate_path(shelldict)) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 109, in get_activate_path binpath = binpath_from_arg(sys.argv[3], shelldict=shelldict) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 66, in binpath_from_arg prefix = prefix_from_arg(arg, shelldict=shelldict) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/cli/activate.py", line 48, in prefix_from_arg from conda.base.context import context, locate_prefix_by_name File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/base/context.py", line 17, in from .._vendor.auxlib.path import expand File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/_vendor/auxlib/path.py", line 8, in import pkg_resources File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/init.py", line 72, in File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/packaging/requirements.py", line 9, in File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/extern/init.py", line 43, in load_module File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 4615, in File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1257, in setParseAction File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1039, in _trim_arity File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1023, in extract_stack File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 207, in extract_stack stack = StackSummary.extract(walk_stack(f), limit=limit) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 358, in extract .... File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/base/context.py", line 17, in from .._vendor.auxlib.path import expand File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/conda/_vendor/auxlib/path.py", line 8, in import pkg_resources File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/init.py", line 72, in File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/packaging/requirements.py", line 9, in File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/extern/init.py", line 43, in load_module File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 4615, in File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1257, in setParseAction File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1039, in _trim_arity File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/_vendor/pyparsing.py", line 1023, in extract_stack File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 207, in extract_stack stack = StackSummary.extract(walk_stack(f), limit=limit) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 358, in extract f.line File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/traceback.py", line 282, in line self._line = linecache.getline(self.filename, self.lineno).strip() File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 16, in getline lines = getlines(filename, module_globals) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 47, in getlines return updatecache(filename, module_globals) File "/app/miniconda3-4.3.11@i86-rhel6.0/miniconda3/lib/python3.6/linecache.py", line 103, in updatecache data = cache[filename][0]()

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kundajelab/atac_dnase_pipelines/issues/71, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOd_FaRCVRhYARRxb4rGNJlBYL5LR3wks5so9rtgaJpZM4PuG9P .

jet456 commented 7 years ago

Hi Jin,

Thanks for the quick response! We're running the pipeline on a cluster.

I've reinstalled miniconda on our cluster and asked the people running the pipeline to check their bash profiles and retest. We can close this for now.

Thanks, Joy