search

dib-lab / dib_rotation

Metagenomics DIB-lab rotation project
https://dib-lab.github.io/dib_rotation/
BSD 3-Clause "New" or "Revised" License
3 stars 8 forks source link

issue creating spacegraphcats conda environment #26

Closed clmattson-zz closed 3 years ago

clmattson-zz commented 3 years ago

I can successfully git clone spacegraphcats to my directory, however, I encounter several errors when trying to activate a conda environment and run SGC. When creating the environment, the final lines of the output read Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work. . . . CondaEnvException: Pip failed. I can activate the environment anyway, but then when trying to run SGC, it says No module named spacegraphcats.__main__; 'spacegraphcats' is a package and cannot be directly executed

A shot in the dark, but the last line of the yaml in the spacegraphcats directory that is used to create the conda environment reads 

pip:
      - https://github.com/dib-lab/sourmash/archive/latest.zip

which doesn't appear to be a real location (and also references sourmash and not spacegraphcats). It is my (potentially incorrect) understanding that they are separate, so I thought this might be the source of the error.

ctb commented 3 years ago

Hi @clmattson, thanks - could you try replacing that https URL With the following and see if it works?

git+https://github.com/dib-lab/sourmash.git#egg=sourmash

Note that environment.yml is installing packages that spacegraphcats depends on. and one of those is sourmash - that's why it's in the environment.yml!

best, --titus

On Sat, Nov 21, 2020 at 07:12:20PM -0800, clmattson wrote:

I can successfully git clone spacegraphcats to my directory, however, I encounter several errors when trying to activate a conda environment and run SGC. When creating the environment, the final lines of the output read Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work. . . . CondaEnvException: Pip failed. I can activate the environment anyway, but then when trying to run SGC, it says No module named spacegraphcats.__main__; 'spacegraphcats' is a package and cannot be directly executed

A shot in the dark, but the last line of the yaml in the spacegraphcats directory that is used to create the conda environment reads 

pip:
      - https://github.com/dib-lab/sourmash/archive/latest.zip

which doesn't appear to be a real location (and also references sourmash and not spacegraphcats). It is my (potentially incorrect) understanding that they are separate, so I thought this might be the source of the error.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dib-lab/dib_rotation/issues/26 -- C. Titus Brown, ctbrown@ucdavis.edu

clmattson-zz commented 3 years ago

Ok, so just to be clear, my new environment.yml looks like this:

name: spacegraphcats channels:

(I tried creating the environment with this file and receive a similar error as before, CondaEnvException: Pip failed)

ctb commented 3 years ago

thanks, looking into it!

ctb commented 3 years ago

hi @clmattson with the following change --

diff --git a/environment.yml b/environment.yml
index 4f4a7c2..3d3e262 100644
--- a/environment.yml
+++ b/environment.yml
@@ -16,4 +16,4 @@ dependencies:
     - bbhash>=0.5.4
     - pip
     - pip:
-      - https://github.com/dib-lab/sourmash/archive/latest.zip
+      - git+https://github.com/dib-lab/sourmash.git#egg=sourmash

--- I managed to get everything to work on farm, with the command:

conda env create -n sgc-test5 -f environment.yml

If that isn't working for you, could you please share the command you're running and the full error message you're getting now? thanks!

clmattson-zz commented 3 years ago

I'm sorry, would you mind copying the complete contents of the .yml file that we should use? Some lines above don't appear to match the existing file.

-- Courtney Mattson Ph.D. Student Microbiology Graduate Group University of California, Davis (410)-610-9282

From: C. Titus Brown notifications@github.com Sent: Monday, November 23, 2020 12:07 PM To: dib-lab/dib_rotation dib_rotation@noreply.github.com Cc: Courtney Mattson clmattson@ucdavis.edu; Mention mention@noreply.github.com Subject: Re: [dib-lab/dib_rotation] issue creating spacegraphcats conda environment (#26)

hi @clmattsonhttps://github.com/clmattson with the following change --

diff --git a/environment.yml b/environment.yml index 4f4a7c2..3d3e262 100644 --- a/environment.yml +++ b/environment.yml @@ -16,4 +16,4 @@ dependencies:

--- I managed to get everything to work on farm, with the command:

conda env create -n sgc-test5 -f environment.yml

If that isn't working for you, could you please share the command you're running and the full error message you're getting now? thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/dib-lab/dib_rotation/issues/26#issuecomment-732296380, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARYPJMRU5CSCYNFR45QBB5TSRKJF3ANCNFSM4T6GEWWA.

ctb commented 3 years ago

hah, sorry, that's unified diff format, producing by editing the file and running git diff!

The full contents --

name: spacegraphcats
channels:
    - conda-forge
    - bioconda
    - defaults
dependencies:
    - bcalm=2.2.3
    - snakemake-minimal=5.20.1
    - screed=1.0.1
    - khmer=3.0.0a3
    - numpy
    - pytest=5.3.2
    - sortedcontainers=2.1.0
    - click=7.1.2
    - pyflakes
    - bbhash>=0.5.4
    - pip
    - pip:
      - git+https://github.com/dib-lab/sourmash.git#egg=sourmash
clmattson-zz commented 3 years ago

Thanks so much. Unfortunately, that didn't work either. I copied the above text into environment.yml and ran the line conda env create -n sgc-test6 -f spacegraphcats/environment.yml from the home directory.

The full output is very long, below are the sections that appear to be error messages: 

Building wheels for collected packages: sourmash
  Building wheel for sourmash (setup.py): started
  Building wheel for sourmash (setup.py): finished with status 'error'
  Running setup.py clean for sourmash
Failed to build sourmash
Installing collected packages: pillow, kiwisolver, cycler, python-dateutil, matplotl
    Running setup.py install for sourmash: started
    Running setup.py install for sourmash: finished with status 'error'
Pip subprocess error:
  ERROR: Command errored out with exit status 1:
   command: /home/cmatt5/miniconda3/envs/sgc-test6/bin/python -u -c 'import sys, set__file__='"'"'/tmp/pip-install-eyzdgeja/sourmash/setup.py'"'"';f=getattr(tokenize, 'se();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-c4
       cwd: /tmp/pip-install-eyzdgeja/sourmash/
  Complete output (90 lines):
copying sourmash/cli/lca/summarize.py -> build/lib/sourmash/cli/lca
  error: [Errno 2] No such file or directory: 'cargo'
  ----------------------------------------
  ERROR: Failed building wheel for sourmash
    ERROR: Command errored out with exit status 1:
     command: /home/cmatt5/miniconda3/envs/sgc-test6/bin/python -u -c 'import sys, s; __file__='"'"'/tmp/pip-install-eyzdgeja/sourmash/setup.py'"'"';f=getattr(tokenize,lose();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-recoeaders /home/cmatt5/miniconda3/envs/sgc-test6/include/python3.8/sourmash
         cwd: /tmp/pip-install-eyzdgeja/sourmash/
  error: [Errno 2] No such file or directory: 'cargo'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/cmatt5/miniconda3/envs/sgc-testl-eyzdgeja/sourmash/setup.py'"'"'; __file__='"'"'/tmp/pip-install-eyzdgeja/sourmash/'"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"')lly-managed --compile --install-headers /home/cmatt5/miniconda3/envs/sgc-test6/inclu

failed

CondaEnvException: Pip failed

Thank you!

ctb commented 3 years ago

ahh, interesting, thanks, that helps a lot!

next question is what to do about it... :) back to ya soon.

taylorreiter commented 3 years ago

addressed by #27 and #29