install issues

[spencer411]

Just trying to install and run this on our cluster. When I run the command: snakemake -rp -s MAGpy --cores 1 --use-conda test

I get the following error, but everything else seems to run fine:

scripts/add_tax.py ran with some errors: File "/network/rit/lab/andamlab/bin/MAGpy/scripts/add_tax.py", line 95 k = names[l] TabError: inconsistent use of tabs and spaces in indentation

Further more if I move to the next step and run: snakemake -rp -s MAGpy --cores 16 --use-conda setup

It finishes very quickly, and reviewing the log file it says:

The flag 'directory' used in rule all is only valid for outputs, not inputs. Building DAG of jobs... Nothing to be done.

Am I missing something here with the install?

[mw55309]

Let me look into this, it's clearly not working

Can you tell me which version of python and Snakemake you're using?

Get Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: spencer411 @.> Sent: Tuesday, September 21, 2021 7:41:07 PM To: WatsonLab/MAGpy @.> Cc: Subscribed @.***> Subject: [WatsonLab/MAGpy] install issues (#39)

This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe.

Just trying to install and run this on our cluster. When I run the command: snakemake -rp -s MAGpy --cores 1 --use-conda test

I get the following error, but everything else seems to run fine:

scripts/add_tax.py ran with some errors: File "/network/rit/lab/andamlab/bin/MAGpy/scripts/add_tax.py", line 95 k = names[l] TabError: inconsistent use of tabs and spaces in indentation

Further more if I move to the next step and run: snakemake -rp -s MAGpy --cores 16 --use-conda setup

It finishes very quickly, and reviewing the log file it says:

The flag 'directory' used in rule all is only valid for outputs, not inputs. Building DAG of jobs... Nothing to be done.

Am I missing something here with the install?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/WatsonLab/MAGpy/issues/39, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AANR6UF7VG7647KTZ3FON6TUDDGUHANCNFSM5EPIX2ZQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

[spencer411]

If I activate the magpy_install env and type "python --version" its says Python 3.9.7. Nevertheless if I type conda info, it says python version : 3.8.8.final.0.

Looking back at when I ran conda env create -f envs/install.yaml it says I installed snakemake-6.8.0

The series of code I ran based on the github site looked like this, just ensure I am doing things correctly:

git clone https://github.com/WatsonLab/MAGpy.git cd MAGpy conda env create -f envs/install.yaml conda activate magpy_install snakemake -rp -s MAGpy --cores 1 --use-conda test snakemake -rp -s MAGpy --cores 16 --use-conda setup

Note that I am using miniconda3

Thanks!

[mw55309]

OK, latest commit should have solved most things

You need to run

rm -rf magpy_dbs

after running the tests, sorry about that

[spencer411]

Having a new issue here when I run "snakemake -rp -s MAGpy --cores 16 --use-conda setup".

It run fine for a while and then I get:

Database sequences 219174961 Database letters 75271144009 Database hash c95ef6a1a2e26795e19cb50f839b44e7 Total time 2711s [Wed Oct 6 09:58:53 2021] Error in rule setup: jobid: 0 output: magpy_dbs conda-env: /network/rit/lab/andamlab/bin/MAGpy/.snakemake/conda/12db78cf454bed3ea850367e3b9143d7 shell:

    mkdir -p magpy_dbs && cd magpy_dbs

    wget -q ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_trembl.fasta.gz
    diamond makedb --threads 4 --in uniprot_trembl.fasta.gz -d uniprot_trembl
    rm uniprot_trembl.fasta.gz

    # Sourmash
    wget -O genbank-k21.lca.json.gz -q https://osf.io/d7rv8/download
    wget -O genbank-k31.lca.json.gz -q https://osf.io/4f8n3/download
    wget -O genbank-k51.lca.json.gz -q https://osf.io/nemkw/download

    # Pfam
    wget -q ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz
    wget -q ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.dat.gz
    #wget -q ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/active_site.dat.gz
    gunzip Pfam-A.hmm.gz Pfam-A.hmm.dat.gz #active_site.dat.gz
    hmmpress Pfam-A.hmm

    # get checkM data
    mkdir checkm_data
    cd checkm_data
    wget -q https://data.ace.uq.edu.au/public/CheckM_databases/checkm_data_2015_01_16.tar.gz
    gunzip < checkm_data_2015_01_16.tar.gz | tar xvf -
    cd ..

    cd ..

    echo '{"uniprot_sprot": "magpy_dbs/uniprot_trembl.dmnd","sourmash_gbk_k21": "magpy_dbs/genbank-k21.lca.json.gz","sourmash_gbk_k31": "magpy_dbs/genbank-k31.lca.json.gz","sourmash_gbk_k51": "magpy_dbs/genbank-k51.lca.json.gz","pfam_dir": "magpy_dbs","checkm_dataroot": "magpy_dbs/checkm_data"}' > config.json

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job setup since they might be corrupted: magpy_dbs Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /network/rit/lab/andamlab/bin/MAGpy/.snakemake/log/2021-10-06T090641.681886.snakemake.log

[mw55309]

Ach that's annoying because it's going to delete everything and only one step went wrong.

I will try and split the steps and see what's happening

I also got an error, and tracked it down to the sourmash wget steps:

Resolving osf.io (osf.io)... 35.190.84.173
Connecting to osf.io (osf.io)|35.190.84.173|:443... connected.
ERROR: cannot verify osf.io's certificate, issued by ‘/C=US/O=Let's Encrypt/CN=R3’:
  Issued certificate has expired.
To connect to osf.io insecurely, use `--no-check-certificate'.

You could try that, or wait until I test this thing...

[mw55309]

I just pushed a new commit which splits up the steps, so shouldn't delete progress if just one of them fails

It all just ran fine for me :)