search

jiarong / VirSorter2

customizable pipeline to identify viral sequences from (meta)genomic data
GNU General Public License v2.0
225 stars 31 forks source link

libmamba Non-conda folder exists at prefix #222

Open joelive opened 1 month ago

joelive commented 1 month ago

When I run the virsorter setup command, it always report error.

(virsort2) bo@bioinofo2:~$ virsorter setup -d virsorter_db/ [2024-10-24 02:47 INFO] VirSorter 2.2.4 [2024-10-24 02:47 INFO] /home/bo/miniconda3/envs/virsort2/bin/virsorter setup -d virsorter_db/ [2024-10-24 02:47 INFO] Setting up VirSorter2 database; this might take ~10 mins and only needs to be done once. [2024-10-24 02:47 INFO] saving /home/bo/virsorter_db as DBDIR to config file /home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/virsorter/template-config.yaml CreateCondaEnvironmentException: Could not create conda environment from /home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/virsorter/rules/../envs/vs2.yaml: error libmamba Non-conda folder exists at prefix critical libmamba Aborting.

File "/home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/snakemake/deployment/conda.py", line 350, in create [2024-10-24 02:47 ERROR] CreateCondaEnvironmentException: Could not create conda environment from /home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/virsorter/rules/../envs/vs2.yaml: error libmamba Non-conda folder exists at prefix critical libmamba Aborting.

File "/home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/snakemake/deployment/conda.py", line 350, in create [2024-10-24 02:47 INFO] First attempt failed; trying the second time. CreateCondaEnvironmentException: Could not create conda environment from /home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/virsorter/rules/../envs/vs2.yaml: error libmamba Non-conda folder exists at prefix critical libmamba Aborting.

File "/home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/snakemake/deployment/conda.py", line 350, in create [2024-10-24 02:47 CRITICAL] Command 'snakemake --snakefile /home/bo/miniconda3/envs/virsort2/lib/python3.10/site-packages/virsorter/rules/setup-retry.smk --directory /home/bo/virsorter_db --quiet --config Skip_deps_install=False --jobs 64 --rerun-incomplete --latency-wait 600 --nolock --use-conda --conda-prefix /home/bo/virsorter_db/conda_envs --conda-frontend mamba ' returned non-zero exit status 1.

ChaoLab commented 1 month ago

Hi Jiarong, I have the same issue as @joelive. I guess it is a conda env setting-up problem. Can you help to solve this issue?

Best, Chao

jiarong commented 4 weeks ago

Sorry, I was quite busy last week. If you do not have to conda/mamba, I recommend installation option 3. Or else, try to remove the database directory (/home/bo/virsorter_db) and rerun virsorter setup. It looks like there was an failed attempt before.

handibles commented 3 weeks ago

Seeing this also on a fresh env. After failed installs, there's a conda_envs dir created in the chosen db dir, possibly mamba is attempting to install the setup.smk dependencies alongside the database, which is going awry (i.e. non-conda folder). See line:471 of virsorter.py, and possibly line:155 of setup.smk.

In any case: doing all of the below:

allowed setup to complete, but does not allow virsorter to run - the error is ~same as the above:

[2024-10-31 16:21 INFO] /mnt/workspace2/jamie/miniforge3/envs/pha/bin/virsorter run -w /mnt/workspace2/jamie/mtu__phamm/6__virsorter/vsort_SPA_ERR10157887 -i /mnt/workspace2/jamie/mtu__phamm/4__spades/ERR10157887/contigs.fasta --min-length 10000 -j 8                                                                                                     
[2024-10-31 16:21 INFO] Using /mnt/workspace2/jamie/miniforge3/envs/pha/lib/python3.10/site-packages/virsorter/template-config.yaml as config template                                                                                    
[2024-10-31 16:21 INFO] conig file written to /mnt/workspace2/jamie/mtu__phamm/6__virsorter/vsort_SPA_ERR10157887/config.yaml                                                                                                             

[2024-10-31 16:21 INFO] Executing: snakemake --snakefile /mnt/workspace2/jamie/miniforge3/envs/pha/lib/python3.10/site-packages/virsorter/Snakefile --directory /mnt/workspace2/jamie/mtu__phamm/6__virsorter/vsort_SPA_ERR10157887 --jobs 8 --configfile /mnt/workspace2/jamie/mtu__phamm/6__virsorter/vsort_SPA_ERR10157887/config.yaml --latency-wait 600 --rerun-incomplete --nolock  --conda-frontend mamba --conda-prefix /mnt/workspace2/jamie/vdb/conda_envs --use-conda    --quiet  all                                                                                                         
CreateCondaEnvironmentException:                                                                                     
Could not create conda environment from /mnt/workspace2/jamie/miniforge3/envs/pha/lib/python3.10/site-packages/virsorter/rules/../envs/vs2.yaml:                                                                                          
error    libmamba Non-conda folder exists at prefix                                                                  
critical libmamba Aborting.                                                                                          

  File "/mnt/workspace2/jamie/miniforge3/envs/pha/lib/python3.10/site-packages/snakemake/deployment/conda.py", line 350, in create
joelive commented 3 weeks ago

I have solved the problem by mamually mamba install all packages

handibles commented 3 weeks ago

I have solved the problem by mamually mamba install all packages

@joelive glad it works! My comment above doesn't allow you to run virsorter, as it still attempts a conda-env when you use virsorter run, and then it crashes - did you need to do anything else to stop conda getting upset?

jiarong commented 3 weeks ago

@all, I just had chance to test this. It looks like the new mamba version 2 is the issue. You can change to mamba version 1 to resolve this issue.

# get into the vs2 env
conda activate vs2
# downgrade mamba to v1
mamba installl -c conda-forge mamba=1
anniedussault commented 2 weeks ago

thanks @jiarong thanks for the solution! I was able to install your app and run it!

sohini5981 commented 4 days ago

Hello @jiarong

I have installed vs2 using mamba (ver 1.5.9) base * /storage/home/svg5863/miniforge3 vs2 /storage/home/svg5863/miniforge3/envs/vs2. However, the problem @joelive faced persists for me. When I try to setup the db folder, the job exists with the following errors:

/storage/home/svg5863/miniforge3/envs/vs2/bin/virsorter setup -d db -j 4 [2024-11-22 13:05 INFO] Setting up VirSorter2 database; this might take ~10 mins and only needs to be done once. [2024-11-22 13:05 INFO] saving /storage/home/svg5863/db as DBDIR to config file /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/virsorter/template-config.yaml CreateCondaEnvironmentException: Could not create conda environment from /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/virsorter/rules/../envs/vs2.yaml: /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/argparse.py:1930: FutureWarning: remote_definition is deprecated and will be removed in 25.9. Use conda env create --file=URL instead. action(self, namespace, argument_values, option_string) Collecting package metadata (repodata.json): ...working... File "/storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/snakemake/deployment/conda.py", line 350, in create [2024-11-22 13:06 ERROR] CreateCondaEnvironmentException: Could not create conda environment from /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/virsorter/rules/../envs/vs2.yaml: /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/argparse.py:1930: FutureWarning: remote_definition is deprecated and will be removed in 25.9. Use conda env create --file=URL instead. action(self, namespace, argument_values, option_string) Collecting package metadata (repodata.json): ...working... File "/storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/snakemake/deployment/conda.py", line 350, in create [2024-11-22 13:06 INFO] First attempt failed; trying the second time. CreateCondaEnvironmentException: Could not create conda environment from /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/virsorter/rules/../envs/vs2.yaml: /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/argparse.py:1930: FutureWarning: remote_definition is deprecated and will be removed in 25.9. Use conda env create --file=URL instead. action(self, namespace, argument_values, option_string) Collecting package metadata (repodata.json): ...working... File "/storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/snakemake/deployment/conda.py", line 350, in create [2024-11-22 13:07 CRITICAL] Command 'snakemake --snakefile /storage/home/svg5863/miniforge3/envs/vs2/lib/python3.10/site-packages/virsorter/rules/setup-retry.smk --directory /storage/home/svg5863/db --quiet --config Skip_deps_install=False --jobs 4 --rerun-incomplete --latency-wait 600 --nolock --use-conda --conda-prefix /storage/home/svg5863/db/conda_envs --conda-frontend mamba ' returned non-zero exit status 1. Any advice??

anniedussault commented 3 days ago

@sohini5981 Have you tried the solution here ? https://github.com/jiarong/VirSorter2/issues/222#issuecomment-2452806948

I had a smilar issue to what you're describing and this resolved my issue. I would start with that!

jiarong commented 3 days ago

@sohini5981 Have you tried the solution here ? #222 (comment)

I had a smilar issue to what you're describing and this resolved my issue. I would start with that!

Right, it's the same issue. The trick is to change mamba within the VS2 env. The problem is the mamba within the VS2 env, not the mamba in your base env.

sohini5981 commented 3 days ago

Thank you all for addressing the issue. I did change the mamba ver within the vs2 env. It did not help. Hence, I used the aptainer method to install the whole thing. That worked and hopefully now I will be able to run it. Thank you all.

On Fri, Nov 22, 2024, 7:39 PM jiarong @.***> wrote:

@sohini5981 https://github.com/sohini5981 Have you tried the solution here ? #222 (comment) https://github.com/jiarong/VirSorter2/issues/222#issuecomment-2452806948

I had a smilar issue to what you're describing and this resolved my issue. I would start with that!

Right, it's the same issue. The trick is to change mamba within the VS2 env. The problem is the mamba within the VS2 env, not the mamba in your base env.

— Reply to this email directly, view it on GitHub https://github.com/jiarong/VirSorter2/issues/222#issuecomment-2495153337, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJXQUR3ATFKP4AYKA2UA2BD2B7FFBAVCNFSM6AAAAABQQCTYRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJVGE2TGMZTG4 . You are receiving this because you were mentioned.Message ID: @.***>