I tried running pling on EBI cluster (installed using python -m pip install git+ssh://git@github.com/iqbal-lab-org/pling.git) on 11 genomes from https://github.com/LeahRoberts/roundhound/issues/162 , and got this issue:
$ pling genomes.txt impossible_to_find_transmissions align
Batching...
Building DAG of jobs...
Creating conda environment venv/lib/python3.9/site-packages/pling/envs/smash.yaml...
Downloading and installing remote packages.
Environment for /hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/batching/../envs/smash.yaml created (location: .snakemake/conda/f6927a7353b92b0cf9a92e4d5987aee8_)
Using shell: /usr/local/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job count
----------- -------
all 1
get_batches 1
total 2
Select jobs to execute...
[Mon Dec 18 16:26:23 2023]
rule get_batches:
output: impossible_to_find_transmissions/batches
jobid: 1
reason: Missing output files: impossible_to_find_transmissions/batches
resources: tmpdir=/hps/nobackup/iqbal/leandro/temp, mem_mb=4000, mem_mib=3815
Activating conda environment: .snakemake/conda/f6927a7353b92b0cf9a92e4d5987aee8_
Traceback (most recent call last):
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/numpy/core/__init__.py", line 24, in <module>
from . import multiarray
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/numpy/core/multiarray.py", line 10, in <module>
from . import overrides
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/numpy/core/overrides.py", line 8, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/numpy/__init__.py", line 130, in <module>
from numpy.__config__ import show as show_config
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/numpy/__config__.py", line 4, in <module>
from numpy.core._multiarray_umath import (
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/numpy/core/__init__.py", line 50, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.10 from "/hps/nobackup/iqbal/leandro/leah/pling/.snakemake/conda/f6927a7353b92b0cf9a92e4d5987aee8_/bin/python"
* The NumPy version is: "1.26.2"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/batching/get_batches.py", line 4, in <module>
import numpy as np
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/numpy/__init__.py", line 135, in <module>
raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
[Mon Dec 18 16:26:23 2023]
Error in rule get_batches:
jobid: 1
output: impossible_to_find_transmissions/batches
conda-env: /hps/nobackup/iqbal/leandro/leah/pling/.snakemake/conda/f6927a7353b92b0cf9a92e4d5987aee8_
shell:
PATH="$CONDA_PREFIX"/bin:$PATH
PYTHONPATH=/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages python /hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/batching/get_batches.py --genomes_list genomes.txt --batch_size 50 --outputpath impossible_to_find_transmissions --smash_threshold 1 --jaccardpath impossible_to_find_transmissions/tmp_files/jaccard_batchwise/not_pairs_jaccard_distance.tsv
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-18T162143.902069.snakemake.log
Command 'snakemake --snakefile /hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/batching/Snakefile --configfile impossible_to_find_transmissions/tmp_files/config.yaml --cores 1 --use-conda --use-singularity --rerun-incomplete ' returned non-zero exit status 1.
Traceback (most recent call last):
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/bin/pling", line 8, in <module>
sys.exit(main())
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/run_pling.py", line 179, in main
pling(args)
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/run_pling.py", line 129, in pling
raise e
File "/hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/run_pling.py", line 124, in pling
subprocess.run(f"snakemake --snakefile {get_pling_path()}/batching/Snakefile {snakemake_args}", shell=True, check=True, capture_output=True)
File "/hps/nobackup/iqbal/leandro/miniconda3/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'snakemake --snakefile /hps/nobackup/iqbal/leandro/leah/pling/venv/lib/python3.9/site-packages/pling/batching/Snakefile --configfile impossible_to_find_transmissions/tmp_files/config.yaml --cores 1 --use-conda --use-singularity --rerun-incomplete ' returned non-zero exit status 1.
... any idea how to fix? Did you have this before?
I tried running
plingon EBI cluster (installed usingpython -m pip install git+ssh://git@github.com/iqbal-lab-org/pling.git) on 11 genomes from https://github.com/LeahRoberts/roundhound/issues/162 , and got this issue:... any idea how to fix? Did you have this before?