Closed famiji closed 1 year ago
Hi,
I have no problem with your command on my installation. Do you have internet in this computer? Because the command download the database from cgsmlt.org. Or do you have permission on the folder you want to create the database?
wgmlst import Cp.db "Clostridium perfringens" --force
[INFO: 2023-03-02 08:56:08,657] One element found : Clostridium perfringens
[INFO: 2023-03-02 08:56:08,657] Downloading the core genome...
[INFO: 2023-03-02 08:56:11,085] Reverse-complemented genes: 926
[INFO: 2023-03-02 08:56:11,085] Database initialized
Benoit
I'm sure I have no problems connecting to the network and creating files Is there a problem with dependency?
+ _libgcc_mutex 0.1 conda_forge conda-forge/linux-64 Cached
+ _openmp_mutex 4.5 2_gnu conda-forge/linux-64 Cached
+ alembic 1.9.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ attrs 22.2.0 pyh71513ae_0 conda-forge/noarch Cached
+ backports 1.0 pyhd8ed1ab_3 conda-forge/noarch Cached
+ backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ beautifulsoup4 4.11.2 pyha770c72_0 conda-forge/noarch Cached
+ biopython 1.81 py311h2582759_0 conda-forge/linux-64 Cached
+ brotlipy 0.7.0 py311hd4cff14_1005 conda-forge/linux-64 Cached
+ bzip2 1.0.8 h7f98852_4 conda-forge/linux-64 Cached
+ ca-certificates 2022.12.7 ha878542_0 conda-forge/linux-64 Cached
+ certifi 2022.12.7 pyhd8ed1ab_0 conda-forge/noarch Cached
+ cffi 1.15.1 py311h409f033_3 conda-forge/linux-64 Cached
+ charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ click 8.1.3 unix_pyhd8ed1ab_2 conda-forge/noarch Cached
+ colorama 0.4.6 pyhd8ed1ab_0 conda-forge/noarch Cached
+ cryptography 39.0.0 py311h211fabd_0 conda-forge/linux-64 Cached
+ decorator 5.1.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ exceptiongroup 1.1.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ greenlet 2.0.2 py311hcafe171_0 conda-forge/linux-64 Cached
+ idna 3.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ importlib-metadata 6.0.0 pyha770c72_0 conda-forge/noarch Cached
+ importlib_resources 5.12.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ kma 1.4.9 h7132678_0 bioconda/linux-64 Cached
+ ld_impl_linux-64 2.40 h41732ed_0 conda-forge/linux-64 Cached
+ libblas 3.9.0 16_linux64_openblas conda-forge/linux-64 Cached
+ libcblas 3.9.0 16_linux64_openblas conda-forge/linux-64 Cached
+ libffi 3.4.2 h7f98852_5 conda-forge/linux-64 Cached
+ libgcc-ng 12.2.0 h65d4601_19 conda-forge/linux-64 Cached
+ libgfortran-ng 12.2.0 h69a702a_19 conda-forge/linux-64 Cached
+ libgfortran5 12.2.0 h337968e_19 conda-forge/linux-64 Cached
+ libgomp 12.2.0 h65d4601_19 conda-forge/linux-64 Cached
+ liblapack 3.9.0 16_linux64_openblas conda-forge/linux-64 Cached
+ libnsl 2.0.0 h7f98852_0 conda-forge/linux-64 Cached
+ libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge/linux-64 Cached
+ libpng 1.6.39 h753d276_0 conda-forge/linux-64 Cached
+ libsqlite 3.40.0 h753d276_0 conda-forge/linux-64 Cached
+ libstdcxx-ng 12.2.0 h46fd767_19 conda-forge/linux-64 Cached
+ libuuid 2.32.1 h7f98852_1000 conda-forge/linux-64 Cached
+ libzlib 1.2.13 h166bdaf_4 conda-forge/linux-64 Cached
+ mafft 7.515 hec16e2b_0 bioconda/linux-64 Cached
+ mako 1.2.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ markupsafe 2.1.2 py311h2582759_0 conda-forge/linux-64 Cached
+ mysql-connector-c 6.1.11 h6eb9d5d_1007 conda-forge/linux-64 Cached
+ ncurses 6.3 h27087fc_1 conda-forge/linux-64 Cached
+ networkx 3.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ numpy 1.24.2 py311h8e6699e_0 conda-forge/linux-64 Cached
+ openssl 1.1.1t h0b41bf4_0 conda-forge/linux-64 Cached
+ packaging 23.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pandas 1.5.3 py311h2872171_0 conda-forge/linux-64 Cached
+ pip 23.0.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pluggy 1.0.0 pyhd8ed1ab_5 conda-forge/noarch Cached
+ prompt-toolkit 3.0.38 pyha770c72_0 conda-forge/noarch 269kB
+ prompt_toolkit 3.0.38 hd8ed1ab_0 conda-forge/noarch 6kB
+ pycparser 2.21 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pymlst 2.1.3 pyh5e36f6f_0 bioconda/noarch Cached
+ pyopenssl 23.0.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pysocks 1.7.1 pyha2e5f31_6 conda-forge/noarch Cached
+ pytest 7.2.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ python 3.11.0 h10a6764_1_cpython conda-forge/linux-64 Cached
+ python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge/noarch Cached
+ python_abi 3.11 3_cp311 conda-forge/linux-64 Cached
+ pytz 2022.7.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ questionary 1.10.0 pyhd8ed1ab_1 conda-forge/noarch Cached
+ readline 8.1.2 h0f457ee_0 conda-forge/linux-64 Cached
+ requests 2.28.2 pyhd8ed1ab_0 conda-forge/noarch Cached
+ setuptools 67.4.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ six 1.16.0 pyh6c4a22f_0 conda-forge/noarch Cached
+ soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ sqlalchemy 2.0.4 py311h2582759_0 conda-forge/linux-64 Cached
+ tk 8.6.12 h27826a3_0 conda-forge/linux-64 Cached
+ tomli 2.0.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ typing-extensions 4.4.0 hd8ed1ab_0 conda-forge/noarch Cached
+ typing_extensions 4.4.0 pyha770c72_0 conda-forge/noarch Cached
+ tzdata 2022g h191b570_0 conda-forge/noarch Cached
+ ucsc-blat 377 ha8a8165_4 bioconda/linux-64 Cached
+ urllib3 1.26.14 pyhd8ed1ab_0 conda-forge/noarch Cached
+ wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge/noarch Cached
+ wheel 0.38.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ xz 5.2.6 h166bdaf_0 conda-forge/linux-64 Cached
+ zipp 3.15.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ zlib 1.2.13 h166bdaf_4 conda-forge/linux-64 Cached
No idea.
Download this file:
wget https://raw.githubusercontent.com/bvalot/pyMLST/master/tests/data/wg/cgmlst_schema.fasta
and try this command:
wgMLST create E.coli.db cgmlst_schema.fasta
I try it, this is the result.
(pymlst) fmj@qood:~$ wget https://raw.githubusercontent.com/bvalot/pyMLST/master/tests/data/wg/cgmlst_schema.fasta
--2023-03-02 16:27:26-- https://raw.githubusercontent.com/bvalot/pyMLST/master/tests/data/wg/cgmlst_schema.fasta
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.108.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2367809 (2.3M) [text/plain]
Saving to: ‘cgmlst_schema.fasta’
cgmlst_schema.fasta 100%[===================================================================>] 2.26M 62.3KB/s in 62s
2023-03-02 16:33:17 (37.3 KB/s) - ‘cgmlst_schema.fasta’ saved [2367809/2367809]
(pymlst) fmj@qood:~$ wgMLST create E.coli.db cgmlst_schema.fasta
Traceback (most recent call last):
File "/home/fmj/miniconda3/envs/pymlst/bin/wgMLST", line 10, in <module>
sys.exit(wg())
^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/pymlst/wg/commands/create.py", line 33, in cli
with pymlst.open_wg(os.path.abspath(database)) as mlst:
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/pymlst/wg/core.py", line 40, in open_wg
mlst = WholeGenomeMLST(file, ref)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/pymlst/wg/core.py", line 442, in __init__
self.__database = DatabaseWG(file, ref)
^^^^^^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/pymlst/wg/core.py", line 63, in __init__
engine = utils.get_updated_engine(file, 'wg')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/pymlst/common/utils.py", line 143, in get_updated_engine
check_type(conn, module)
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/pymlst/common/utils.py", line 113, in check_type
set_type(conn, mlst_type)
File "/home/fmj/miniconda3/envs/pymlst/lib/python3.11/site-packages/pymlst/common/utils.py", line 126, in set_type
conn.execute(
TypeError: Connection.execute() got an unexpected keyword argument 'name'
ok, same problem.
You seems to use 3.11 python version. I think this is the same problem as indicate in 12
Can you try to install pymlst on python 3.9 environnement?
Can you try to install pymlst on python 3.9 environnement?
conda create -n "pymlst3.9" python=3.9 pymlst
I followed your advice, but failed again
you need to activate the good environnement
conda activate pymlst3.9
Can you report the result of this command:
which wgMLST
/home/fmj/miniconda3/envs/pymlst3.9/bin/wgMLST
Normally, it must work with this version of python.
Can you report this command result:
python --version
Python 3.9.16
And you have the same error with the import command?
Yes,it still error. Although we haven't found the problem yet, thank you for your patience and timely response.
a new error appear
[INFO: 2023-03-02 16:55:24,099] One element found : Clostridium perfringens
[INFO: 2023-03-02 16:55:24,099] Downloading the core genome...
Traceback (most recent call last):
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/common/web.py", line 231, in get_cgmlst_file
iterator = next(SeqIO.parse(os.path.join(fas_tmp, fasta), 'fasta'))
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/Bio/SeqIO/__init__.py", line 605, in parse
return iterator_generator(handle)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/Bio/SeqIO/FastaIO.py", line 223, in __init__
super().__init__(source, mode="t", fmt="Fasta")
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/Bio/SeqIO/Interfaces.py", line 45, in __init__
self.stream = open(source, "r" + mode)
OSError: [Errno 24] Too many open files: '/tmp/tmp81po3jrs/fas/CPF_RS01715.fasta'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/fmj/miniconda3/envs/pymlst3.92/bin/wgmlst", line 10, in <module>
sys.exit(wg())
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/wg/commands/import.py", line 52, in cli
skipped = web.get_cgmlst_file(url, tmp)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/common/web.py", line 237, in get_cgmlst_file
return skipped
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/tempfile.py", line 830, in __exit__
self.cleanup()
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/tempfile.py", line 834, in cleanup
self._rmtree(self.name)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/tempfile.py", line 816, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/shutil.py", line 730, in rmtree
onerror(os.open, path, sys.exc_info())
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/shutil.py", line 727, in rmtree
fd = os.open(path, os.O_RDONLY)
OSError: [Errno 24] Too many open files: '/tmp/tmp81po3jrs'
The first problem is link to the change of SQLachemy library from 1.4 to 2.0 version.
Try the command :
wgMLST create E.coli.db cgmlst_schema.fasta
(pymlst3.92) fmj@qood:~$ wgMLST create E.coli.db cgmlst_schema.fasta
Usage: wgMLST create [OPTIONS] DATABASE COREGENE
Try 'wgMLST create -h' for help.
Error: Database alreadly exists, use --force to override it
(pymlst3.92) fmj@qood:~$ wgMLST create E.coli.db cgmlst_schema.fasta --force
Traceback (most recent call last):
File "/home/fmj/miniconda3/envs/pymlst3.92/bin/wgMLST", line 10, in <module>
sys.exit(wg())
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/wg/commands/create.py", line 33, in cli
with pymlst.open_wg(os.path.abspath(database)) as mlst:
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/wg/core.py", line 40, in open_wg
mlst = WholeGenomeMLST(file, ref)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/wg/core.py", line 442, in __init__
self.__database = DatabaseWG(file, ref)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/wg/core.py", line 63, in __init__
engine = utils.get_updated_engine(file, 'wg')
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/common/utils.py", line 143, in get_updated_engine
check_type(conn, module)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/common/utils.py", line 113, in check_type
set_type(conn, mlst_type)
File "/home/fmj/miniconda3/envs/pymlst3.92/lib/pypy3.9/site-packages/pymlst/common/utils.py", line 126, in set_type
conn.execute(
TypeError: execute() got an unexpected keyword argument 'name'
OK, this is the same problem of SQLAlchemy library. You can try to downgrade SQLAlchemy version to 1.4 series
I'm looking for modify pyMLST for compatibility to SQLAlchemy >2.0
I executed the following command and the result was successful
mamba install SQLAlchemy=1.4
ulimit -n 2048
wgmlst import Cp.db "Clostridium perfringens" --force
Thank you again for your patience, timely and professional reply
good.
I can check to update to SQLAlchemy 2.0 with no pression.
Hi I install this too by mamba when i try to import database, i receive a sql error
Could you help me finding the mistake i do?
cheers.