Closed DrBinZhu closed 11 months ago
@DrBinZhu Please use rgi auto_load
as it will configure the latest databases needed.
@DrBinZhu Dang it! I'm getting this error too. I will investigate. Cheers.
@DrBinZhu Ok, so far, it looks like it MacOS a related issue. I used Monterey and got the same error, but using Ubuntu 18.04.6 LTS I have no issues.
@DrBinZhu I was able to load databases using new version see #235. Re-open the issue if you still have the same problem. Cheers.
Thanks, @raphenya. Bin
On Mon, Sep 25, 2023 at 1:02 PM amos @.***> wrote:
@DrBinZhu https://github.com/DrBinZhu I was able to load databases using new version see #235 https://github.com/arpcard/rgi/issues/235. Re-open the issue if you still have the same problem. Cheers.
— Reply to this email directly, view it on GitHub https://github.com/arpcard/rgi/issues/230#issuecomment-1734141947, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM7O2IQ2VO3OBZ2I5RF75VDX4G2L3ANCNFSM6AAAAAAYUBSYQM . You are receiving this because you were mentioned.Message ID: @.***>
Hello, I reopen this issue as I am encountering the same problem as described above by @DrBinZhu, with the same Error message
` This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.`
My machine is a Mac, my processor is osx-arm 64, I also had issues installing Conda environments which I solved by using the following example:
conda create -n rgi_env conda activate rgi_env conda config --env --set subdir osx-64
In this way I was able to install rig which seems to be fine and I also used the autoload function you suggested in #235
The databases seem loaded correctly and rgi database --all -v
outputs the list of databases as expected.
However when I try and run the code I get the error mentioned above. Using the --debug mode the job produces temporary files up to the blast annotation step (arg_contigs.fasta.temp.contig.fsa.blastRes.xml) and seems to freeze and have issues with the multiprocessing/threading.
I am not sure how to solve this, therefore I look forward receiving your suggestions! thanks
the packages in my Conda env
archspec 0.2.2 pyhd8ed1ab_0 conda-forge
arrow-cpp 10.0.1 h57ddcff_4_cpu conda-forge
aws-c-auth 0.6.21 h268417e_3 conda-forge
aws-c-cal 0.5.20 h2f96456_3 conda-forge
aws-c-common 0.8.5 hb7f2c08_0 conda-forge
aws-c-compression 0.2.16 hfb1f584_0 conda-forge
aws-c-event-stream 0.2.17 h38cbd52_1 conda-forge
aws-c-http 0.7.0 h4f27208_0 conda-forge
aws-c-io 0.13.12 hb3bb6ef_1 conda-forge
aws-c-mqtt 0.7.13 h32e11d9_12 conda-forge
aws-c-s3 0.2.1 h9eec77e_4 conda-forge
aws-c-sdkutils 0.1.7 hfb1f584_0 conda-forge
aws-checksums 0.1.14 hfb1f584_0 conda-forge
aws-crt-cpp 0.18.16 ha874d0c_8 conda-forge
aws-sdk-cpp 1.9.379 hdb1f066_7 conda-forge
bamtools 2.5.1 h6151dfb_10 bioconda
beautifulsoup4 4.9.3 pyhb0f4dca_0 conda-forge
bedtools 2.31.1 h6372da2_0 bioconda
biopython 1.78 py38h96a0964_2 conda-forge
blast 2.14.1 pl5321h91c44f7_0 bioconda
bokeh 3.1.1 pyhd8ed1ab_0 conda-forge
bowtie2 2.5.3 py38hbee1746_0 bioconda
brotli 1.0.9 hb7f2c08_9 conda-forge
brotli-bin 1.0.9 hb7f2c08_9 conda-forge
brotlipy 0.7.0 py38hcafd530_1006 conda-forge
bwa 0.7.17 h45fc8d7_11 bioconda
bzip2 1.0.8 h10d778d_5 conda-forge
c-ares 1.26.0 h10d778d_0 conda-forge
ca-certificates 2024.2.2 h8857fd0_0 conda-forge
certifi 2024.2.2 pyhd8ed1ab_0 conda-forge
cffi 1.14.6 py38h9688ba1_0 conda-forge
chardet 3.0.4 py38h5347e94_1008 conda-forge
charset-normalizer 3.3.2 pypi_0 pypi
click 8.1.7 unix_pyh707e725_0 conda-forge
cloudpickle 3.0.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
contourpy 1.1.1 py38h15a1a5b_1 conda-forge
cryptography 39.0.0 py38ha6c3189_0 conda-forge
curl 7.87.0 haf73cf8_0 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
cytoolz 0.12.3 py38hae2e43d_0 conda-forge
dask 2023.5.0 pyhd8ed1ab_0 conda-forge
dask-core 2023.5.0 pyhd8ed1ab_0 conda-forge
diamond 0.8.36 h94ec1f5_7 bioconda
distributed 2023.5.0 pyhd8ed1ab_0 conda-forge
entrez-direct 16.2 h193322a_1 bioconda
exceptiongroup 1.2.0 pyhd8ed1ab_2 conda-forge
filetype 1.2.0 pyhd8ed1ab_0 conda-forge
fonttools 4.48.1 py38hae2e43d_0 conda-forge
freetype 2.12.1 h60636b9_2 conda-forge
fsspec 2024.2.0 pyhca7485f_0 conda-forge
gettext 0.21.1 h8a4c099_0 conda-forge
gflags 2.2.2 hb1e8313_1004 conda-forge
giflib 5.2.1 hb7f2c08_3 conda-forge
glog 0.6.0 h8ac2a54_0 conda-forge
htslib 1.9 h356306b_9 bioconda
icu 73.2 hf5e326d_0 conda-forge
idna 2.10 pyh9f0ad1d_0 conda-forge
importlib-metadata 7.0.1 pyha770c72_0 conda-forge
importlib-resources 6.1.1 pyhd8ed1ab_0 conda-forge
importlib_metadata 7.0.1 hd8ed1ab_0 conda-forge
importlib_resources 6.1.1 pyhd8ed1ab_0 conda-forge
iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge
jbig 2.1 h0d85af4_2003 conda-forge
jellyfish 2.2.10 h04f5b5a_1 bioconda
jemalloc 5.3.0 hf0c8a7f_0 conda-forge
jinja2 3.1.3 pyhd8ed1ab_0 conda-forge
jpeg 9e hb7f2c08_3 conda-forge
kiwisolver 1.4.5 py38h15a1a5b_1 conda-forge
kma 1.4.14 h45fc8d7_0 bioconda
krb5 1.20.1 h0165f36_0 conda-forge
lcms2 2.12 h577c468_0 conda-forge
lerc 2.2.1 h046ec9c_0 conda-forge
libabseil 20220623.0 cxx17_h844d122_6 conda-forge
libarrow 10.0.1 hc216794_4_cpu conda-forge
libblas 3.9.0 20_osx64_openblas conda-forge
libbrotlicommon 1.0.9 hb7f2c08_9 conda-forge
libbrotlidec 1.0.9 hb7f2c08_9 conda-forge
libbrotlienc 1.0.9 hb7f2c08_9 conda-forge
libcblas 3.9.0 20_osx64_openblas conda-forge
libcrc32c 1.1.2 he49afe7_0 conda-forge
libcurl 7.87.0 haf73cf8_0 conda-forge
libcxx 16.0.6 hd57cbcb_0 conda-forge
libdeflate 1.7 h35c211d_5 conda-forge
libedit 3.1.20191231 hed1e85f_2 conda-forge
libev 4.33 h10d778d_2 conda-forge
libevent 2.1.10 h815e4d9_4 conda-forge
libffi 3.3 h046ec9c_2 conda-forge
libgfortran 5.0.0 13_2_0_h97931a8_2 conda-forge
libgfortran5 13.2.0 h2873a65_2 conda-forge
libgoogle-cloud 2.5.0 h32e9745_1 conda-forge
libgrpc 1.51.1 h834a566_0 conda-forge
libhwloc 2.9.3 default_h24e0189_1009 conda-forge
libiconv 1.17 hd75f5a5_2 conda-forge
libidn2 2.3.7 h10d778d_0 conda-forge
libjemalloc 5.3.0 hf0c8a7f_0 conda-forge
liblapack 3.9.0 20_osx64_openblas conda-forge
libnghttp2 1.51.0 h0dd9d14_0 conda-forge
libopenblas 0.3.25 openmp_hfef2a42_0 conda-forge
libpng 1.6.42 h92b6c6a_0 conda-forge
libprotobuf 3.21.12 h7d26f99_2 conda-forge
libssh2 1.10.0 h7535e13_3 conda-forge
libthrift 0.16.0 h08c06f4_2 conda-forge
libtiff 4.3.0 h1167814_1 conda-forge
libunistring 0.9.10 h0d85af4_0 conda-forge
libutf8proc 2.8.0 hb7f2c08_0 conda-forge
libwebp 1.2.2 h28dabe5_0 conda-forge
libwebp-base 1.2.2 h0d85af4_1 conda-forge
libxcb 1.13 h0d85af4_1004 conda-forge
libxml2 2.12.5 hc0ae0f7_0 conda-forge
libxslt 1.1.39 h03b04e6_0 conda-forge
libzlib 1.2.13 h8a1eda9_5 conda-forge
llvm-openmp 17.0.6 hb6ac08f_0 conda-forge
locket 1.0.0 pyhd8ed1ab_0 conda-forge
lxml 5.1.0 pypi_0 pypi
lz4 4.3.3 py38h5f57667_0 conda-forge
lz4-c 1.9.4 hf0c8a7f_0 conda-forge
markupsafe 2.1.5 py38hae2e43d_0 conda-forge
matplotlib 3.7.3 py38h50d1736_0 conda-forge
matplotlib-base 3.7.3 py38hcd1b199_0 conda-forge
msgpack-python 1.0.7 py38ha94643f_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
ncbi-vdb 3.0.0 pl5321h9722bc1_0 bioconda
ncurses 6.1 h0a44026_1002 conda-forge
numpy 1.24.4 py38h9a4a08f_0 conda-forge
oligoarrayaux 3.8 h770b8ee_0 bioconda
openjpeg 2.5.0 h69f46e4_0 conda-forge
openssl 1.1.1w h8a1eda9_0 conda-forge
orc 1.8.2 ha9d861c_2 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 2.0.3 py38h78e6021_1 conda-forge
parquet-cpp 1.5.1 2 conda-forge
partd 1.4.1 pyhd8ed1ab_0 conda-forge
patsy 0.5.6 pyhd8ed1ab_0 conda-forge
pcre 8.45 he49afe7_0 conda-forge
perl 5.32.1 7_h10d778d_perl5 conda-forge
perl-archive-tar 2.40 pl5321hdfd78af_0 bioconda
perl-carp 1.50 pl5321hd8ed1ab_0 conda-forge
perl-common-sense 3.75 pl5321hd8ed1ab_0 conda-forge
perl-compress-raw-bzip2 2.201 pl5321h775f41a_0 conda-forge
perl-compress-raw-zlib 2.202 pl5321h775f41a_0 conda-forge
perl-encode 3.19 pl5321hb7f2c08_0 conda-forge
perl-exporter 5.74 pl5321hd8ed1ab_0 conda-forge
perl-exporter-tiny 1.002002 pl5321hd8ed1ab_0 conda-forge
perl-extutils-makemaker 7.70 pl5321hd8ed1ab_0 conda-forge
perl-io-compress 2.201 pl5321h7133b54_2 bioconda
perl-io-zlib 1.14 pl5321hdfd78af_0 bioconda
perl-json 4.10 pl5321hdfd78af_0 bioconda
perl-json-xs 2.34 pl5321hcd10b59_5 bioconda
perl-list-moreutils 0.430 pl5321hdfd78af_0 bioconda
perl-list-moreutils-xs 0.430 pl5321ha5712d3_1 bioconda
perl-parent 0.241 pl5321hd8ed1ab_0 conda-forge
perl-pathtools 3.75 pl5321h775f41a_0 conda-forge
perl-scalar-list-utils 1.63 pl5321h775f41a_0 conda-forge
perl-storable 3.15 pl5321h775f41a_0 conda-forge
perl-types-serialiser 1.01 pl5321hdfd78af_0 bioconda
pillow 9.1.1 py38h21af888_1 conda-forge
pip 24.0 pyhd8ed1ab_0 conda-forge
platformdirs 4.2.0 pyhd8ed1ab_0 conda-forge
pluggy 1.4.0 pyhd8ed1ab_0 conda-forge
pooch 1.8.0 pyhd8ed1ab_0 conda-forge
prodigal 2.6.3 h2413b67_7 bioconda
psutil 5.9.8 py38hae2e43d_0 conda-forge
pthread-stubs 0.4 hc929b4f_1001 conda-forge
pyahocorasick 1.4.0 py38h4d0b108_2 conda-forge
pyarrow 10.0.1 py38h450cdb6_4_cpu conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pyfaidx 0.8.1.1 pyhdfd78af_0 bioconda
pyopenssl 23.2.0 pyhd8ed1ab_1 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pyrodigal 3.3.0 py38h604ca31_0 bioconda
pysam 0.16.0.1 py38h4ebacbe_3 bioconda
pysocks 1.7.1 pyha2e5f31_6 conda-forge
pytest 8.0.0 pyhd8ed1ab_0 conda-forge
python 3.8.10 h0e5c897_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-tzdata 2023.4 pyhd8ed1ab_0 conda-forge
python_abi 3.8 4_cp38 conda-forge
pytz 2024.1 pyhd8ed1ab_0 conda-forge
pyvcf3 1.0.3 pyhdfd78af_0 bioconda
pyyaml 6.0.1 py38hcafd530_1 conda-forge
re2 2022.06.01 hb486fe8_1 conda-forge
readline 8.0 hed1e85f_1 conda-forge
requests 2.31.0 pypi_0 pypi
rgi 6.0.3 pypi_0 pypi
samtools 1.9 h8aa4d43_12 bioconda
scipy 1.10.1 py38h9cf86d3_3 conda-forge
seaborn 0.13.2 hd8ed1ab_0 conda-forge
seaborn-base 0.13.2 pyhd8ed1ab_0 conda-forge
setuptools 69.0.3 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.10 h225ccf5_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.5 pyhd8ed1ab_1 conda-forge
sqlite 3.40.1 h880c91c_0
statsmodels 0.14.1 py38h7092abf_0 conda-forge
tbb 2021.11.0 h7728843_1 conda-forge
tblib 3.0.0 pyhd8ed1ab_0 conda-forge
tk 8.6.13 h1abcd95_1 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
toolz 0.12.1 pyhd8ed1ab_0 conda-forge
tornado 6.3.3 py38hcafd530_1 conda-forge
unicodedata2 15.1.0 py38hcafd530_0 conda-forge
urllib3 1.25.11 py_0 conda-forge
wget 1.20.3 h52ee1ee_1 conda-forge
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.11 h0dc2134_0 conda-forge
xorg-libxdmcp 1.1.3 h35c211d_0 conda-forge
xyzservices 2023.10.1 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h775f41a_0 conda-forge
yaml 0.2.5 h0d85af4_2 conda-forge
zict 3.0.0 pyhd8ed1ab_0 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 h8a1eda9_5 conda-forge
zstd 1.5.5 h829000d_0 conda-forge
After installing the RGI using 'mamba create -c iuc -c conda-forge -c bioconda -c defaults --no-channel-priority --name RGI rgi', I ran the following commands:
rgi card_annotation -i path_to/card.json > card_annotation.log 2>&1 rgi wildcard_annotation -i wildcard --card_json path_to/card.json -v 4.0.0 > wildcard_annotation.log 2>&1
rgi load \ --card_json path_to/card.json \ --debug --local \ --card_annotation card_database_v3.2.7.fasta \ --card_annotation_all_models card_database_v3.2.7_all.fasta \ --wildcard_annotation wildcard_database_v4.0.0.fasta \ --wildcard_annotation_all_models wildcard_database_v4.0.0_all.fasta \ --wildcard_index path_to/wildcard/index-for-model-sequences.txt \ --wildcard_version 4.0.0 \ --amr_kmers path_to/wildcard/all_amr_61mers.txt \ --kmer_database path_to/wildcard/61_kmer_db.json \ --kmer_size 61
Then the information shows: ... filename: card.json, validate_json: False ... filename: card_reference.fasta, validate_json: False ... every validate_json checks are False.
rgi database --version --local 3.2.7
When running a test using this command: 'rgi main --input_sequence path_to/GCF_002861965.1_ASM286196v1_genomic.fna --output_file a --local --clean', there are bugs showing below:
Traceback (most recent call last): File "", line 1, in
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/spawn.py", line 125, in _main
prepare(preparation_data)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/runpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/binzhu/miniconda3/envs/RGI/bin/rgi", line 4, in
MainBase()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/site-packages/app/MainBase.py", line 83, in init
getattr(self, args.command)()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/site-packages/app/MainBase.py", line 88, in main
self.main_run(args)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/site-packages/app/MainBase.py", line 128, in main_run
rgi_obj.run()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/site-packages/app/RGI.py", line 255, in run
self.filter_process()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/site-packages/app/RGI.py", line 387, in filter_process
filter_obj.run()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/site-packages/app/Filter.py", line 174, in run
self.process_xml_file()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/site-packages/app/Filter.py", line 102, in process_xml_file
model_thread.start()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 42, in _launch
prep_data = spawn.get_preparation_data(process_obj._name)
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/spawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "/Users/binzhu/miniconda3/envs/RGI/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
I'm using macOS Big Sur, version 11.1
Do you have any idea to deal with the problem? Many thanks, Bin