urllib.error.URLError: Downloading SwissProt Fasta files

[narib]

Hi all,

I use mamba with aspera, it was able to download up to a point here .in step 3...for swissprot files, can someone give me any idea?

(microbeannotator) rsaa% microbeannotator_db_builder -d ~/db/MicrobeAnnotator_DB -m blast -t 24 2023-04-03 10:39:40,073 [INFO]: This is MicrobeAnnotator v2.0.5 2023-04-03 10:39:40,073 [INFO]: I will download and format the databases I use. 2023-04-03 10:39:40,073 [INFO]: Creating database folders 2023-04-03 10:39:40,073 [INFO]: Step 1 2023-04-03 10:39:40,073 [INFO]: Downloading KOfam data 100% [...................................................................] 827446 / 827446gzip: /home/brianho/db/MicrobeAnnotator_DB/kofam_data/ko_list already exists; do you wish to overwrite (y or n)? n not overwritten 100% [...........................................................] 1414909362 / 14149093622023-04-03 10:43:37,137 [INFO]: Extracting profiles 2023-04-03 10:44:14,390 [INFO]: Finished 2023-04-03 10:44:14,390 [INFO]: Step 2 2023-04-03 10:44:14,390 [INFO]: Formatting KOfam profiles 2023-04-03 10:44:29,608 [INFO]: Finished 2023-04-03 10:44:29,611 [INFO]: Step 3 2023-04-03 10:44:31,924 [INFO]: Downloading SwissProt Fasta files Traceback (most recent call last): File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 2437, in retrfile conn, retrlen = self.ftp.ntransfercmd(cmd) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 366, in ntransfercmd host, port = self.makepasv() File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 340, in makepasv untrusted_host, port = parse227(self.sendcmd('PASV')) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 275, in sendcmd return self.getresp() File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 248, in getresp raise error_perm(resp) ftplib.error_perm: 500 OOPS: vsf_sysutil_bind

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 1573, in ftp_open fp, retrlen = fw.retrfile(file, type) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 2441, in retrfile sys.exc_info()[2]) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 2437, in retrfile conn, retrlen = self.ftp.ntransfercmd(cmd) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 366, in ntransfercmd host, port = self.makepasv() File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 340, in makepasv untrusted_host, port = parse227(self.sendcmd('PASV')) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 275, in sendcmd return self.getresp() File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 248, in getresp raise error_perm(resp) urllib.error.URLError: <urlopen error ftp error: error_perm('500 OOPS: vsf_sysutil_bind')>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/bin/microbeannotator_db_builder", line 445, in main() File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/bin/microbeannotator_db_builder", line 437, in main single_step, aspera, keep_temp, bin_path) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/bin/microbeannotator_db_builder", line 105, in database_duilder swissprot_fasta = uniprot.swissprot_fasta_downloader(database_directory) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/site-packages/microbeannotator/database/uniprot_data_downloader.py", line 76, in swissprot_fasta_downloader wget.download(fasta_url, out=str(output_fasta)) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/site-packages/wget.py", line 526, in download (tmpfile, headers) = ulib.urlretrieve(binurl, tmpfile, callback) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 247, in urlretrieve with contextlib.closing(urlopen(url, data)) as fp: File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 525, in open response = self._open(req, data) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 543, in _open '_open', req) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 503, in _call_chain result = func(*args) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 1584, in ftp_open raise exc.with_traceback(sys.exc_info()[2]) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 1573, in ftp_open fp, retrlen = fw.retrfile(file, type) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 2441, in retrfile sys.exc_info()[2]) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/urllib/request.py", line 2437, in retrfile conn, retrlen = self.ftp.ntransfercmd(cmd) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 366, in ntransfercmd host, port = self.makepasv() File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 340, in makepasv untrusted_host, port = parse227(self.sendcmd('PASV')) File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 275, in sendcmd return self.getresp() File "/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/lib/python3.7/ftplib.py", line 248, in getresp raise error_perm(resp) urllib.error.URLError: <urlopen error ftp error: URLError("ftp error: error_perm('500 OOPS: vsf_sysutil_bind')")>

My $PATH env /home/brianho/.aspera/connect/bin/:/home/brianho/.conda/envs/mambaforge/envs/microbeannotator/bin:/home/brianho/.conda/envs/mambaforge/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

Here is my mamba env (microbeannotator) rsaa% mamba list

packages in environment at /home/brianho/.conda/envs/mambaforge/envs/microbeannotator:

#

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge alsa-lib 1.2.8 h166bdaf_0 conda-forge attr 2.5.1 h166bdaf_1 conda-forge attrs 20.3.0 pyhd3deb0d_0 conda-forge biopython 1.78 py37h5e8e339_2 conda-forge blast 2.11.0 pl5262h3289130_1 bioconda boost-cpp 1.74.0 h75c5d50_8 conda-forge brotlipy 0.7.0 py37h540881e_1004 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.18.1 h7f98852_0 conda-forge ca-certificates 2022.12.7 ha878542_0 conda-forge cairo 1.16.0 ha61ee94_1014 conda-forge certifi 2022.12.7 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py37h43b0acd_1 conda-forge chardet 4.0.0 py37h89c1867_3 conda-forge click 8.1.3 pypi_0 pypi cryptography 38.0.2 py37h5994e8b_1 conda-forge curl 7.88.1 hdc1c0ab_1 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge dbus 1.13.6 h5008d03_3 conda-forge diamond 2.0.9 hdcc8f71_0 bioconda entrez-direct 16.2 he881be0_1 bioconda exceptiongroup 1.1.1 pypi_0 pypi expat 2.5.0 h27087fc_0 conda-forge fasta-reader 1.0.0 pypi_0 pypi fftw 3.3.10 nompi_hf0379b8_106 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.2 h14ed4e7_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge freetype 2.12.1 hca18f0e_1 conda-forge gettext 0.21.1 h27087fc_0 conda-forge gff-io 0.0.2 pypi_0 pypi glib 2.74.1 h6239696_1 conda-forge glib-tools 2.74.1 h6239696_1 conda-forge graphite2 1.3.13 h58526e2_1001 conda-forge gst-plugins-base 1.22.0 h4243ec0_2 conda-forge gstreamer 1.22.0 h25f0c4b_2 conda-forge gstreamer-orc 0.4.33 h166bdaf_0 conda-forge harfbuzz 6.0.0 h8e241bc_0 conda-forge hmmer 0.1.0 pypi_0 pypi icu 70.1 h27087fc_0 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge importlib-metadata 6.1.0 pypi_0 pypi importlib-resources 5.12.0 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi isal 1.1.0 pypi_0 pypi jack 1.9.22 h11f4161_0 conda-forge jpeg 9e h0b41bf4_3 conda-forge keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.4 py37h7cecad7_0 conda-forge krb5 1.20.1 h81ceb04_0 conda-forge lame 3.100 h166bdaf_1003 conda-forge lcms2 2.14 h6ed2654_0 conda-forge ld_impl_linux-64 2.40 h41732ed_0 conda-forge lerc 4.0.0 h27087fc_0 conda-forge libblas 3.9.0 16_linux64_openblas conda-forge libcap 2.67 he9d0100_0 conda-forge libcblas 3.9.0 16_linux64_openblas conda-forge libclang 15.0.7 default_had23c3d_1 conda-forge libclang13 15.0.7 default_h3e3d535_1 conda-forge libcups 2.3.3 h36d4200_3 conda-forge libcurl 7.88.1 hdc1c0ab_1 conda-forge libdb 6.2.32 h9c3ff4c_0 conda-forge libdeflate 1.14 h166bdaf_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 h28343ad_4 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libflac 1.4.2 h27087fc_0 conda-forge libgcc-ng 12.2.0 h65d4601_19 conda-forge libgcrypt 1.10.1 h166bdaf_0 conda-forge libgfortran-ng 12.2.0 h69a702a_19 conda-forge libgfortran5 12.2.0 h337968e_19 conda-forge libglib 2.74.1 h606061b_1 conda-forge libgomp 12.2.0 h65d4601_19 conda-forge libgpg-error 1.46 h620e276_0 conda-forge libiconv 1.17 h166bdaf_0 conda-forge libidn2 2.3.4 h166bdaf_0 conda-forge liblapack 3.9.0 16_linux64_openblas conda-forge libllvm15 15.0.7 hadd5161_1 conda-forge libnghttp2 1.52.0 h61bc06f_0 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libogg 1.3.4 h7f98852_1 conda-forge libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge libopus 1.3.1 h7f98852_1 conda-forge libpng 1.6.39 h753d276_0 conda-forge libpq 15.2 hb675445_0 conda-forge libsndfile 1.2.0 hb75c966_0 conda-forge libsqlite 3.40.0 h753d276_0 conda-forge libssh2 1.10.0 hf14f497_3 conda-forge libstdcxx-ng 12.2.0 h46fd767_19 conda-forge libsystemd0 253 h8c4010b_1 conda-forge libtiff 4.4.0 h82bc61c_5 conda-forge libtool 2.4.7 h27087fc_0 conda-forge libudev1 253 h0b41bf4_1 conda-forge libunistring 0.9.10 h7f98852_0 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libvorbis 1.3.7 h9c3ff4c_0 conda-forge libwebp-base 1.3.0 h0b41bf4_0 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxkbcommon 1.5.0 h79f4944_1 conda-forge libxml2 2.10.3 hca2bb57_4 conda-forge libzlib 1.2.13 h166bdaf_4 conda-forge lz4-c 1.9.4 hcb278e6_0 conda-forge matplotlib 3.4.1 py37h89c1867_0 conda-forge matplotlib-base 3.4.1 py37hdd32ed1_0 conda-forge microbeannotator 2.0.5 py37_0 cruizperez more-itertools 9.1.0 pypi_0 pypi mpg123 1.31.3 hcb278e6_0 conda-forge mysql-common 8.0.32 ha901b37_1 conda-forge mysql-libs 8.0.32 hd7da12d_1 conda-forge ncurses 6.3 h27087fc_1 conda-forge nspr 4.35 h27087fc_0 conda-forge nss 3.89 he45b914_0 conda-forge numpy 1.21.6 py37h976b520_0 conda-forge openjpeg 2.5.0 h7d73246_1 conda-forge openssl 3.1.0 h0b41bf4_0 conda-forge packaging 23.0 pyhd8ed1ab_0 conda-forge pandas 1.2.4 py37h2531618_0 patsy 0.5.3 pyhd8ed1ab_0 conda-forge pcre 8.45 h9c3ff4c_0 conda-forge pcre2 10.40 hc3806b6_0 conda-forge perl 5.26.2 h36c2ea0_1008 conda-forge perl-archive-tar 2.32 pl526_0 bioconda perl-carp 1.38 pl526_3 bioconda perl-common-sense 3.74 pl526_2 bioconda perl-compress-raw-bzip2 2.087 pl526he1b5a44_0 bioconda perl-compress-raw-zlib 2.087 pl526hc9558a2_0 bioconda perl-exporter 5.72 pl526_1 bioconda perl-exporter-tiny 1.002001 pl526_0 bioconda perl-extutils-makemaker 7.36 pl526_1 bioconda perl-io-compress 2.087 pl526he1b5a44_0 bioconda perl-io-zlib 1.10 pl526_2 bioconda perl-json 4.02 pl526_0 bioconda perl-json-xs 2.34 pl526h6bb024c_3 bioconda perl-list-moreutils 0.428 pl526_1 bioconda perl-list-moreutils-xs 0.428 pl526_0 bioconda perl-pathtools 3.75 pl526h14c3975_1 bioconda perl-scalar-list-utils 1.52 pl526h516909a_0 bioconda perl-types-serialiser 1.0 pl526_2 bioconda perl-xsloader 0.24 pl526_0 bioconda pillow 9.2.0 py37h850a105_2 conda-forge pip 23.0.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge platformdirs 3.2.0 pypi_0 pypi pluggy 1.0.0 pypi_0 pypi ply 3.11 py_1 conda-forge pooch 1.7.0 pypi_0 pypi psutil 5.8.0 py37h5e8e339_2 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge pulseaudio 16.1 hcb278e6_3 conda-forge pulseaudio-client 16.1 h5195f5e_3 conda-forge pulseaudio-daemon 16.1 ha8d29e2_3 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pyopenssl 23.1.1 pyhd8ed1ab_0 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pyqt 5.15.7 py37hf30b843_1 conda-forge pyqt5-sip 12.11.0 py37hd23a5d3_1 conda-forge pysocks 1.7.1 py37h89c1867_5 conda-forge pytest 7.2.2 pypi_0 pypi python 3.7.12 hf930737_100_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python_abi 3.7 3_cp37m conda-forge pytz 2023.2 pyhd8ed1ab_0 conda-forge pywget 3.2 py37h06a4308_0 qt-main 5.15.8 h5d23da1_6 conda-forge readline 8.2 h8228510_1 conda-forge requests 2.25.1 pyhd3deb0d_0 conda-forge scipy 1.7.3 py37hf2a6cf1_0 conda-forge seaborn 0.11.1 hd8ed1ab_1 conda-forge seaborn-base 0.11.1 pyhd8ed1ab_1 conda-forge setuptools 67.6.1 pyhd8ed1ab_0 conda-forge sip 6.7.2 py37hd23a5d3_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge sqlite 3.40.0 h4ff8645_0 conda-forge statsmodels 0.13.2 py37hda87dfa_0 conda-forge sword 1.0.4 h9f5acd7_2 bioconda tk 8.6.12 h27826a3_0 conda-forge toml 0.10.2 pyhd8ed1ab_0 conda-forge tomli 2.0.1 pypi_0 pypi tornado 6.2 py37h540881e_0 conda-forge typing-extensions 4.5.0 hd8ed1ab_0 conda-forge typing_extensions 4.5.0 pyha770c72_0 conda-forge urllib3 1.26.15 pyhd8ed1ab_0 conda-forge wget 1.20.3 ha35d2d1_1 conda-forge wheel 0.40.0 pyhd8ed1ab_0 conda-forge xcb-util 0.4.0 h516909a_0 conda-forge xcb-util-image 0.4.0 h166bdaf_0 conda-forge xcb-util-keysyms 0.4.0 h516909a_0 conda-forge xcb-util-renderutil 0.3.9 h166bdaf_0 conda-forge xcb-util-wm 0.4.1 h516909a_0 conda-forge xkeyboard-config 2.38 h0b41bf4_0 conda-forge xopen 1.7.0 pypi_0 pypi xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.0.10 h7f98852_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.8.4 h0b41bf4_0 conda-forge xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h0b41bf4_2 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xz 5.2.6 h166bdaf_0 conda-forge zipp 3.15.0 pypi_0 pypi zlib 1.2.13 h166bdaf_4 conda-forge zstd 1.5.2 h3eb15da_6 conda-forge

[mscarbor]

Hi @narib -- Did you have any luck solving this error? I am getting the same. Thanks in advance!

[Krasnopeev]

Hi all, I am getting the same error. Here is my solution: in file ~/mambaforge/envs/microbeannotator/lib/python3.7/site-packages/microbeannotator/database/uniprot_data_downloader.py find urls like: ftp://ftp.uniprot.org/pub/databases/uniprot/ and change them to https like https://ftp.uniprot.org/pub/databases/uniprot/ make for all occurances in file and save it. Then restart microbeannotator_db_builder with --step 3 for continue run.