search

ncbi / amr

AMRFinderPlus - Identify AMR genes and point mutations, and virulence and stress resistance genes in assembled bacterial nucleotide and protein sequence.
https://www.ncbi.nlm.nih.gov/pathogens/antimicrobial-resistance/AMRFinder/
Other
254 stars 34 forks source link

amrfinder_update cannot connect to FTP and thus download #bug #help #120

Open geboro opened 1 year ago

geboro commented 1 year ago

Hello,

I'm struggling installing AMRFinder in my institution. It keeps failing because it cannot find the ftp site: 

amrfinder_update  --database /home/bioinf/bioinf_archive/98_mökiDatabases/bakta/db/amrfinderplus-db
Running: amrfinder_update --database /home/bioinf/bioinf_archive/98_mökiDatabases/bakta/db/amrfinderplus-db
Looking up the published databases at https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

*** ERROR ***
CURL: Cannot read from https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

HOSTNAME: ?
SHELL: /bin/bash
PWD: /home/bioinf
PATH: /home/bioinf/mambaforge/envs/bakta/bin:/home/bioinf/mambaforge/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
Progam name:  amrfinder_update
Command line: amrfinder_update --database /home/bioinf/bioinf_archive/98_mökiDatabases/bakta/db/amrfinderplus-db

but curl is working fine:

curl https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
 <head>
  <title>Index of /pathogen/Antimicrobial_resistance/AMRFinderPlus/database</title>
 </head>
 <body>
<h1>Index of /pathogen/Antimicrobial_resistance/AMRFinderPlus/database</h1>
<pre>Name                                              Last modified      Size  <hr><a href="/pathogen/Antimicrobial_resistance/AMRFinderPlus/">Parent Directory</a>                                                       -   
<a href="3.10/">3.10/</a>                                             2021-12-22 13:07    -   
<a href="3.2/">3.2/</a>                                              2019-10-31 11:24    -   
<a href="3.6/">3.6/</a>                                              2020-03-25 08:36    -   
<a href="3.8/">3.8/</a>                                              2020-10-02 15:43    -   
<a href="3.9/">3.9/</a>                                              2020-12-21 10:09    -   
<a href="3.10/">3.10/</a>                                              2022-10-12 10:00    -
<a href="3.11/">3.11/</a>                                              2022-10-12 10:00    -
<a href="latest/">latest/</a>                                             2021-12-22 13:19    -   
<hr></pre>

So two questions:

  1. Is there any way to separate database installation from download?

  2. I downloaded and indexed the databases manually, but when I run 'amrfinder' it cannot find the path of the databases by default. Since I am working with a wrapper that I cannot edit, and the binaries in AMRFinder are uneditable, is there any way to change the default path where amrfinder looks for the database?

Thanks.

vbrover commented 1 year ago

Can you run this command?

amrfinder -u
vbrover commented 1 year ago

This URL is accessible: https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

vbrover commented 1 year ago
  1. Is there any way to separate database installation from download?

There is no separate step "installation". There are 2 steps: 1) download the files from FTP; 2) index them.

I downloaded and indexed the databases manually,

If the files are downloaded manually then they can be indexed by amrfinder_index.

https://github.com/ncbi/amr/wiki/Upgrading#database-updates

is there any way to change the default path where amrfinder looks for the database?

amrfinder -d DATABASE_DIR

evolarjun commented 1 year ago

I think Slava covered it above, but just wanted to add pointers to the documentation for amrfinder_index and amrfinder_update:

Please let us know if you are still having trouble or if you managed to get it working in case others are having this issue.

geboro commented 1 year ago

Can you run this command?

amrfinder -u

No. This gives the following error: 

Running: amrfinder -u
Software directory: '/home/bioinf/mambaforge/envs/bakta/bin/'
Software version: 3.11.11
Running: /home/bioinf/mambaforge/envs/bakta/bin/amrfinder_update -d /home/bioinf/mambaforge/envs/bakta/share/amrfinderplus/data
Looking up the published databases at https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

*** ERROR ***
CURL: Cannot read from https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

HOSTNAME: ?
SHELL: /bin/bash
PWD: /home/bioinf/bioinf_data/57_boge/Erwinia_tasmaniensis/Genomes/seti_Erwinia_tasmaniensis_54G
PATH: /home/bioinf/mambaforge/envs/bakta/bin:/home/bioinf/mambaforge/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
Progam name:  amrfinder_update
Command line: /home/bioinf/mambaforge/envs/bakta/bin/amrfinder_update -d /home/bioinf/mambaforge/envs/bakta/share/amrfinderplus/data

*** ERROR ***
'/home/bioinf/mambaforge/envs/bakta/bin/amrfinder_update'  -d '/home/bioinf/mambaforge/envs/bakta/share/amrfinderplus/data' > /tmp/amrfinder.8rWEYL/log
status = 256

HOSTNAME: ?
SHELL: /bin/bash
PWD: /home/bioinf/bioinf_data/57_boge/Erwinia_tasmaniensis/Genomes/seti_Erwinia_tasmaniensis_54G
PATH: /home/bioinf/mambaforge/envs/bakta/bin:/home/bioinf/mambaforge/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
Progam name:  amrfinder
Command line: amrfinder -u

Still,curl can access the available URL:

curl https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
 <head>
  <title>Index of /pathogen/Antimicrobial_resistance/AMRFinderPlus/database</title>
 </head>
 <body>
<h1>Index of /pathogen/Antimicrobial_resistance/AMRFinderPlus/database</h1>
<pre>Name                                              Last modified      Size  <hr><a href="/pathogen/Antimicrobial_resistance/AMRFinderPlus/">Parent Directory</a>                                                       -   
<a href="3.10/">3.10/</a>                                             2021-12-22 13:07    -   
<a href="3.2/">3.2/</a>                                              2019-10-31 11:24    -   
<a href="3.6/">3.6/</a>                                              2020-03-25 08:36    -   
<a href="3.8/">3.8/</a>                                              2020-10-02 15:43    -   
<a href="3.9/">3.9/</a>                                              2020-12-21 10:09    -   
<a href="3.10/">3.10/</a>                                              2022-10-12 10:00    -
<a href="3.11/">3.11/</a>                                              2022-10-12 10:00    -
<a href="latest/">latest/</a>                                             2021-12-22 13:19    -   
<hr></pre>

For now, I manually dowloaded the databases and formatted them with 'amrfinder_index' as suggested, and it runs fine when pointing to the database using '-d', thanks a lot.

Still, it would be good to know why 'amrfinder_update' is not working, since I am trying to use AMRfinder wrapped in 'bakta', and I cannot point to the appropriate database there.

I suspect there is a problem with proxy certificates, since the conda recipe installs its own version of 'curl' and 'openssl'. Setting this ip inside a conda environment is beyond my expertise though.

evolarjun commented 1 year ago

Thanks for the report. We haven't, so far, been able to replicate the issue, but we're trying to figure out what could be going on with this. There appears to be something different about the what libcurl is doing when called by AMRFinderPlus and what the command-line curl is doing, but we haven't figured anything out yet. We'll look into it and hopefully find something.

You're the second person we've heard having trouble with updates from within BAKTA. I don't think it's something BAKTA-specific, but as you suggest it could also be bioconda-related.

Do you know if you're running a version of command-line curl installed with bioconda?

Thanks again for your report, we're still trying to figure it out.

vbrover commented 1 year ago

Was your amrfinder_update or amrfinder -u successful on the same computer in the past? Is your amrfinder_update or amrfinder -u successful on a different computer?

vbrover commented 1 year ago

Please try the newly released amrfinder ver. 3.11.14 which will print cURL error messages.

ireneortega commented 5 months ago

I got the same error:

(campype) campype@campype-VirtualBox:~/CamPype$ amrfinder --update
Running: amrfinder --update
Software directory: '/home/campype/anaconda3/envs/campype/bin/'
Software version: 3.11.2
The number of threads cannot be greater than 1 on this computer
The current number of threads is 4, reducing to 1
Running: /home/campype/anaconda3/envs/campype/bin/amrfinder_update -d /home/campype/anaconda3/envs/campype/share/amrfinderplus/data
Looking up the published databases at https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

*** ERROR ***
CURL: Cannot read from https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

HOSTNAME: ?
SHELL: /bin/bash
PWD: /home/campype/CamPype
PATH: /home/campype/anaconda3/envs/campype/bin:/home/campype/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
Progam name:  amrfinder_update
Command line: /home/campype/anaconda3/envs/campype/bin/amrfinder_update -d /home/campype/anaconda3/envs/campype/share/amrfinderplus/data

*** ERROR ***
'/home/campype/anaconda3/envs/campype/bin/amrfinder_update'  -d '/home/campype/anaconda3/envs/campype/share/amrfinderplus/data' > /tmp/amrfinder.ujBNZb/log
status = 256

HOSTNAME: ?
SHELL: /bin/bash
PWD: /home/campype/CamPype
PATH: /home/campype/anaconda3/envs/campype/bin:/home/campype/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
Progam name:  amrfinder
Command line: amrfinder --update

What can I do?

vbrover commented 5 months ago

Can you connect to 

https://ftp.ncbi.nlm.nih.gov/pathogen/Antimicrobial_resistance/AMRFinderPlus/database/

from your browser on the computer where amrfinder is run? (Or using wget or curl etc.)

vbrover commented 5 months ago

Also it would be better to use the most recent available version (3.11.26) of amrfinder.