rpetit3 / fastq-dl

Download FASTQ files from SRA or ENA repositories.
MIT License
271 stars 25 forks source link

Error STATUS 400 #20

Closed pavlo888 closed 7 months ago

pavlo888 commented 1 year ago

Hi @rpetit3

I am very excited to use the fastq-dl tool to download some SRA data but I am having some issues.

When I run the following command: fastq-dl SRR12333278 SRA I get the following error: 2023-05-29 00:26:51:root:ERROR - There was an issue querying ENA, exiting... 2023-05-29 00:26:51:root:ERROR - STATUS: 400 2023-05-29 00:26:51:root:ERROR - TEXT: Invalid fieldName(s) supplied: cram_index_ftp,cram_index_aspera,cram_index_galaxy

I thought I might be having some connection issues with the Aspera Connect tool (ascp). However, this tool is already in my current temporary path since after running the command "echo $PATH" I get: /Users/pavlo/mambaforge/envs/fastq-dl/bin:/Users/pavlo/mambaforge/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Aspera

Any idea what the problem might be?

rpetit3 commented 1 year ago

Hi @pavlo888

Can you verify the version number?

Thanks!

mbhall88 commented 1 year ago

Using the latest version (v2.0.1) of fastq-dl I can't reproduce this error

fastq-dl -a SRR12333278 --provider sra

Downloads the fastqs no problem.

Given your command line usage, I suspect you're using an old version @pavlo888?

pavlo888 commented 1 year ago

Hi @rpetit3,

I have checked my current fast-dl version and it is 1.1.1. I have installed this version following the conda recipe as suggested in the GitHub. (fastq-dl) pavlo@Pablos-MacBook-Air sra-reads-amf-lsu % fastq-dl --version fastq-dl 1.1.1

How could I update the current version?

mbhall88 commented 1 year ago

What happens when you run conda install fastq-dl=2.0?

pavlo888 commented 1 year ago

When I do the command conda install fastq-dl=2.0 I get `Collecting package metadata (current_repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Solving environment: | Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. failed

UnsatisfiableError: `

Mind you I already added the channels bioconda and conda-forge

mbhall88 commented 1 year ago

Hmm weird. Is this an environment with other packages in it or a new one just for fastq-dl? What is the python version in the environment?

rpetit3 commented 1 year ago

Agree with @mbhall88

At this point it might be easier to create a new environment:

# If ok overwriting existing
mamba create -n fastq-dl 'fastq-dl=2.0.1'

# else
mamba create -n fastq-dl-v2 'fastq-dl=2.0.1'
pavlo888 commented 1 year ago

@mbhall88

I tried to create a new environment for the new fastq-dl version. The python version is 3.9.6

@rpetit3 I have tried creating a new environment with mamba and I got the following error `Could not solve for environment specs Encountered problems while solving:

The environment can't be solved, aborting the operation`

Perhaps I should also mention that I am on a Mac environment.

rpetit3 commented 1 year ago

There's our issue. Looks like sra-tools is not built for OSX

https://github.com/bioconda/bioconda-recipes/blob/master/recipes/sra-tools/meta.yaml#L14-L16

Might have to go the docker route for the time being. I'll see if there are any folks on the SRA/NCBI side interested in fixing the build for OSX.

The newer versions of sra-tools no longer require the manual user interaction (haha had to implement a hacky solution to avoid this)

pavlo888 commented 1 year ago

I see. Thanks for helping me out on this one.

Is there any place where I can find the documentation for using the docker image of fastq-dl? I have only found this repository https://hub.docker.com/r/biodancer/fastq-dl/tags but I am not so sure it is the right one.

I am not sure how to mount my working directory to the image and start downloading some SRA files.

rpetit3 commented 1 year ago

No problem, you'll want the one from biocontainers, as it's based on the Bioconda build.

docker pull quay.io/biocontainers/fastq-dl:2.0.1--pyhdfd78af_0

Then something like this to download to ${PWD}

docker run --rm -u $(id -u):$(id -g) -v ${PWD}:/data quay.io/biocontainers/fastq-dl:2.0.1--pyhdfd78af_0 \
    fastq-dl --accession SRR10532144 --outdir /data/SRR10532144
rpetit3 commented 7 months ago

Hi @pavlo888,

I think this might have gotten sorted, if not, please feel free to reopen!

Cheers, Robert