Open bgphd opened 5 years ago
You need to configure your aws client - have you completed the instructions as described here? https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
Hello, it is super easy to configure the client using the Windows installation package, viz:
D:\nanopore>aws --version aws-cli/1.16.181 Python/3.6.0 Windows/10 botocore/1.12.171
D:\nanopore>
the python method on Mac is still very buggy (at least in 10.13.6).
Bert
On Fri, Jun 21, 2019 at 1:17 PM mattloose notifications@github.com wrote:
You need to configure your aws client - have you completed the instructions as described here? https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nanopore-wgs-consortium/NA12878/issues/68?email_source=notifications&email_token=AKPPWL6Z2WVDIGONRYNDZPDP3UZNNA5CNFSM4H2UQ5L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJPQSI#issuecomment-504559689, or mute the thread https://github.com/notifications/unsubscribe-auth/AKPPWL7KAWWQW3HLKR7BCDLP3UZNNANCNFSM4H2UQ5LQ .
Oh, so my configuration is not the issue...
On Fri, Jun 21, 2019 at 1:17 PM mattloose notifications@github.com wrote:
You need to configure your aws client - have you completed the instructions as described here? https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nanopore-wgs-consortium/NA12878/issues/68?email_source=notifications&email_token=AKPPWL6Z2WVDIGONRYNDZPDP3UZNNA5CNFSM4H2UQ5L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJPQSI#issuecomment-504559689, or mute the thread https://github.com/notifications/unsubscribe-auth/AKPPWL7KAWWQW3HLKR7BCDLP3UZNNANCNFSM4H2UQ5LQ .
Hi,
I tried Adam Phillippy's suggestion this morning after two failed attempts to https:// sequences
Using AWS CLI I did the following and got the following results ( a different error than I had been getting, perhaps because I first entered some credentials, as shown below). Yes, my CLI is working properly (I think). Suggestions appreciated.
D:\nanopore>aws configure
AWS Access Key ID [None]: AKIA6F4YYBA
D:\nanopore\bam>aws --no-sign-request s3 cp s3://nanopore-human-wgs/rel3-nanopore-wgs288418386-FAB39088.fastq.gz . fatal error: An error occurred (400) when calling the HeadObject operation: Bad Request
D:\nanopore\bam>aws --no-sign-request s3 cp s3://nanopore-human-wgs/rel5-guppy-0.3.0-chunk10k.sorted.bam . fatal error: An error occurred (400) when calling the HeadObject operation: Bad Request
D:\nanopore\bam>aws --version aws-cli/1.16.181 Python/3.6.0 Windows/10 botocore/1.12.171
On Fri, Jun 21, 2019 at 1:17 PM mattloose notifications@github.com wrote:
You need to configure your aws client - have you completed the instructions as described here? https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nanopore-wgs-consortium/NA12878/issues/68?email_source=notifications&email_token=AKPPWL6Z2WVDIGONRYNDZPDP3UZNNA5CNFSM4H2UQ5L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJPQSI#issuecomment-504559689, or mute the thread https://github.com/notifications/unsubscribe-auth/AKPPWL7KAWWQW3HLKR7BCDLP3UZNNANCNFSM4H2UQ5LQ .
OK - I still think this is an issue with your client configuration. You default region name is us-west-2 but the data are held on eu-west-1 - there are some reports on the internet for an issue on the amazon client when you try and copy cross regions resulting in 400 errors as you are seeing. I suggest looking in to your region configuration.
Also - from the links you are giving, it looks as though you are trying to download all the fastq datasets - note that we have just made rel6 available which are the latest data.
Hi!
I have never gotten this to work :-{
It either times out or gives an error message.
I found some chromosome X assemblies elsewhere (presumbably from this same project, because it was Nanopore on NA12878) and successfully used that!
Now I see several more human genomes are available via HELEN and I would like to secure the bam files or at least the bam files of chromosome X. How can I do this?
Bert Gold
On Mon, Jun 24, 2019 at 11:38 AM Bert Gold bert.gold@billiontoone.com wrote:
Hi,
I tried Adam Phillippy's suggestion this morning after two failed attempts to https:// sequences
Using AWS CLI I did the following and got the following results ( a different error than I had been getting, perhaps because I first entered some credentials, as shown below). Yes, my CLI is working properly (I think). Suggestions appreciated.
D:\nanopore>aws configure AWS Access Key ID [None]: AKIA6F4YYBA
AWS Secret Access Key [None]: pg2HtYj8hi7IWPKw Default region name [None]: us-west-2 Default output format [None]: D:\nanopore\bam>aws --no-sign-request s3 cp s3://nanopore-human-wgs/rel3-nanopore-wgs288418386-FAB39088.fastq.gz . fatal error: An error occurred (400) when calling the HeadObject operation: Bad Request
D:\nanopore\bam>aws --no-sign-request s3 cp s3://nanopore-human-wgs/rel5-guppy-0.3.0-chunk10k.sorted.bam . fatal error: An error occurred (400) when calling the HeadObject operation: Bad Request
D:\nanopore\bam>aws --version aws-cli/1.16.181 Python/3.6.0 Windows/10 botocore/1.12.171
On Fri, Jun 21, 2019 at 1:17 PM mattloose notifications@github.com wrote:
You need to configure your aws client - have you completed the instructions as described here? https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nanopore-wgs-consortium/NA12878/issues/68?email_source=notifications&email_token=AKPPWL6Z2WVDIGONRYNDZPDP3UZNNA5CNFSM4H2UQ5L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJPQSI#issuecomment-504559689, or mute the thread https://github.com/notifications/unsubscribe-auth/AKPPWL7KAWWQW3HLKR7BCDLP3UZNNANCNFSM4H2UQ5LQ .
I've tried downloading the 2018 Nanopore NA12878 bam file to no avail.
I have tried the https:// method twice on two different computers (one mac, one workstation PC), using two different connections (wired), but it always fails out with partial downloads.
I have also installed the Amazon CLI, but I am getting this:
D:>cd nanopore
D:\nanopore>aws s3 cp s3://nanopore-human-wgs/rel3-nanopore-wgs-288418386-FAB39088.fastq.gz . fatal error: Unable to locate credentials
D:\nanopore>