search

aws-quickstart / quickstart-illumina-dragen

AWS Quick Start Team
Apache License 2.0
24 stars 26 forks source link

Very slow speeds for downloading fastq from s3 #47

Open siyervervetx opened 3 years ago

siyervervetx commented 3 years ago

Download speeds are very slow from within the dragen code that downloads from s3. Please see here for a subset of the log file.

I verified that its not a problem with the instance because a direct "aws s3 cp" command shows a download speed > 100 mb/s

AWS support suggested that this was because of high CPU utilization. Could you please help resolve this? Is there a parameter I can pass to dragen to speed up?

I am running this on an F1.16X machine and its very expensive to pay for slow download speeds caused by the program.

STL_Tre_0.75.R1.gz: 28880MB received (+48MB in 10s, 0.6MB/s)
STL_Tre_0.75.R2.gz: 30336MB received (+48MB in 10s, 0.6MB/s)
STL_Tre_0.75.R2.gz: 30432MB received (+96MB in 10s, 8.6MB/s)
STL_Tre_0.75.R1.gz: 28976MB received (+96MB in 10s, 8.5MB/s)
STL_Tre_0.75.R2.gz: 30672MB received (+240MB in 10s, 7.7MB/s)
STL_Tre_0.75.R1.gz: 29200MB received (+224MB in 10s, 7.1MB/s)
STL_Tre_0.75.R2.gz: 30800MB received (+128MB in 10s, 9.3MB/s)
STL_Tre_0.75.R1.gz: 29328MB received (+128MB in 10s, 5.3MB/s)
STL_Tre_0.75.R2.gz: 30816MB received (+16MB in 10s, 1.5MB/s)
STL_Tre_0.75.R2.gz: 30944MB received (+128MB in 10s, 1.9MB/s)
STL_Tre_0.75.R1.gz: 29456MB received (+128MB in 10s, 1.9MB/s)
STL_Tre_0.75.R2.gz: 31248MB received (+304MB in 10s, 10.8MB/s)
STL_Tre_0.75.R1.gz: 29744MB received (+288MB in 10s, 10.2MB/s)
STL_Tre_0.75.R2.gz: 31472MB received (+224MB in 10s, 22.3MB/s)
STL_Tre_0.75.R1.gz: 29952MB received (+208MB in 10s, 10.6MB/s)
STL_Tre_0.75.R2.gz: 31504MB received (+32MB in 10s, 3.2MB/s)
STL_Tre_0.75.R1.gz: 30000MB received (+48MB in 10s, 2.9MB/s)
STL_Tre_0.75.R2.gz: 31536MB received (+32MB in 10s, 2.0MB/s)
STL_Tre_0.75.R2.gz: 31616MB received (+80MB in 10s, 6.2MB/s)
STL_Tre_0.75.R1.gz: 30080MB received (+80MB in 10s, 6.2MB/s)
STL_Tre_0.75.R2.gz: 31664MB received (+48MB in 10s, 2.0MB/s)
STL_Tre_0.75.R1.gz: 30128MB received (+48MB in 10s, 2.0MB/s)
STL_Tre_0.75.R1.gz: 30224MB received (+96MB in 10s, 9.5MB/s)
STL_Tre_0.75.R2.gz: 31792MB received (+128MB in 10s, 12.2MB/s)
STL_Tre_0.75.R1.gz: 30624MB received (+400MB in 10s, 39.6MB/s)
STL_Tre_0.75.R2.gz: 32224MB received (+432MB in 10s, 43.0MB/s)
STL_Tre_0.75.R1.gz: 31152MB received (+528MB in 10s, 51.8MB/s)
STL_Tre_0.75.R2.gz: 32784MB received (+560MB in 10s, 55.5MB/s)
STL_Tre_0.75.R2.gz: 33360MB received (+576MB in 10s, 55.7MB/s)
STL_Tre_0.75.R1.gz: 31696MB received (+544MB in 10s, 52.0MB/s)
STL_Tre_0.75.R1.gz: 31792MB received (+96MB in 10s, 7.8MB/s)
STL_Tre_0.75.R2.gz: 33472MB received (+112MB in 10s, 9.0MB/s)
STL_Tre_0.75.R1.gz: 31840MB received (+48MB in 10s, 2.6MB/s)
STL_Tre_0.75.R2.gz: 33520MB received (+48MB in 10s, 2.6MB/s)
STL_Tre_0.75.R2.gz: 33664MB received (+144MB in 10s, 10.3MB/s)
STL_Tre_0.75.R1.gz: 31984MB received (+144MB in 10s, 10.2MB/s)
STL_Tre_0.75.R2.gz: 33808MB received (+144MB in 10s, 13.2MB/s)
STL_Tre_0.75.R1.gz: 32112MB received (+128MB in 10s, 11.8MB/s)
STL_Tre_0.75.R2.gz: 33872MB received (+64MB in 10s, 5.4MB/s)
STL_Tre_0.75.R1.gz: 32176MB received (+64MB in 10s, 5.3MB/s)
STL_Tre_0.75.R2.gz: 33920MB received (+48MB in 10s, 4.7MB/s)
STL_Tre_0.75.R1.gz: 32224MB received (+48MB in 10s, 4.6MB/s)
quilesw commented 3 years ago

Apologies this is happening. Can you please contact me at my work email (quilesw@amazon.com) so that we can get more information and try to identify what is going on?