search

luntergroup / octopus

Bayesian haplotype-based mutation calling
MIT License
299 stars 37 forks source link

Failed to download forest files:germline.v0.7.4.forest.gz and somatic.v0.7.4.forest.gz #226

Closed pmad-gumed closed 2 years ago

pmad-gumed commented 2 years ago

Describe the bug: Hi, I try to install octopus using command: install.py --dependencies --forests but it stops with error on downloading forest files step.

Version: octopus version 0.7.4 (develop 738c90e3) Target: x86_64 Linux 4.9.0-17-amd64 SIMD extension: AVX512 Compiler: GNU 11.2.0 Boost: 1_76

Command line to install octopus and ouput:

install.py --dependencies --forests ... ... ... Install the project... -- Install configuration: "Release" -- Installing: /opt/octopus/bin/octopus Downloading https://storage.googleapis.com/luntergroup/octopus/forests/germline.v0.7.4.forest.gz to /opt/octopus/resources/forests/germline.v0.7.4.forest.gz Failed to download forest germline.v0.7.4.forest.gz Downloading https://storage.googleapis.com/luntergroup/octopus/forests/somatic.v0.7.4.forest.gz to /opt/octopus/resources/forests/somatic.v0.7.4.forest.gz Failed to download forest somatic.v0.7.4.forest.gz

xie186 commented 2 years ago

@pmad-gumed , did you find a solution?

I tried the command line below to download the forest files from google cloud&prefix=&forceOnObjectsSortingFiltering=false):

gsutil -m cp -r   "gs://luntergroup/octopus/forests/"   .

It throwed error message shown as below:

Copying gs://luntergroup/octopus/forests/README...
Copying gs://luntergroup/octopus/forests/cell.v0.7.4.forest.gz...
Copying gs://luntergroup/octopus/forests/germline.v0.7.2.forest.gz...
==> NOTE: You are downloading one or more large file(s), which would
run significantly faster if you enabled sliced object downloads. This
feature is enabled by default but requires that compiled crcmod be
installed (see "gsutil help crcmod").

Copying gs://luntergroup/octopus/forests/germline.v0.7.4.forest.gz...
Copying gs://luntergroup/octopus/forests/polyploid.v0.7.2.forest.gz...
Copying gs://luntergroup/octopus/forests/somatic.v0.7.2.forest.gz...
Copying gs://luntergroup/octopus/forests/somatic.v0.7.4.forest.gz...
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/luntergroup/o/octopus%2Fforests%2FREADME?generation=1531814130741188&alt=media>: response: <{'x-guploader-uploadid': 'ADPycdvj1wNstep23gM8IFUDLqtLK2AwnmZ65l4xqzXak5EG1ETQmiZJ2N0HdWkoZUDSwyT1AB-cbha0tnre4oVeg1A', 'content-type': 'text/html; charset=UTF-8', 'date': 'Wed, 19 Jan 2022 14:38:29 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Wed, 19 Jan 2022 14:38:29 GMT', 'cache-control': 'private, max-age=0', 'content-length': '69', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}>, content <The project to be billed is associated with a closed billing account.>
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/luntergroup/o/octopus%2Fforests%2Fcell.v0.7.4.forest.gz?generation=1621422570380545&alt=media>: response: <{'x-guploader-uploadid': 'ADPycduRa5pbKYZPBJkEinhVS9BV3Djj3U9X3m2XmMH2aXszr7u1bQ4VR4p1vFIxXHrJhElrC2RwJANjpUCdjGKiSggfvN6GRA', 'content-type': 'text/html; charset=UTF-8', 'date': 'Wed, 19 Jan 2022 14:38:30 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Wed, 19 Jan 2022 14:38:30 GMT', 'cache-control': 'private, max-age=0', 'content-length': '69', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}>, content <The project to be billed is associated with a closed billing account.>
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/luntergroup/o/octopus%2Fforests%2Fgermline.v0.7.4.forest.gz?generation=1620131419076793&alt=media>: response: <{'x-guploader-uploadid': 'ADPycdtx1YKiVSaqFAzHCVxVRhPoP7eHS8rjjr2fpBRr7O399StCN4uvb2-mxKEcJPpxX8UTSi5PTjBOeOFWviyOFCM', 'content-type': 'text/html; charset=UTF-8', 'date': 'Wed, 19 Jan 2022 14:38:30 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Wed, 19 Jan 2022 14:38:30 GMT', 'cache-control': 'private, max-age=0', 'content-length': '69', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}>, content <The project to be billed is associated with a closed billing account.>
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/luntergroup/o/octopus%2Fforests%2Fgermline.v0.7.2.forest.gz?generation=1617207002576956&alt=media>: response: <{'x-guploader-uploadid': 'ADPycdtBPLPHDS_pl4nOFPVEyE3wDFf9REZ3fTnl4YbqM3J8tYGGxoywrckXaCcpd6Hz1qeZx-0PoNPIllEgY4WU5NM', 'content-type': 'text/html; charset=UTF-8', 'date': 'Wed, 19 Jan 2022 14:38:30 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Wed, 19 Jan 2022 14:38:30 GMT', 'cache-control': 'private, max-age=0', 'content-length': '69', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}>, content <The project to be billed is associated with a closed billing account.>
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/luntergroup/o/octopus%2Fforests%2Fsomatic.v0.7.2.forest.gz?generation=1617207010410871&alt=media>: response: <{'x-guploader-uploadid': 'ADPycdsEyf5MnZHZYjAHT2ph_ojtOytpl6VPn0RaiVXSbiJsgn6UZWj5vqUh6eap6O_galyzCBlY1V2_PiUXB_5BMUs', 'content-type': 'text/html; charset=UTF-8', 'date': 'Wed, 19 Jan 2022 14:38:30 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Wed, 19 Jan 2022 14:38:30 GMT', 'cache-control': 'private, max-age=0', 'content-length': '69', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}>, content <The project to be billed is associated with a closed billing account.>
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/luntergroup/o/octopus%2Fforests%2Fpolyploid.v0.7.2.forest.gz?generation=1617207027680995&alt=media>: response: <{'x-guploader-uploadid': 'ADPycdsQUdqEmHUvjHX9oVhv07U2KdIzpTfS_GEpVXdH8kwfC1V0UmqIJScFR-fHMTZeFQnDDtOTTU0qJhU-KjtT9V8', 'content-type': 'text/html; charset=UTF-8', 'date': 'Wed, 19 Jan 2022 14:38:30 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Wed, 19 Jan 2022 14:38:30 GMT', 'cache-control': 'private, max-age=0', 'content-length': '69', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}>, content <The project to be billed is associated with a closed billing account.>
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/luntergroup/o/octopus%2Fforests%2Fsomatic.v0.7.4.forest.gz?generation=1620131427272374&alt=media>: response: <{'x-guploader-uploadid': 'ADPycdvKzub6_XAqE9XNUdvdExz-bXjG95-RWjdEE2Ldr_aBbnrzkJ0AwVwUSbaYf8e5zadVJB9lSs3ITge17U0qmtg', 'content-type': 'text/html; charset=UTF-8', 'date': 'Wed, 19 Jan 2022 14:38:30 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Wed, 19 Jan 2022 14:38:30 GMT', 'cache-control': 'private, max-age=0', 'content-length': '69', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}>, content <The project to be billed is associated with a closed billing account.>
CommandException: 7 files/objects could not be transferred.
pmad-gumed commented 2 years ago

@pmad-gumed , did you find a solution?

No - I'm still waiting for the authors' response...

dancooke commented 2 years ago

Hi @pmad-gumed - sorry for the slow response. The forest failed to download as I my debit card paying for the cloud account expired. I've now updated so it should work now.

pmad-gumed commented 2 years ago

Hi @dancooke - thank you very much - everything works now.