Closed achave11-ucsc closed 7 months ago
The file might have been empty. An attempt to download the contents from the Google signed URL was unsuccessful, it seems like there's nothing to download from the provided URL.
@timestamp | @message | @log |
---|---|---|
2023-12-14 17:27:00.752 | START RequestId: d52d8472-9a3d-4847-a94b-dd495996227e Version: $LATEST |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:00.752 | [INFO] 2023-12-14T17:27:00.752Z d52d8472-9a3d-4847-a94b-dd495996227e azul.chalice Received GET request for '/fetch/repository/files/f6c7670c-2fa2-48eb-aa74-31b7ec94b148', with {"query": {"catalog": "anvil3-it", "version": "2022-06-01T00:00:00.000000Z"}, "headers": {"accept-encoding": "identity", "cloudfront-forwarded-proto": "https", "cloudfront-is-desktop-viewer": "true", "cloudfront-is-mobile-viewer": "false", "cloudfront-is-smarttv-viewer": "false", "cloudfront-is-tablet-viewer": "false", "cloudfront-viewer-asn": "14618", "cloudfront-viewer-country": "US", "host": "service.prod.anvil.gi.ucsc.edu", "user-agent": "python-urllib3/1.26.18", "via": "1.1 51391527dd8c879c45b44b119905c872.cloudfront.net (CloudFront)", "x-amz-cf-id": "OhuKMdeDonyPQiN2DP15opjHOIiqHZBvLWaq4TybRT8h54ZaZTT-6A==", "x-amzn-trace-id": "Root=1-657b3ae4-5f7e54562a47e5287ec1d28e", "x-forwarded-for": "35.168.152.160, 130.176.98.94", "x-forwarded-port": "443", "x-forwarded-proto": "https"}}. | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:01.229 | [INFO] 2023-12-14T17:27:01.229Z d52d8472-9a3d-4847-a94b-dd495996227e azul.terra Making GET request to 'https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef' | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:01.229 | [DEBUG] 2023-12-14T17:27:01.229Z d52d8472-9a3d-4847-a94b-dd495996227e azul.terra … without request body | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:06.296 | [DEBUG] 2023-12-14T17:27:06.296Z d52d8472-9a3d-4847-a94b-dd495996227e azul.chalice Returning 503 response with headers {"Access-Control-Allow-Origin": "*", "Access-Control-Allow-Headers": "Authorization,Content-Type,X-Amz-Date,X-Amz-Security-Token,X-Api-Key", "Strict-Transport-Security": "max-age=31536000; includeSubDomains"}. See next line for the first 1024 characters of the body. {"Code": "ServiceUnavailableError", "Message": "No response from https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef within 5 seconds"} |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:06.298 | REPORT RequestId: d52d8472-9a3d-4847-a94b-dd495996227e Duration: 5545.76 ms Billed Duration: 5546 ms Memory Size: 2048 MB Max Memory Used: 167 MB |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:06.339 | START RequestId: 86528675-501e-4785-aa89-b165f653bec3 Version: $LATEST |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:06.340 | [INFO] 2023-12-14T17:27:06.340Z 86528675-501e-4785-aa89-b165f653bec3 azul.chalice Received GET request for '/fetch/repository/files/f6c7670c-2fa2-48eb-aa74-31b7ec94b148', with {"query": {"catalog": "anvil3-it", "version": "2022-06-01T00:00:00.000000Z"}, "headers": {"accept-encoding": "identity", "cloudfront-forwarded-proto": "https", "cloudfront-is-desktop-viewer": "true", "cloudfront-is-mobile-viewer": "false", "cloudfront-is-smarttv-viewer": "false", "cloudfront-is-tablet-viewer": "false", "cloudfront-viewer-asn": "14618", "cloudfront-viewer-country": "US", "host": "service.prod.anvil.gi.ucsc.edu", "user-agent": "python-urllib3/1.26.18", "via": "1.1 51391527dd8c879c45b44b119905c872.cloudfront.net (CloudFront)", "x-amz-cf-id": "f3EZ2MA9kkyCHQ-gCN_YnXTDnPAz_VMfrR1zu_yqx-p6JBNPrJP9Dw==", "x-amzn-trace-id": "Root=1-657b3aea-69672e44613ae1807382f5d5", "x-forwarded-for": "35.168.152.160, 130.176.98.94", "x-forwarded-port": "443", "x-forwarded-proto": "https"}}. | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:06.363 | [INFO] 2023-12-14T17:27:06.363Z 86528675-501e-4785-aa89-b165f653bec3 azul.terra Making GET request to 'https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef' | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:06.363 | [DEBUG] 2023-12-14T17:27:06.363Z 86528675-501e-4785-aa89-b165f653bec3 azul.terra … without request body | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:11.390 | [DEBUG] 2023-12-14T17:27:11.390Z 86528675-501e-4785-aa89-b165f653bec3 azul.chalice Returning 503 response with headers {"Access-Control-Allow-Origin": "*", "Access-Control-Allow-Headers": "Authorization,Content-Type,X-Amz-Date,X-Amz-Security-Token,X-Api-Key", "Strict-Transport-Security": "max-age=31536000; includeSubDomains"}. See next line for the first 1024 characters of the body. {"Code": "ServiceUnavailableError", "Message": "No response from https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef within 5 seconds"} |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:11.392 | REPORT RequestId: 86528675-501e-4785-aa89-b165f653bec3 Duration: 5052.50 ms Billed Duration: 5053 ms Memory Size: 2048 MB Max Memory Used: 167 MB |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:11.424 | START RequestId: 9152a383-0212-4222-9869-5b9e5676edc0 Version: $LATEST |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:11.424 | [INFO] 2023-12-14T17:27:11.424Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.chalice Received GET request for '/fetch/repository/files/f6c7670c-2fa2-48eb-aa74-31b7ec94b148', with {"query": {"catalog": "anvil3-it", "version": "2022-06-01T00:00:00.000000Z"}, "headers": {"accept-encoding": "identity", "cloudfront-forwarded-proto": "https", "cloudfront-is-desktop-viewer": "true", "cloudfront-is-mobile-viewer": "false", "cloudfront-is-smarttv-viewer": "false", "cloudfront-is-tablet-viewer": "false", "cloudfront-viewer-asn": "14618", "cloudfront-viewer-country": "US", "host": "service.prod.anvil.gi.ucsc.edu", "user-agent": "python-urllib3/1.26.18", "via": "1.1 51391527dd8c879c45b44b119905c872.cloudfront.net (CloudFront)", "x-amz-cf-id": "Ul50UYg5UQISCs_BcDu-t2Apu8WBuBU0_T4qN12bINo-SrSNKWIEqg==", "x-amzn-trace-id": "Root=1-657b3aef-178257e03a3343e74146d6dd", "x-forwarded-for": "35.168.152.160, 130.176.98.71", "x-forwarded-port": "443", "x-forwarded-proto": "https"}}. | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:11.445 | [INFO] 2023-12-14T17:27:11.445Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra Making GET request to 'https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef' | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:11.445 | [DEBUG] 2023-12-14T17:27:11.445Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra … without request body | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:12.141 | [INFO] 2023-12-14T17:27:12.140Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra Got 200 response after 0.695s from GET to https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:12.141 | [DEBUG] 2023-12-14T17:27:12.141Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra … with response headers HTTPHeaderDict({'Date': 'Thu, 14 Dec 2023 17:27:11 GMT', 'Server': 'Apache', 'X-Frame-Options': 'SAMEORIGIN', 'Access-Control-Allow-Headers': 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization,Accept,Referer,X-App-Id,Origin', 'Access-Control-Allow-Methods': 'GET,POST,DELETE,PUT,PATCH,OPTIONS,HEAD', 'X-Content-Type-Options': 'nosniff', 'Strict-Transport-Security': 'max-age=31536000;includeSubDomains', 'Cache-Control': 'no-cache,no-store,must-revalidate', 'X-Request-ID': 'a0kK84Qj', 'Content-Type': 'application/json', 'Content-Length': '3862', 'Vary': 'Origin', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000'}) | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:12.141 | [DEBUG] 2023-12-14T17:27:12.141Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra … with response body b'{"id":"v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef","name":"summary_stats_variant_counts_windows_dchr20.GIH.counts.txt","self_uri...' | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:12.142 | [INFO] 2023-12-14T17:27:12.142Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra Making GET request to 'https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef/access/gcp-us*c3746ca1-61ec-431e-9c9c-1d14e9dcbafd' | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:12.142 | [DEBUG] 2023-12-14T17:27:12.142Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra … without request body | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:13.197 | [INFO] 2023-12-14T17:27:13.197Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra Got 200 response after 1.055s from GET to https://data.terra.bio/ga4gh/drs/v1/objects/v2_9c0c5f14-c714-3c5e-b96f-e4545ba098ef/access/gcp-us*c3746ca1-61ec-431e-9c9c-1d14e9dcbafd | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:13.197 | [DEBUG] 2023-12-14T17:27:13.197Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra … with response headers HTTPHeaderDict({'Date': 'Thu, 14 Dec 2023 17:27:12 GMT', 'Server': 'Apache', 'X-Frame-Options': 'SAMEORIGIN', 'Access-Control-Allow-Headers': 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization,Accept,Referer,X-App-Id,Origin', 'Access-Control-Allow-Methods': 'GET,POST,DELETE,PUT,PATCH,OPTIONS,HEAD', 'X-Content-Type-Options': 'nosniff', 'Strict-Transport-Security': 'max-age=31536000;includeSubDomains', 'Cache-Control': 'no-cache,no-store,must-revalidate', 'X-Request-ID': 'pYnyJ5dP', 'Content-Type': 'application/json', 'Content-Length': '1029', 'Vary': 'Origin', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000'}) | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:13.197 | [DEBUG] 2023-12-14T17:27:13.197Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.terra … with response body b'{"url":"https://storage.googleapis.com/fc-47de7dae-e8e6-429c-b760-b4ba49136eee/summary_stats/variant_counts_windows/dchr20.GI...' | 465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:13.198 | [DEBUG] 2023-12-14T17:27:13.198Z 9152a383-0212-4222-9869-5b9e5676edc0 azul.chalice Returning 200 response with headers {"Access-Control-Allow-Origin": "*", "Access-Control-Allow-Headers": "Authorization,Content-Type,X-Amz-Date,X-Amz-Security-Token,X-Api-Key", "Strict-Transport-Security": "max-age=31536000; includeSubDomains"}. See next line for the first 1024 characters of the body. {"Status": 302, "Location": "https://storage.googleapis.com/fc-47de7dae-e8e6-429c-b760-b4ba49136eee/summary_stats/variant_counts_windows/dchr20.GIH.counts.txt?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=tdr-ingest-sa%40datarepo-9e80499e.iam.gserviceaccount.com%2F20231214%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20231214T172713Z&X-Goog-Expires=900&X-Goog-SignedHeaders=host&requestedBy=azul-ucsc-1-public-anvilprod%40platform-anvil-prod.iam.gserviceaccount.com&userProject=datarepo-02ad84ea&X-Goog-Signature=592e011867cdc11a940295919de1c105f6bc43230b1b43bac70d84572642603479a24dde23fc41497261d12527b4c371533334279dbba6c31b3a7a0ff865e3f9ec352f0156c12cb592eb72058991d0e02d7997a9119eb3acd937711f9cda6b5f9beb6638fd25741e677572bcb3a08822e1889ccf2329a4d4ac94e7cdae0f1f234d9caf72f35817e952b277484c05e05d39462a207631f1d90fe7b283758785136c671a2c7aedcb50563a2cc0c6c608cb461ae60dd7e40a5cda690326496f5b42dea5d24d815ee495ccaf8a0281ee3feda43a0846d8173196f86a8063a8a0e7a67068c689dbb09859f4fd7bdb8f2859e08d43342c8128b1d4d003208 |
465330168186:/aws/lambda/azul-service-anvilprod |
2023-12-14 17:27:13.199 | REPORT RequestId: 9152a383-0212-4222-9869-5b9e5676edc0 Duration: 1775.67 ms Billed Duration: 1776 ms Memory Size: 2048 MB Max Memory Used: 168 MB |
465330168186:/aws/lambda/azul-service-anvilprod |
Assignee to consider next steps.
We do fetch the file metadata earlier during the test. We should only assert a non-empty body for files that have a size of zero. We are currently already extracting the file extension from the file metadata, and instead of adding a second parameter to pass down the call chain we should either pass the entire inner file dictionary or derive an instance of a new, IT-specific, attrs-decorated File
class from the file metadata and pass that down.
Passing test suffices for demo.
https://gitlab.prod.anvil.gi.ucsc.edu/ucsc/azul/-/jobs/17859/raw