search

cogeotiff / rio-tiler

User friendly Rasterio plugin to read raster datasets.
https://cogeotiff.github.io/rio-tiler/
BSD 3-Clause "New" or "Revised" License
511 stars 106 forks source link

Investigate higher HEAD/GET Request with GDAL >3.7 #697

Open vincentsarago opened 5 months ago

vincentsarago commented 5 months ago

with https://github.com/cogeotiff/rio-tiler/blob/main/tests/benchmarks/requests.py we test some tile request fetching and use tilebench to monitor the number or GET/HEAD requests made.

with the latest commits, this test is failing https://github.com/cogeotiff/rio-tiler/blob/4d95c28962ac0f0d1465038d44b12f2c03ac2367/tests/benchmarks/requests.py#L35-L67

👇 this used to return HEAD: 1 and GET 3 🤷 

tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 | jq
{
  "HEAD": {
    "count": 3
  },
  "GET": {
    "count": 4,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 3.494839668273926
}
vincentsarago commented 5 months ago

Rasterio 1.3.9 / GDAL 3.6.4

tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 | jq
{
  "HEAD": {
    "count": 1
  },
  "GET": {
    "count": 3,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 2.6649599075317383
}

Rasterio 1.3.10 / GDAL 3.7.3

{
  "HEAD": {
    "count": 3
  },
  "GET": {
    "count": 4,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 3.2888236045837402
}

Rasterio 1.3.10 / GDAL 3.8.4

 tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 | jq
{
  "HEAD": {
    "count": 3
  },
  "GET": {
    "count": 4,
    "bytes": 2293760,
    "ranges": [
      "0-32767",
      "9011200-10158079",
      "11075584-12189695"
    ]
  },
  "Timing": 2.7414448261260986
}

CURL Logs

tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 --add-stdout | jq -r '.logs[]' | grep "CURL_INFO_HEADER_OUT"
CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/ HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
``` tilebench profile https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif --tile 11-493-741 --add-stdout | jq '.logs' [ "Entering env context: ", "Starting outermost env", "No GDAL environment exists", "New GDAL environment created", "Installing FilePath filesystem handler plugin...", "GDAL_DATA found in environment.", "PROJ_LIB found in environment.", "Started GDALEnv: self=.", "Entered env context: ", "Got a copy of environment options", "Entering env context: ", "Got a copy of environment options", "Entered env context: ", "Sharing flag: 0", "CPLE_None in HTTP: libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0", "CPLE_None in HTTP: GDAL was built against curl 7.87.0, but is running against 8.4.0.", "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults", "CPLE_None in CURL_INFO_TEXT: Trying 52.218.232.177:443...", "CPLE_None in CURL_INFO_TEXT: Connected to sentinel-cogs.s3.us-west-2.amazonaws.com (52.218.232.177) port 443", "CPLE_None in CURL_INFO_TEXT: ALPN: curl offers h2,http/1.1", "CPLE_None in CURL_INFO_TEXT: (304) (OUT), TLS handshake, Client hello (1):", "CPLE_None in CURL_INFO_TEXT: CAfile: /Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/certifi/cacert.pem", "CPLE_None in CURL_INFO_TEXT: CApath: none", "CPLE_None in CURL_INFO_TEXT: (304) (IN), TLS handshake, Server hello (2):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Certificate (11):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Server key exchange (12):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Server finished (14):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (OUT), TLS handshake, Finished (20):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):", "CPLE_None in CURL_INFO_TEXT: TLSv1.2 (IN), TLS handshake, Finished (20):", "CPLE_None in CURL_INFO_TEXT: SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256", "CPLE_None in CURL_INFO_TEXT: ALPN: server accepted http/1.1", "CPLE_None in CURL_INFO_TEXT: Server certificate:", "CPLE_None in CURL_INFO_TEXT: subject: CN=*.s3-us-west-2.amazonaws.com", "CPLE_None in CURL_INFO_TEXT: start date: Mar 15 00:00:00 2024 GMT", "CPLE_None in CURL_INFO_TEXT: expire date: Feb 15 23:59:59 2025 GMT", "CPLE_None in CURL_INFO_TEXT: subjectAltName: host \"sentinel-cogs.s3.us-west-2.amazonaws.com\" matched cert's \"*.s3.us-west-2.amazonaws.com\"", "CPLE_None in CURL_INFO_TEXT: issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01", "CPLE_None in CURL_INFO_TEXT: SSL certificate verify ok.", "CPLE_None in CURL_INFO_TEXT: using HTTP/1.1", "CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1", "Host: sentinel-cogs.s3.us-west-2.amazonaws.com", "User-Agent: GDAL/3.7.3", "Accept: */*", "", "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 200 OK", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: glv7Xi1AW/YS7UJsgsQAoYokXq/gB35c8LMEJYo2FLKpI+o1tR6xsyfWKou6TgrfZgcuY2ooDYQ=", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 2VMQ56MHK9DHVN5M", "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:32 GMT", "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT", "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256", "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable", "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes", "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized", "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3", "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 200959614", "CPLE_None in CURL_INFO_HEADER_IN: ", "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact", "CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614 response_code=200", "CPLE_None in VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...", "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults", "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]", "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to", "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com", "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1", "Host: sentinel-cogs.s3.us-west-2.amazonaws.com", "User-Agent: GDAL/3.7.3", "Accept: */*", "Range: bytes=0-32767", "", "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: pTwUP0oEk8NsXKYMEHOeoWlAF5VlM6s/F4HLQPcdYMBbkPD13uqFkzOzdrCLv/JkFVg1LMJdeIc=", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 2VMYTCRG7WAEDXPJ", "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:32 GMT", "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT", "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256", "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable", "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes", "CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 0-32767/200959614", "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized", "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3", "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 32768", "CPLE_None in CURL_INFO_HEADER_IN: ", "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact", "CPLE_None in VSICURL: Got response_code=206", "CPLE_None in GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11f249ca0) succeeds as GTiff.", "Nodata success: 1, Nodata value: 0.000000", "Nodata success: 1, Nodata value: 0.000000", "Nodata success: 1, Nodata value: 0.000000", "Dataset is started.", "Exiting env context: ", "Cleared existing options", "Stopped GDALEnv .", "No GDAL environment exists", "New GDAL environment created", "GDAL_DATA found in environment.", "PROJ_LIB found in environment.", "Started GDALEnv: self=.", "Exited env context: ", "CPLE_None in GTiff: ScanDirectories()", "CPLE_None in GTiff: Opened 5490x5490 overview.", "CPLE_None in GTiff: Opened 2745x2745 overview.", "CPLE_None in GTiff: Opened 1373x1373 overview.", "CPLE_None in GTiff: Opened 687x687 overview.", "Matched. confidence=100, c_code=b'3857', c_name=b'EPSG'", "Matched. confidence=100, c_code=b'32615', c_name=b'EPSG'", "CPLE_None in VRT: No valid sources found for band in VRT file ", "CPLE_None in GDAL: GDALOpen(PROJCS[\"WGS 84 / UTM zone 15N\",GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-93],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH],AUTHORITY[\"EPSG\",\"32615\"]]399960.0,10.0,0.0,5000040.0,0.0,-10.0, this=0x13fc0a1f0) succeeds as VRT.", "CPLE_None in GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538", "Created exact transformer", "CPLE_None in GDAL: GDALClose(PROJCS[\"WGS 84 / UTM zone 15N\",GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-93],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH],AUTHORITY[\"EPSG\",\"32615\"]]399960.0,10.0,0.0,5000040.0,0.0,-10.0, this=0x13fc0a1f0)", "Exported CRS to WKT.", "Warp_extras: {'init_dest': 'NO_DATA'}", "Nodata success: 1, Nodata value: 0.000000", "Nodata success: 1, Nodata value: 0.000000", "Nodata success: 1, Nodata value: 0.000000", "Dataset is started.", "Output nodata value read from file: 0.0", "Output nodata value read from file: 0.0", "Output nodata value read from file: 0.0", "Output nodata values: [0.0, 0.0, 0.0]", "all_valid: False", "mask_flags: ([], [], [])", "Jump straight to _read()", "Window: None", "IO window xoff=0.0 yoff=0.0 width=1391.0 height=1391.0", "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc0a1f0) creation.", "CPLE_None in VSICURL: GetFileList(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A)", "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults", "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]", "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to", "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com", "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/ HTTP/1.1", "Host: sentinel-cogs.s3.us-west-2.amazonaws.com", "User-Agent: GDAL/3.7.3", "Accept: */*", "", "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXKH2VG0DDEA2TP", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: 9ZbkWtfMg77R0o/nF4+/8n2OmcPdpK0ibQYNx5hoUXfPkKO3L4KlHK5qiimT/t5SmuSFoajsV5k=", "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: application/xml", "CPLE_None in CURL_INFO_HEADER_IN: Transfer-Encoding: chunked", "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:31 GMT", "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3", "CPLE_None in CURL_INFO_HEADER_IN: ", "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact", "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults", "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]", "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to", "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com", "CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml HTTP/1.1", "Host: sentinel-cogs.s3.us-west-2.amazonaws.com", "User-Agent: GDAL/3.7.3", "Accept: */*", "", "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXNYD4M1D6W2XSH", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: ww4bqRaqhxyjtj+DTfhePvtfVUJnIYdVC8HlpSTobCbJFGRomigMMM06NOTZZnCZ2DC8ntOMK1Y=", "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: application/xml", "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:31 GMT", "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3", "CPLE_None in CURL_INFO_HEADER_IN: ", "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact", "CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml): response_code=404, server error msg=HTTP/1.1 404 Not Found", "x-amz-request-id: EJXNYD4M1D6W2XSH", "x-amz-id-2: ww4bqRaqhxyjtj+DTfhePvtfVUJnIYdVC8HlpSTobCbJFGRomigMMM06NOTZZnCZ2DC8ntOMK1Y=", "Content-Type: application/xml", "Date: Thu, 18 Apr 2024 13:54:31 GMT", "Server: AmazonS3", "", "", "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults", "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]", "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to", "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com", "CPLE_None in CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML HTTP/1.1", "Host: sentinel-cogs.s3.us-west-2.amazonaws.com", "User-Agent: GDAL/3.7.3", "Accept: */*", "", "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXQCKA8MA3349AW", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: Lbj0y6wdZs3xlpQjojuE0A7vFO/skRqxCSFwJRMcbFT7Uj8jRRp0D+hCtYazzSXB2Rax1RghP7k=", "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: application/xml", "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:31 GMT", "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3", "CPLE_None in CURL_INFO_HEADER_IN: ", "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact", "CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML): response_code=404, server error msg=HTTP/1.1 404 Not Found", "x-amz-request-id: EJXQCKA8MA3349AW", "x-amz-id-2: Lbj0y6wdZs3xlpQjojuE0A7vFO/skRqxCSFwJRMcbFT7Uj8jRRp0D+hCtYazzSXB2Rax1RghP7k=", "Content-Type: application/xml", "Date: Thu, 18 Apr 2024 13:54:31 GMT", "Server: AmazonS3", "", "", "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11f416860) creation.", "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc349c0) creation.", "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc35350) creation.", "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc48aa0) creation.", "CPLE_None in GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13fc534c0) creation.", "CPLE_None in GDAL: GDAL_CACHEMAX = 12288 MB", "CPLE_None in VSICURL: Downloading 9011200-10158079 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...", "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults", "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]", "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to", "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com", "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1", "Host: sentinel-cogs.s3.us-west-2.amazonaws.com", "User-Agent: GDAL/3.7.3", "Accept: */*", "Range: bytes=9011200-10158079", "", "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: +NsF6qFLnSt0sbQ9WvuJl3iuifVS2Ggbah7FMHrObNpF9hc3IDG7IJPjkuFm8VlBdq8/m4pEpHE=", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: EJXS85WYC3GDRM76", "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:33 GMT", "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT", "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256", "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable", "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes", "CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 9011200-10158079/200959614", "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized", "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3", "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 1146880", "CPLE_None in CURL_INFO_HEADER_IN: ", "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact", "CPLE_None in VSICURL: Got response_code=206", "CPLE_None in GDAL: GDALWarpKernel()::GWKNearestByte() Src=1824,1911,351x130 Dst=0,0,348x128", "CPLE_None in VSICURL: Downloading 11075584-12189695 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...", "CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults", "CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x13f8f81e0 [serially]", "CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to", "CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com", "CPLE_None in CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1", "Host: sentinel-cogs.s3.us-west-2.amazonaws.com", "User-Agent: GDAL/3.7.3", "Accept: */*", "Range: bytes=11075584-12189695", "", "CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: dBrzELUWdBHGmu7gG4b5UgZ1SgZfl3nn4OF+bQjnPGRxHmUnjcHE24thEHncL6q6j2AixFF3GYI=", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 4B1HMCNGJGFWVKXB", "CPLE_None in CURL_INFO_HEADER_IN: Date: Thu, 18 Apr 2024 13:54:34 GMT", "CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT", "CPLE_None in CURL_INFO_HEADER_IN: ETag: \"3e23880cd7ac8a32ae71c55ad4f85c5d-24\"", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING", "CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256", "CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable", "CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes", "CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 11075584-12189695/200959614", "CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized", "CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3", "CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 1114112", "CPLE_None in CURL_INFO_HEADER_IN: ", "CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact", "CPLE_None in VSICURL: Got response_code=206", "CPLE_None in GDAL: GDALWarpKernel()::GWKNearestByte() Src=1824,2039,351x130 Dst=0,128,348x128", "CPLE_None in GDAL: GDALWarpKernel()::GWKNearestByte() Src=1823,2167,352x94 Dst=0,256,348x92", "IO window xoff=0.0 yoff=0.0 width=1391.0 height=1391.0", "CPLE_None in GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11f249ca0)", "Exiting env context: ", "Cleared existing options", "Stopped GDALEnv .", "Exiting outermost env", "Exited env context: " ] ```
vincentsarago commented 5 months ago

Some weird stuff from the logs:

CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
CPLE_None in VSICURL: GetFileList(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A)
CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml): response_code=404, server error msg=HTTP/1.1 404 Not Found
CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML): response_code=404, server error msg=HTTP/1.1 404 Not Found
rouault commented 5 months ago

I don't confirm that using plain GDAL command line (3.8.5 or master):

$ CPL_VSIL_SHOW_NETWORK_STATS=YES GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR CPL_DEBUG=ON gdalinfo /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
HTTP: libcurl/7.68.0 GnuTLS/3.6.13 zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
VSICURL: Downloading 0-16383 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x55daa42fd160) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GTiff: ScanDirectories()
GTiff: Opened 5490x5490 overview.
GTiff: Opened 2745x2745 overview.
GTiff: Opened 1373x1373 overview.
GTiff: Opened 687x687 overview.
OGR_CT: Source CRS: 'EPSG:32615'
OGR_CT: Target CRS: 'GEOGCRS["WGS 84",DATUM["World Geodetic System 1984",ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]],ID["EPSG",6326]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8901]],CS[ellipsoidal,2],AXIS["geodetic latitude (Lat)",north,ORDER[1],ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]],AXIS["geodetic longitude (Lon)",east,ORDER[2],ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]]]'
Driver: GTiff/GeoTIFF
Files: /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
Size is 10980, 10980
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 15N",
    BASEGEOGCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["UTM zone 15N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-93,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["(E)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["(N)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["Navigation and medium accuracy spatial referencing."],
        AREA["Between 96°W and 90°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - Manitoba; Nunavut; Ontario. Ecuador -Galapagos. Guatemala. Mexico. United States (USA)."],
        BBOX[0,-96,84,-90]],
    ID["EPSG",32615]]
Data axis to CRS axis mapping: 1,2
Origin = (399960.000000000000000,5000040.000000000000000)
Pixel Size = (10.000000000000000,-10.000000000000000)
Metadata:
  OVR_RESAMPLING_ALG=AVERAGE
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=DEFLATE
  INTERLEAVE=PIXEL
  PREDICTOR=2
Corner Coordinates:
Upper Left  (  399960.000, 5000040.000) ( 94d16'21.18"W, 45d 8'48.29"N)
Lower Left  (  399960.000, 4890240.000) ( 94d15' 4.02"W, 44d 9'30.66"N)
Upper Right (  509760.000, 5000040.000) ( 92d52'33.00"W, 45d 9'13.57"N)
Lower Right (  509760.000, 4890240.000) ( 92d52'40.53"W, 44d 9'55.08"N)
Center      (  454860.000, 4945140.000) ( 93d34' 9.68"W, 44d39'29.53"N)
Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red
  NoData Value=0
  Overviews: 5490x5490, 2745x2745, 1373x1373, 687x687
Band 2 Block=1024x1024 Type=Byte, ColorInterp=Green
  NoData Value=0
  Overviews: 5490x5490, 2745x2745, 1373x1373, 687x687
Band 3 Block=1024x1024 Type=Byte, ColorInterp=Blue
  NoData Value=0
  Overviews: 5490x5490, 2745x2745, 1373x1373, 687x687
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x55daa42fd160)
GDAL: In GDALDestroy - unloading GDAL shared library.
Network statistics:
{
  "methods":{
    "HEAD":{
      "count":1
    },
    "GET":{
      "count":1,
      "downloaded_bytes":16384
    }
  },
  "handlers":{
    "vsicurl":{
      "methods":{
        "HEAD":{
          "count":1
        },
        "GET":{
          "count":1,
          "downloaded_bytes":16384
        }
      },
      "files":{
        "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
          "methods":{
            "GET":{
              "count":1,
              "downloaded_bytes":16384
            }
          },
          "actions":{
            "Read":{
              "methods":{
                "GET":{
                  "count":1,
                  "downloaded_bytes":16384
                }
              }
            }
          }
        }
      },
      "actions":{
        "Stat":{
          "methods":{
            "HEAD":{
              "count":1
            }
          },
          "files":{
            "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
              "methods":{
                "HEAD":{
                  "count":1
                }
              }
            }
          }
        }
      }
    }
  }
}
vincentsarago commented 5 months ago

For some reason doing INFO works fine, it's when reading the data that we get additional HEAD/GET

info is tested here and works https://github.com/cogeotiff/rio-tiler/blob/473de24a633303346b09be44d6e501ea60e8af33/tests/benchmarks/requests.py#L14-L36

vincentsarago commented 5 months ago

well gdalwarp gives the expected result 

CPL_VSIL_SHOW_NETWORK_STATS=YES GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR CPL_DEBUG=ON gdalwarp -q /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif out.tif -t_srs EPSG:3857 -ts 256 256 -te -10390543.876974 5518141.945963 -10370975.997733 5537709.825204 -overwrite
HTTP: libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0
VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x10fe23d90) succeeds as GTiff.
GDAL: Using GTiff driver
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
GDAL: GDALDriver::Create(GTiff,out.tif,256,256,3,Byte,0x10ffdced0)
WARP: Copying metadata from first source to destination dataset
GTiff: ScanDirectories()
GTiff: Opened 5490x5490 overview.
GTiff: Opened 2745x2745 overview.
GTiff: Opened 1373x1373 overview.
GTiff: Opened 687x687 overview.
WARP: Selecting overview level 1 for /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13100c690) creation.
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#0
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#1
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#2
GDALWARP: Defining SKIP_NOSOURCE=YES
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
VSICURL: Downloading 9019424-10144695 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Downloading 11076968-12175156 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
VSICURL: Download completed
GDAL: GDAL_CACHEMAX = 12288 MB
GDAL: GDALWarpKernel()::GWKNearestByte() Src=1823,1911,352x350 Dst=0,0,256x256
GTiff: Adjusted bytes to write from 7680 to 4608.
GDAL: GDALClose(out.tif, this=0x131005630)
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x10fe23d90)
Network statistics:
{
  "methods":{
    "HEAD":{
      "count":1
    },
    "GET":{
      "count":2,
      "downloaded_bytes":2256229
    }
  },
  "handlers":{
    "vsicurl":{
      "methods":{
        "HEAD":{
          "count":1
        },
        "GET":{
          "count":2,
          "downloaded_bytes":2256229
        }
      },
      "files":{
        "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
          "methods":{
            "GET":{
              "count":2,
              "downloaded_bytes":2256229
            }
          },
          "actions":{
            "Read":{
              "methods":{
                "GET":{
                  "count":1,
                  "downloaded_bytes":32768
                }
              }
            },
            "ReadMultiRange":{
              "methods":{
                "GET":{
                  "count":1,
                  "downloaded_bytes":2223461
                }
              }
            }
          }
        }
      },
      "actions":{
        "Stat":{
          "methods":{
            "HEAD":{
              "count":1
            }
          },
          "files":{
            "\/vsicurl\/https:\/\/sentinel-cogs.s3.us-west-2.amazonaws.com\/sentinel-s2-l2a-cogs\/15\/T\/VK\/2023\/10\/S2B_15TVK_20231008_0_L2A\/TCI.tif":{
              "methods":{
                "HEAD":{
                  "count":1
                }
              }
            }
          }
        }
      }
    }
  }
}

😬

rouault commented 5 months ago

you may want to break into GTiffDataset::LookForProjectionFromXML() . Normally papszSiblingFiles = GetSiblingFiles() should be such that papszSiblingFiles != nullptr and papszSiblingFiles[0] == nullptr if GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR

vincentsarago commented 5 months ago

just to summarize:

with gdalwarp we can 1 HEAD + 2 GET request, while with rio-tiler/rasterio we get 3 HEAD / 4 GET 😬

vincentsarago commented 5 months ago

@rouault is LookForProjectionFromXML something that changed between gdal 3.6 and 3.7?

rouault commented 5 months ago

is LookForProjectionFromXML something that changed between gdal 3.6 and 3.7?

it has been introduced in 3.7

vincentsarago commented 4 days ago

I'm going back to this and to me there seems to be a bug somewhere in GDAL

Rasterio (1.4.1, GDAL=3.9.2) + WarpedVRT

import logging

import rasterio
from rasterio.vrt import WarpedVRT

logging.basicConfig(level=logging.DEBUG)

src_path = "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif"

with rasterio.Env(GDAL_DISABLE_READDIR_ON_OPEN="EMPTY_DIR", GDAL_INGESTED_BYTES_AT_OPEN=32768, GDAL_HTTP_MERGE_CONSECUTIVE_RANGES="YES", CPL_DEBUG="ON", CPL_CURL_VERBOSE="YES"):
    with rasterio.open(src_path) as src:
        with WarpedVRT(src, dst_crs="epsg:3857") as vrt:
            vrt.read(indexes=1, out_shape=(1, vrt.height//5, vrt.width//5))

logs

DEBUG:rasterio.env:Entering env context: <rasterio.env.Env object at 0x103ead2e0>
DEBUG:rasterio.env:Starting outermost env
DEBUG:rasterio.env:No GDAL environment exists
DEBUG:rasterio.env:New GDAL environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> created
DEBUG:rasterio._filepath:Installing FilePath filesystem handler plugin...
DEBUG:rasterio._env:GDAL_DATA found in environment.
DEBUG:rasterio._env:PROJ_LIB found in environment.
DEBUG:rasterio._env:Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:Entered env context: <rasterio.env.Env object at 0x103ead2e0>
DEBUG:rasterio.env:Got a copy of environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio.env:Entering env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio.env:Got a copy of environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio.env:Entered env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio._base:Sharing flag: 0
DEBUG:rasterio._env:CPLE_None in HTTP: libcurl/8.8.0 OpenSSL/1.1.1w zlib/1.2.12 nghttp2/1.46.0
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Host sentinel-cogs.s3.us-west-2.amazonaws.com:443 was resolved.
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: IPv6: (none)
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: IPv4: 3.5.84.124, 52.92.209.90, 52.92.235.82, 3.5.80.209, 3.5.80.200, 52.218.246.41, 3.5.86.120, 52.92.212.234
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Trying 3.5.84.124:443...
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Connected to sentinel-cogs.s3.us-west-2.amazonaws.com (3.5.84.124) port 443
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: ALPN: curl offers h2,http/1.1
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (OUT), TLS handshake, Client hello (1):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  CAfile: [/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/certifi/cacert.pem](https://file+.vscode-resource.vscode-cdn.net/Users/vincentsarago/Dev/venv/py39/lib/python3.9/site-packages/certifi/cacert.pem)
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  CApath: none
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Server hello (2):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Certificate (11):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, CERT verify (15):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (IN), TLS handshake, Finished (20):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: TLSv1.3 (OUT), TLS handshake, Finished (20):
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: SSL connection using TLSv1.3 [/](https://file+.vscode-resource.vscode-cdn.net/) TLS_AES_128_GCM_SHA256 [/](https://file+.vscode-resource.vscode-cdn.net/) [blank] [/](https://file+.vscode-resource.vscode-cdn.net/) UNDEF
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: ALPN: server accepted http/1.1
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Server certificate:
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  subject: CN=*.s3-us-west-2.amazonaws.com
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  start date: Sep 14 00:00:00 2024 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  expire date: Aug 29 23:59:59 2025 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  subjectAltName: host "sentinel-cogs.s3.us-west-2.amazonaws.com" matched cert's "*.s3.us-west-2.amazonaws.com"
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:  SSL certificate verify ok.
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Certificate level 0: Public key type ? (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Certificate level 1: Public key type ? (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT:   Certificate level 2: Public key type ? (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: using HTTP/1.x
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_OUT: HEAD [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 200 OK
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: 66ZukBWtLNWUF+BSUWt3j8cFljhBQOTe77fxbWCjR7Nlc2FSHEtka2GSAdtyURpFIrzytM6T2qTXMIB7Rndz1A==
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 04M946PJ3Z8KRAHG
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:19 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 200959614
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: 
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._env:CPLE_None in VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
DEBUG:rasterio._env:CPLE_None in VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=0-32767

DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-id-2: TBEzLxd3o5rOMfIOKBl52WyRHPU85i6pqIx/nvSPCiB3SHJTt/XNUFxm8Phv2i7nRn32kK6VFlJEM8W45QOSTw==
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-request-id: 04MA0QVA6SDSNJ1N
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:19 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Range: bytes 0-32767/200959614
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: Content-Length: 32768
DEBUG:rasterio._env:CPLE_None in CURL_INFO_HEADER_IN: 
DEBUG:rasterio._env:CPLE_None in CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._env:CPLE_None in VSICURL: Got response_code=206
DEBUG:rasterio._env:CPLE_None in GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13de06f60) succeeds as GTiff.
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Dataset <open DatasetReader name='https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif' mode='r'> is started.
DEBUG:rasterio.env:Exiting env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio.env:Cleared existing <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio._env:Stopped GDALEnv <rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:No GDAL environment exists
DEBUG:rasterio.env:New GDAL environment <rasterio._env.GDALEnv object at 0x10bdbdfa0> created
DEBUG:rasterio._env:GDAL_DATA found in environment.
DEBUG:rasterio._env:PROJ_LIB found in environment.
DEBUG:rasterio._env:Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:Exited env context: <rasterio.env.Env object at 0x10be4d8e0>
DEBUG:rasterio._warp:Exported CRS to WKT.
DEBUG:rasterio._warp:Warp_extras: {'init_dest': 'NO_DATA'}
DEBUG:rasterio._env:CPLE_None in GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Nodata success: 1, Nodata value: 0.000000
DEBUG:rasterio._base:Dataset <closed WarpedVRT name='WarpedVRT(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)' mode='r'> is started.
DEBUG:rasterio._io:Output nodata value read from file: 0.0
DEBUG:rasterio._io:Output nodata values: [0.0]
DEBUG:rasterio._io:all_valid: False
DEBUG:rasterio._io:mask_flags: ([<MaskFlags.nodata: 8>], [<MaskFlags.nodata: 8>], [<MaskFlags.nodata: 8>])
DEBUG:rasterio._io:Jump straight to _read()
DEBUG:rasterio._io:Window: None
DEBUG:rasterio._io:IO window xoff=0.0 yoff=0.0 width=11055.0 height=11075.0
DEBUG:rasterio._err:GTiff: ScanDirectories()
DEBUG:rasterio._err:GTiff: Opened 5490x5490 overview.
DEBUG:rasterio._err:GTiff: Opened 2745x2745 overview.
DEBUG:rasterio._err:GTiff: Opened 1373x1373 overview.
DEBUG:rasterio._err:GTiff: Opened 687x687 overview.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4b4f30) creation.
DEBUG:rasterio._err:VSICURL: GetFileList(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A)
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 04M3W5D1X1QG1QGH
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: SWw3TihMYc69BHEGWgTDst0ipH9aYZKc31WNlx9XyBX6+EU9bUH+3JlcttL5NhleQQLJp2i5SwVJXzYqRei6NA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: application/xml
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Transfer-Encoding: chunked
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:18 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: HEAD [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39Q0YGBWGZG9N8W9
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: edsMl8Micg3K9YVsD6fZAUmphAeuYjtLNt2FzSu9+LKIyEnKMVzns8T+MXGkluiHb5jb7/f4ndx3lJJNPoSctQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: application/xml
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:18 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml): response_code=404, server error msg=HTTP/1.1 404 Not Found
x-amz-request-id: 39Q0YGBWGZG9N8W9
x-amz-id-2: edsMl8Micg3K9YVsD6fZAUmphAeuYjtLNt2FzSu9+LKIyEnKMVzns8T+MXGkluiHb5jb7/f4ndx3lJJNPoSctQ==
Content-Type: application/xml
Date: Wed, 02 Oct 2024 08:52:18 GMT
Server: AmazonS3

DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: HEAD [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 404 Not Found
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39Q5MNF1V3W72Z28
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: cPhXCpikJGdDx1TmISOlQWMRG8OfAKyE08AO6zCln+QFJ5EUIDq3VT/OR1UfKPyyITiDtfZ3lNDnmU9Uea2UcQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: application/xml
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:18 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML): response_code=404, server error msg=HTTP/1.1 404 Not Found
x-amz-request-id: 39Q5MNF1V3W72Z28
x-amz-id-2: cPhXCpikJGdDx1TmISOlQWMRG8OfAKyE08AO6zCln+QFJ5EUIDq3VT/OR1UfKPyyITiDtfZ3lNDnmU9Uea2UcQ==
Content-Type: application/xml
Date: Wed, 02 Oct 2024 08:52:18 GMT
Server: AmazonS3

DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4b6860) creation.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4bb390) creation.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x10c216510) creation.
DEBUG:rasterio._err:GDAL: Selecting overview 2764 x 2769
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e319550) creation.
DEBUG:rasterio._err:GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x11e4cedb0) creation.
DEBUG:rasterio._err:GDAL: GDAL_CACHEMAX = 12288 MB
DEBUG:rasterio._err:VSICURL: Downloading 3096576-3112959 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3096576-3112959

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: 6n/vbvz2oquK4DZ5+Mw1zpJeZD9NT7yt2cBN5woxpSn5+CfMHRRdd0kjHtT57ItyALyRLlH90RZtfD4kpBdQPQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39Q5X6179ZERSXSV
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:20 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3096576-3112959/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 16384
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,0,509x115 Dst=0,0,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=507,0,511x120 Dst=512,0,512x128
DEBUG:rasterio._err:VSICURL: Downloading 3112960-3309567 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3112960-3309567

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: 6GHyhzRGc8oWWQGG7SMKOEpWhq0qGTCwGppQCwweQh/4NSqBnBvymMhpnoyARnG8WemCvkj0RW4ygGRpORhWwg==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 39QB0EX4HEZM4MND
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:20 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3112960-3309567/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 196608
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1015,0,511x124 Dst=1024,0,512x128
DEBUG:rasterio._err:VSICURL: Downloading 3309568-3948543 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3309568-3948543

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: yqgPY6dA7F2DbbL4C+JqpsA9M8GccY9rC1vkN0TiFcPqtjtnxMxM7YQkdRfD5nWV9gISKhcTO8omelWhSNRyIw==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 87PS0WKV3DTGYJCY
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:21 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3309568-3948543/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 638976
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1524,0,510x127 Dst=1536,0,512x128
DEBUG:rasterio._err:VSICURL: Downloading 3948544-5226495 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=3948544-5226495

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: hgo/SksLd2yLV14hkryD2kRDi6AhnyDyamg60BHBnUaP9E4r0MDn4cDGsIwhUuYK+eADCkHT/GJgnTsK6nE1CA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 87PG3YQ9RDSFSVVP
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:21 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 3948544-5226495/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 1277952
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2033,0,510x127 Dst=2048,0,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,0,203x127 Dst=2560,0,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,107,508x135 Dst=0,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=505,114,511x133 Dst=512,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1014,119,511x132 Dst=1024,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1523,123,511x130 Dst=1536,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2033,126,510x128 Dst=2048,128,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,126,203x128 Dst=2560,128,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,233,506x135 Dst=0,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=503,241,512x133 Dst=512,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1013,246,511x132 Dst=1024,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1523,250,511x130 Dst=1536,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2032,252,511x129 Dst=2048,256,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,253,203x128 Dst=2560,256,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,360,504x136 Dst=0,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=502,367,512x134 Dst=512,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1012,373,512x132 Dst=1024,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1522,377,511x130 Dst=1536,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2032,379,511x129 Dst=2048,384,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,380,203x128 Dst=2560,384,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,487,503x136 Dst=0,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=500,495,513x133 Dst=512,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1011,500,512x132 Dst=1024,512,512x128
DEBUG:rasterio._err:VSICURL: Downloading 5226496-7782399 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=5226496-7782399

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: Knzkc7DKDTFgOy0+3db6tnzTCXfKw+V1NO33XW3qdsVN/DbpdXZXrgIjdWWQVKp7cA9LCLtNLwPTnaW4CkVfRQ==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: 3RTD6FNTYN4SPTAB
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:22 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 5226496-7782399/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1521,504,512x130 Dst=1536,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2032,506,511x129 Dst=2048,512,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,507,203x128 Dst=2560,512,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,614,501x136 Dst=0,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=499,622,513x133 Dst=512,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1010,627,512x132 Dst=1024,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1520,631,513x131 Dst=1536,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2031,633,512x129 Dst=2048,640,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,634,203x128 Dst=2560,640,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,742,500x135 Dst=0,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=497,749,514x134 Dst=512,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1008,754,513x133 Dst=1024,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1520,758,512x131 Dst=1536,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2031,761,512x129 Dst=2048,768,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,761,203x129 Dst=2560,768,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,869,498x136 Dst=0,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=496,876,513x134 Dst=512,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1007,882,514x132 Dst=1024,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1519,886,513x130 Dst=1536,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2030,888,513x129 Dst=2048,896,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,889,203x128 Dst=2560,896,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,996,497x136 Dst=0,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=494,1004,514x134 Dst=512,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1006,1009,514x133 Dst=1024,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1518,1013,513x131 Dst=1536,1024,512x128
DEBUG:rasterio._err:VSICURL: Downloading 7782400-10338303 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=7782400-10338303

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: XAY9dwZ6cgnWABIB8CPEhAIn+mCl6tr5wODh+ydISgMNGwdfH1cp6b/SNsPaEmehIRDRhaRSeNcsIoTCnXvfKA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: BV7W5P11Z1QSEW27
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:24 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 7782400-10338303/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2030,1015,513x130 Dst=2048,1024,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1016,203x129 Dst=2560,1024,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1124,495x136 Dst=0,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=492,1131,515x134 Dst=512,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1005,1137,514x132 Dst=1024,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1517,1141,514x131 Dst=1536,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2030,1143,513x129 Dst=2048,1152,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1143,203x129 Dst=2560,1152,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1252,493x136 Dst=0,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=491,1259,515x134 Dst=512,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1004,1264,514x133 Dst=1024,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1516,1268,515x131 Dst=1536,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2029,1271,514x129 Dst=2048,1280,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1271,203x129 Dst=2560,1280,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1379,492x136 Dst=0,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=489,1387,516x134 Dst=512,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1002,1392,515x133 Dst=1024,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1516,1396,514x131 Dst=1536,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2029,1398,514x130 Dst=2048,1408,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1399,203x129 Dst=2560,1408,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1507,490x136 Dst=0,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=488,1514,515x135 Dst=512,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1001,1520,516x133 Dst=1024,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1515,1524,515x131 Dst=1536,1536,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2029,1526,514x130 Dst=2048,1536,512x128
DEBUG:rasterio._err:VSICURL: Downloading 10338304-12894207 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=10338304-12894207

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: e4pv6zaja1zW0J4XYxBisnAKWyTauRyNx712+0fFACOu3pOc4yanIfVTh8OvvIxh6s7sb23EAdQyQjvLMx4QeA==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: JNYCDH9Z4BH47JEH
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:25 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 10338304-12894207/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1527,203x129 Dst=2560,1536,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1635,489x136 Dst=0,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=486,1642,516x135 Dst=512,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1000,1648,516x133 Dst=1024,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1514,1652,516x131 Dst=1536,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2028,1654,515x130 Dst=2048,1664,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1655,203x129 Dst=2560,1664,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1763,487x136 Dst=0,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=484,1770,517x135 Dst=512,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=999,1776,516x133 Dst=1024,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1513,1780,516x131 Dst=1536,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2028,1782,515x130 Dst=2048,1792,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1783,203x129 Dst=2560,1792,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,1891,485x137 Dst=0,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=483,1898,517x135 Dst=512,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=998,1904,516x133 Dst=1024,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1513,1908,516x132 Dst=1536,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2028,1910,515x130 Dst=2048,1920,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,1911,203x129 Dst=2560,1920,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2020,484x136 Dst=0,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=481,2027,518x135 Dst=512,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=997,2032,517x134 Dst=1024,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1512,2036,517x132 Dst=1536,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2027,2039,516x130 Dst=2048,2048,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,2039,203x130 Dst=2560,2048,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2148,482x136 Dst=0,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=480,2155,518x135 Dst=512,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=995,2161,518x133 Dst=1024,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1511,2165,517x131 Dst=1536,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2027,2167,516x130 Dst=2048,2176,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,2167,203x130 Dst=2560,2176,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2276,481x137 Dst=0,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=478,2283,518x136 Dst=512,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=994,2289,518x134 Dst=1024,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1510,2293,518x132 Dst=1536,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2026,2295,518x131 Dst=2048,2304,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2542,2296,203x130 Dst=2560,2304,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2405,479x137 Dst=0,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=476,2412,519x135 Dst=512,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=993,2418,518x133 Dst=1024,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1509,2422,518x131 Dst=1536,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2026,2424,518x130 Dst=2048,2432,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2543,2424,202x130 Dst=2560,2432,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2533,477x137 Dst=0,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=475,2541,519x135 Dst=512,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=992,2546,518x134 Dst=1024,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1509,2550,518x132 Dst=1536,2560,512x128
DEBUG:rasterio._err:VSICURL: Downloading 12894208-15450111 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
DEBUG:rasterio._err:CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
DEBUG:rasterio._err:CURL_INFO_TEXT: Found bundle for host: 0x11e2cfb30 [serially]
DEBUG:rasterio._err:CURL_INFO_TEXT: Can not multiplex, even if we wanted to
DEBUG:rasterio._err:CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
DEBUG:rasterio._err:CURL_INFO_HEADER_OUT: GET [/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif](https://file+.vscode-resource.vscode-cdn.net/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif) HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.9.2
Accept: */*
Range: bytes=12894208-15450111

DEBUG:rasterio._err:CURL_INFO_TEXT: Request completely sent off
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-id-2: oakZsSEL5Nty2AuT0Tghf6F4EPMC3cOkQVIEHBJF7CTvZ8coj8cYBjaDpZspjbuklLkkMGuRb5Z0/Zsmk2Zn4w==
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-request-id: X56SHGFYSG8N7ND9
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:52:26 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Accept-Ranges: bytes
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Range: bytes 12894208-15450111/200959614
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Server: AmazonS3
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: Content-Length: 2555904
DEBUG:rasterio._err:CURL_INFO_HEADER_IN: 
DEBUG:rasterio._err:CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
DEBUG:rasterio._err:VSICURL: Got response_code=206
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2026,2552,518x131 Dst=2048,2560,512x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2543,2553,202x130 Dst=2560,2560,204x128
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,2662,476x83 Dst=0,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=474,2669,519x76 Dst=512,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=991,2675,519x70 Dst=1024,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=1508,2679,519x66 Dst=1536,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2025,2681,519x64 Dst=2048,2688,512x81
DEBUG:rasterio._err:GDAL: GDALWarpKernel()::GWKNearestByte() Src=2543,2682,202x63 Dst=2560,2688,204x81
DEBUG:rasterio._env:CPLE_None in GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x13de06f60)
DEBUG:rasterio.env:Exiting env context: <rasterio.env.Env object at 0x103ead2e0>
DEBUG:rasterio.env:Cleared existing <rasterio._env.GDALEnv object at 0x10bdbdfa0> options
DEBUG:rasterio._env:Stopped GDALEnv <rasterio._env.GDALEnv object at 0x10bdbdfa0>.
DEBUG:rasterio.env:Exiting outermost env
DEBUG:rasterio.env:Exited env context: <rasterio.env.Env object at 0x103ead2e0>

@rouault I don't see any mentione of GDAL_DISABLE_READDIR_ON_OPEN in https://github.com/OSGeo/gdal/blob/96608941276cc29a4c17b55d453ab8bd616e34dd/frmts/gtiff/gtiffdataset_read.cpp#L5828-L5854

what I don't understand is why gdalwarp doesn't result in the same behaviour

GDAL 3.8.5 (gdalwarp)

GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR GDAL_INGESTED_BYTES_AT_OPEN=32768 GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES CPL_DEBUG=ON CPL_CURL_VERBOSE=YES gdalwarp -q /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif out.tif -b 1 -t_srs EPSG:3857 -ts 2215 2211 -overwrite
HTTP: libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0
CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
CURL_INFO_TEXT:   Trying 52.92.129.34:443...
CURL_INFO_TEXT: Connected to sentinel-cogs.s3.us-west-2.amazonaws.com (52.92.129.34) port 443
CURL_INFO_TEXT: ALPN: curl offers h2,http/1.1
CURL_INFO_TEXT: (304) (OUT), TLS handshake, Client hello (1):
CURL_INFO_TEXT:  CAfile: /etc/ssl/cert.pem
CURL_INFO_TEXT:  CApath: none
CURL_INFO_TEXT: (304) (IN), TLS handshake, Server hello (2):
CURL_INFO_TEXT: (304) (IN), TLS handshake, Unknown (8):
CURL_INFO_TEXT: (304) (IN), TLS handshake, Certificate (11):
CURL_INFO_TEXT: (304) (IN), TLS handshake, CERT verify (15):
CURL_INFO_TEXT: (304) (IN), TLS handshake, Finished (20):
CURL_INFO_TEXT: (304) (OUT), TLS handshake, Finished (20):
CURL_INFO_TEXT: SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256
CURL_INFO_TEXT: ALPN: server accepted http/1.1
CURL_INFO_TEXT: Server certificate:
CURL_INFO_TEXT:  subject: CN=*.s3-us-west-2.amazonaws.com
CURL_INFO_TEXT:  start date: Sep 14 00:00:00 2024 GMT
CURL_INFO_TEXT:  expire date: Aug 29 23:59:59 2025 GMT
CURL_INFO_TEXT:  subjectAltName: host "sentinel-cogs.s3.us-west-2.amazonaws.com" matched cert's "*.s3.us-west-2.amazonaws.com"
CURL_INFO_TEXT:  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
CURL_INFO_TEXT:  SSL certificate verify ok.
CURL_INFO_TEXT: using HTTP/1.1
CURL_INFO_HEADER_OUT: HEAD /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.8.5
Accept: */*

CURL_INFO_HEADER_IN: HTTP/1.1 200 OK
CURL_INFO_HEADER_IN: x-amz-id-2: IHhHcFOmwMl5CwUr/ArLtiOnzqWsBK+yFiUaR1rUNgfsI4XF1jyoAgUrqZNSLwRBpyQZy+8NNh8=
CURL_INFO_HEADER_IN: x-amz-request-id: W45JSEN9T8VZGB6P
CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:51:25 GMT
CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
CURL_INFO_HEADER_IN: Accept-Ranges: bytes
CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
CURL_INFO_HEADER_IN: Server: AmazonS3
CURL_INFO_HEADER_IN: Content-Length: 200959614
CURL_INFO_HEADER_IN: 
CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614  response_code=200
VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
CURL_INFO_TEXT: Found bundle for host: 0x111f3f360 [serially]
CURL_INFO_TEXT: Can not multiplex, even if we wanted to
CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.8.5
Accept: */*
Range: bytes=0-32767

CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
CURL_INFO_HEADER_IN: x-amz-id-2: aWHsd1h8TIJ1yXExGjnPQvf80Ai2gHjgfEzSwpY0740oDPSLUP8YD0qqCj43x3Q8ohS4fCZLJwA=
CURL_INFO_HEADER_IN: x-amz-request-id: W45M68E8CENK6F25
CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:51:25 GMT
CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
CURL_INFO_HEADER_IN: Accept-Ranges: bytes
CURL_INFO_HEADER_IN: Content-Range: bytes 0-32767/200959614
CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
CURL_INFO_HEADER_IN: Server: AmazonS3
CURL_INFO_HEADER_IN: Content-Length: 32768
CURL_INFO_HEADER_IN: 
CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x112804fe0) succeeds as GTiff.
GDAL: GDALOpen(out.tif, this=0x121f62210) succeeds as GTiff.
GDAL: GDALClose(out.tif, this=0x121f62210)
GDAL: Using GTiff driver
GDAL: GDALOpen(out.tif, this=0x121f61ae0) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(out.tif, this=0x121f61ae0)
GDAL: GDALOpen(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x111fcea10) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x111fcea10)
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
GDAL: QuietDelete(out.tif) invoking Delete()
GDAL: GDALOpen(out.tif, this=0x121f67700) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(out.tif, this=0x121f67700)
GDAL: GDALDriver::Create(GTiff,out.tif,2215,2211,1,Byte,0x0)
WARP: Copying metadata from first source to destination dataset
GTiff: ScanDirectories()
GTiff: Opened 5490x5490 overview.
GTiff: Opened 2745x2745 overview.
GTiff: Opened 1373x1373 overview.
GTiff: Opened 687x687 overview.
WARP: Selecting overview level 1 for /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif
GDAL: GDALOverviewDataset(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x12282baa0) creation.
WARP: srcNoData=0.000000 dstNoData=0.000000
WARP: calling GDALSetRasterNoDataValue() for band#0
GDALWARP: Defining SKIP_NOSOURCE=YES
GDAL: Computing area of interest: -94.2725, 44.1585, -92.8758, 45.1538
VSICURL: Downloading 3096576-13139967 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)...
CURL_INFO_TEXT: Couldn't find host sentinel-cogs.s3.us-west-2.amazonaws.com in the .netrc file; using defaults
CURL_INFO_TEXT: Found bundle for host: 0x111f3f360 [serially]
CURL_INFO_TEXT: Can not multiplex, even if we wanted to
CURL_INFO_TEXT: Re-using existing connection with host sentinel-cogs.s3.us-west-2.amazonaws.com
CURL_INFO_HEADER_OUT: GET /sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif HTTP/1.1
Host: sentinel-cogs.s3.us-west-2.amazonaws.com
User-Agent: GDAL/3.8.5
Accept: */*
Range: bytes=3096576-13139967

CURL_INFO_HEADER_IN: HTTP/1.1 206 Partial Content
CURL_INFO_HEADER_IN: x-amz-id-2: daI+K5fgNIqmSsx/+eJOL9nP0WR221sRPb9cBKyZq4OMnjSbMG/KKuPGLRAXhkWf6rUVf7yZLP4=
CURL_INFO_HEADER_IN: x-amz-request-id: DCC9TEV1T906696B
CURL_INFO_HEADER_IN: Date: Wed, 02 Oct 2024 08:51:26 GMT
CURL_INFO_HEADER_IN: Last-Modified: Mon, 09 Oct 2023 01:23:09 GMT
CURL_INFO_HEADER_IN: ETag: "3e23880cd7ac8a32ae71c55ad4f85c5d-24"
CURL_INFO_HEADER_IN: x-amz-storage-class: INTELLIGENT_TIERING
CURL_INFO_HEADER_IN: x-amz-server-side-encryption: AES256
CURL_INFO_HEADER_IN: Cache-Control: public, max-age=31536000, immutable
CURL_INFO_HEADER_IN: Accept-Ranges: bytes
CURL_INFO_HEADER_IN: Content-Range: bytes 3096576-13139967/200959614
CURL_INFO_HEADER_IN: Content-Type: image/tiff; application=geotiff; profile=cloud-optimized
CURL_INFO_HEADER_IN: Server: AmazonS3
CURL_INFO_HEADER_IN: Content-Length: 10043392
CURL_INFO_HEADER_IN: 
CURL_INFO_TEXT: Connection #0 to host sentinel-cogs.s3.us-west-2.amazonaws.com left intact
VSICURL: Got response_code=206
GDAL: GDAL_CACHEMAX = 12288 MB
GDAL: GDALWarpKernel()::GWKNearestByte() Src=0,0,2745x2745 Dst=0,0,2215x2211
GDAL: GDALClose(out.tif, this=0x122825390)
GDAL: GDALClose(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif, this=0x112804fe0)

We can also see that rasterio doesn't merge the Range request

I've tested with both gdal 3.8 and 3.9