Closed mmaitre314 closed 2 weeks ago
Thank you for your feedback. Tagging and routing to the team member best able to assist.
@mmaitre314 for context and better understanding, which services and C++ client SDK are you using in your application, that motivated this? Is it storage blobs and the BlobClient or some other service/SDK?
DuckDB uses Azure::Storage::Blobs::BlobServiceClient
. Here is the code: https://github.com/duckdb/duckdb_azure/blob/097bb13aadb186ca43ae9b5dc6a21c20e56ad4dd/src/azure_storage_account_client.cpp#L329
Investigating an issue with HTTP range requests, it would also be helpful to include x-ms-range
,range
, accept-ranges
, content-range
:
[2024-07-15T11:19:48.3297721Z T: 7ff86e6e9700] INFO : HTTP Request : GET https://<snip>.blob.core.windows.net/<snip>.gz.parquet?se=REDACTED&sig=REDACTED&ske=REDACTED&skoid=REDACTED&sks=REDACTED&skt=REDACTED&sktid=REDACTED&skv=REDACTED&sp=REDACTED&sr=REDACTED&st=REDACTED&sv=REDACTED
if-match : "0x8DC7356EF4FDCDA"
user-agent : azsdk-cpp-storage-blobs/12.10.0 (Linux 5.15.0-1064-azure x86_64 #73~20.04.1-Ubuntu SMP Mon May 6 09:43:44 UTC 2024)
x-ms-client-request-id : 8afbe7a1-48fd-4857-8c86-47047fd756b9
x-ms-date : REDACTED
x-ms-range : REDACTED
x-ms-version : REDACTED
[2024-07-15T11:19:48.3297856Z T: 7ff86e6e9700] DEBUG : [CURL Transport Adapter]: Creating a new session.
[2024-07-15T11:19:48.3297975Z T: 7ff86e6e9700] DEBUG : [CURL Transport Adapter]: Re-using connection from the pool.
[2024-07-15T11:19:48.3298108Z T: 7ff86e6e9700] DEBUG : [CURL Transport Adapter]: No Host in request headers. Adding it
[2024-07-15T11:19:48.3298239Z T: 7ff86e6e9700] DEBUG : [CURL Transport Adapter]: Send request without payload
[2024-07-15T11:19:48.3298616Z T: 7ff86e6e9700] DEBUG : [CURL Transport Adapter]: Parse server response
[2024-07-15T11:19:48.3484566Z T: 7ff86e6e9700] DEBUG : [CURL Transport Adapter]: Request completed. Moving response out of session and session to response.
[2024-07-15T11:19:48.3484913Z T: 7ff86e6e9700] INFO : HTTP Response (18ms) : 206 Partial Content
accept-ranges : REDACTED
content-length : 1048576
content-range : REDACTED
content-type : application/octet-stream
date : Mon, 15 Jul 2024 11:19:47 GMT
etag : "0x8DC7356EF4FDCDA"
last-modified : Mon, 13 May 2024 14:13:59 GMT
server : Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-blob-type : REDACTED
x-ms-client-request-id : 8afbe7a1-48fd-4857-8c86-47047fd756b9
x-ms-copy-completion-time : REDACTED
x-ms-copy-id : REDACTED
x-ms-copy-progress : REDACTED
x-ms-copy-source : REDACTED
x-ms-copy-status : REDACTED
x-ms-creation-time : REDACTED
x-ms-group : REDACTED
x-ms-lease-state : REDACTED
x-ms-lease-status : REDACTED
x-ms-meta-hdi_permission : REDACTED
x-ms-owner : REDACTED
x-ms-permissions : REDACTED
x-ms-request-id : 74787e37-f01e-0024-20a8-d61070000000
x-ms-resource-type : REDACTED
x-ms-server-encrypted : REDACTED
x-ms-version : REDACTED
[2024-07-15T11:19:48.3485269Z T: 7ff86e6e9700] INFO : HTTP status code 206 won't be retried.
Is your feature request related to a problem? Please describe.
I am debugging REST requests to a Storage Account hitting auth failures in DuckDB and the logs redacted headers like
WWW-Authenticate
,x-ms-version
,x-ms-date
which could have helped root-causing the issue.Including
x-ms-error-code
might also be helpful but I could not find documentation on this header to understand what it can contain.Describe the solution you'd like
Include
WWW-Authenticate
,x-ms-version
,x-ms-date
ing_defaultAllowedHttpHeaders
If this sounds OK I can likely contribute a PR.
Describe alternatives you've considered
It looks like DuckDB could customize redacted headers but doing that in the Azure SDK would be a more generic solution.
Additional context
Related issue: #3708
WWW-Authenticate
is included in .NET and Java SDKs here:x-ms-version
andx-ms-date
are not included there but theDate
header is and theapi-version
query parameter also is, so that seems like a reasonable inclusion.Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report