search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.82k stars 322 forks source link

Download from local Azurite Docker container with SAS token returns 403 AuthorizationFailure #2478

Open brbarnett opened 6 days ago

brbarnett commented 6 days ago

Summary

I am able to upload a file to an Azurite container using the default connection string DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=[http://127.0.0.1:10000/devstoreaccount1;](http://127.0.0.1:10000/devstoreaccount1;%60), but generating a SAS token does not allow me to download that file.

Details

Container image: mcr.microsoft.com/azure-storage/azurite:3.32.0

Python package version: azure-storage-blob 12.23.1

Docker compose service:

  azurite:
    container_name: azurite
    image: mcr.microsoft.com/azure-storage/azurite:3.32.0
    ports:
      - "10000:10000"
    restart: unless-stopped

Generating the SAS token:

start_time = datetime.now(timezone.utc)
expiry_time = start_time + timedelta(hours=48)
sas_token = generate_blob_sas(
    account_key="Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==",
    account_name="devstoreaccount1",
    blob_name="file.txt",
    container_name="uploads",
    expiry=expiry_time,
    permission=BlobSasPermissions(read=True, write=False, delete=False),
    start=start_time,
)

return f"{url}?{sas_token}"

Generated URL: http://127.0.0.1:10000/devstoreaccount1/uploads/file.txt?st=2024-10-16T20%3A07%3A16Z&se=2024-10-18T20%3A07%3A16Z&sp=r&sv=2024-11-04&sr=b&sig=jS0Luf8yHVWYVH4X5x6Wwp8KjGc%2BsNkYHl6oVNA7Wv4%3D

Formatted:

st: 2024-10-16T20:07:16Z
se: 2024-10-18T20:07:16Z
sp: r
sv: 2024-11-04
sr: b
sig: jS0Luf8yHVWYVH4X5x6Wwp8KjGc+sNkYHl6oVNA7Wv4=

Error:

<Error>
<Code>AuthorizationFailure</Code>
<Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:acb5401c-3dce-48bc-abbe-7f2854c15a3f Time:2024-10-16T20:17:56.086Z</Message>
</Error>

Happy to provide whatever other information would be useful.

EmmaZhu commented 6 days ago

Hi @brbarnett ,

Seems you implemented a generate_blob_sas function by yourself? Could you share the string to sign signature?

Could you also share the debug.log output from Azurite?

brbarnett commented 6 days ago

Hi @brbarnett ,

Seems you implemented a generate_blob_sas function by yourself? Could you share the string to sign signature?

Could you also share the debug.log output from Azurite?

Thanks for your response, @EmmaZhu

I am using the Python SDK: https://pypi.org/project/azure-storage-blob/ (v12.23.1)

from azure.storage.blob import BlobSasPermissions, generate_blob_sas

Can you help me find debug.log? I assume it's somewhere in the container file system.

brbarnett commented 6 days ago

Nevermind, I found it:

2024-10-17T10:14:23.681Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1/uploads/a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx?st=2024-10-17T10%3A12%3A27Z&se=2024-10-19T10%3A12%3A27Z&sp=r&sv=2024-11-04&sr=b&sig=u4lWXqrRiTVwtM8QJkA0GvJ3G0FtPWK3lS71LzsjDFc%3D RequestHeaders:{"host":"127.0.0.1:10000","connection":"keep-alive","cache-control":"max-age=0","sec-ch-ua":"\"Google Chrome\";v=\"129\", \"Not=A?Brand\";v=\"8\", \"Chromium\";v=\"129\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","sec-fetch-site":"cross-site","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","referer":"http://localhost:5173/","accept-encoding":"gzip, deflate, br, zstd","accept-language":"en-US,en;q=0.9"} ClientIP=172.19.0.1 Protocol=http HTTPVersion=1.1
2024-10-17T10:14:23.682Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=uploads Blob=a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx
2024-10-17T10:14:23.682Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd verbose: DispatchMiddleware: Dispatching request...
2024-10-17T10:14:23.682Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: DispatchMiddleware: Operation=Blob_Download
2024-10-17T10:14:23.682Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-10-17T10:14:23.682Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-10-17T10:14:23.682Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-10-17T10:14:23.682Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: uploads, blob: a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container uploads
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: BlobSharedKeyAuthenticator:validate() Request doesn't include valid authentication header. Skip shared key authentication.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: AccountSASAuthenticator:validate() Start validation against account Shared Access Signature pattern.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: AccountSASAuthenticator:validate() Getting account properties...
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: AccountSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: uploads, blob: a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: AccountSASAuthenticator:validate() Got account properties successfully.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: AccountSASAuthenticator:validate() Retrieved signature from URL parameter sig: u4lWXqrRiTVwtM8QJkA0GvJ3G0FtPWK3lS71LzsjDFc=
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: AccountSASAuthenticator:validate() Failed to get valid account SAS values from request.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: BlobSASAuthenticator:validate() Start validation against blob service Shared Access Signature pattern.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() Getting account properties...
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: uploads, blob: a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() Got account properties successfully.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() Retrieved signature from URL parameter sig: u4lWXqrRiTVwtM8QJkA0GvJ3G0FtPWK3lS71LzsjDFc=
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() Signed resource type is b.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() Successfully got valid blob service SAS values from request. {"version":"2024-11-04","startTime":"2024-10-17T10:12:27Z","expiryTime":"2024-10-19T10:12:27Z","permissions":"r","containerName":"uploads","blobName":"a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx","signedResource":"b"}
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: BlobSASAuthenticator:validate() Validate signature based account key1.
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() String to sign is: "r\n2024-10-17T10:12:27Z\n2024-10-19T10:12:27Z\n/blob/devstoreaccount1/uploads/a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx\n\n\n\n2024-11-04\nb\n\n\n\n\n\n\n"
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd debug: BlobSASAuthenticator:validate() Calculated signature is: faT+AMpY9oRyhK0Mou3m+dGEi8mcy+kViWLi/Gebq9g=
2024-10-17T10:14:23.683Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: BlobSASAuthenticator:validate() Signature based on key1 validation failed.
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.  ErrorHTTPStatusCode=403 ErrorHTTPStatusMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. ErrorHTTPHeaders={"x-ms-error-code":"AuthorizationFailure","x-ms-request-id":"d2ed78c7-35ad-4852-a25d-fb6308cc72dd"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>AuthorizationFailure</Code>\n  <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:d2ed78c7-35ad-4852-a25d-fb6308cc72dd\nTime:2024-10-17T10:14:23.683Z</Message>\n</Error>" ErrorStack="StorageError: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\n    at StorageErrorFactory.getAuthorizationFailure (/opt/azurite/dist/src/blob/errors/StorageErrorFactory.js:140:16)\n    at /opt/azurite/dist/src/blob/middlewares/AuthenticationMiddlewareFactory.js:25:56\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: Set HTTP code: 403
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: Set HTTP status message: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=AuthorizationFailure
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=d2ed78c7-35ad-4852-a25d-fb6308cc72dd
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: Set content type: application/xml
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>AuthorizationFailure</Code>\n  <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:d2ed78c7-35ad-4852-a25d-fb6308cc72dd\nTime:2024-10-17T10:14:23.683Z</Message>\n</Error>"
2024-10-17T10:14:23.684Z d2ed78c7-35ad-4852-a25d-fb6308cc72dd info: EndMiddleware: End response. TotalTimeInMS=3 StatusCode=403 StatusMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. Headers={"server":"Azurite-Blob/3.32.0","x-ms-error-code":"AuthorizationFailure","x-ms-request-id":"d2ed78c7-35ad-4852-a25d-fb6308cc72dd","content-type":"application/xml"}
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/favicon.ico RequestHeaders:{"host":"127.0.0.1:10000","connection":"keep-alive","sec-ch-ua-platform":"\"Windows\"","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","sec-ch-ua":"\"Google Chrome\";v=\"129\", \"Not=A?Brand\";v=\"8\", \"Chromium\";v=\"129\"","sec-ch-ua-mobile":"?0","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"http://127.0.0.1:10000/devstoreaccount1/uploads/a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx?st=2024-10-17T10%3A12%3A27Z&se=2024-10-19T10%3A12%3A27Z&sp=r&sv=2024-11-04&sr=b&sig=u4lWXqrRiTVwtM8QJkA0GvJ3G0FtPWK3lS71LzsjDFc%3D","accept-encoding":"gzip, deflate, br, zstd","accept-language":"en-US,en;q=0.9"} ClientIP=172.19.0.1 Protocol=http HTTPVersion=1.1
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c info: BlobStorageContextMiddleware: Account=favicon.ico Container=undefined Blob=
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c verbose: DispatchMiddleware: Dispatching request...
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c error: DispatchMiddleware: Incoming URL doesn't match any of swagger defined request patterns.
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c error: ErrorMiddleware: ErrorName=UnsupportedRequestError ErrorMessage=Incoming URL doesn't match any of swagger defined request patterns.  ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=undefined ErrorHTTPHeaders=undefined ErrorHTTPBody=undefined ErrorStack="UnsupportedRequestError: Incoming URL doesn't match any of swagger defined request patterns.\n    at dispatchMiddleware (/opt/azurite/dist/src/blob/generated/middleware/dispatch.middleware.js:41:30)\n    at /opt/azurite/dist/src/blob/generated/ExpressMiddlewareFactory.js:50:47\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:328:13)\n    at /opt/azurite/node_modules/express/lib/router/index.js:286:9\n    at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:346:12)\n    at next (/opt/azurite/node_modules/express/lib/router/index.js:280:10)\n    at blobStorageContextMiddleware (/opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:137:5)\n    at /opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:15:16\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)"
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c error: ErrorMiddleware: Set HTTP code: 400
2024-10-17T10:14:23.727Z 49854111-939a-4ded-8d5b-9721fe63893c error: ErrorMiddleware: Set HTTP body: undefined
2024-10-17T10:14:23.728Z 49854111-939a-4ded-8d5b-9721fe63893c info: EndMiddleware: End response. TotalTimeInMS=0 StatusCode=400 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.32.0"}

I see that Signature based on key1 validation failed. in there but I'm not sure how to resolve that based on the string to sign. Note: I simplified things a bit in my first prompt here but the URL here is accurate: http://127.0.0.1:10000/devstoreaccount1/uploads/a0e59b28-a325-4ed4-9449-234ef6cc9fc1/DEMO_DOC_2_2.docx