Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.82k stars 323 forks source link

Azure Shared Access Signature - Signature did not match using ingress nginx #2448

Closed rlugovyy closed 1 month ago

rlugovyy commented 2 months ago

I have problems connecting to azure.blob.storage using ingress as a proxy to have 1 entry point for my app. When I try to connect directly, everything works, while through ingress I get an error: Connection to Azure Blob Storage failed! Reason is: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

this.aborterController = new AbortController(); 
this.sharedKeyCredential = new AzureStorage.StorageSharedKeyCredential(  
  getFileStorageBlobDbConfigs().storageName,     
  getFileStorageBlobDbConfigs().key1,     
);
this.blobServiceClient = new AzureStorage.BlobServiceClient(        
  `https://${getFileStorageBlobDbConfigs().storageName}.blob.core.windows.net\`,
  AzureStorage.newPipeline(this.sharedKeyCredential),
);

this.sharedKeyCredential = new AzureStorage.StorageSharedKeyCredential(
  getFileStorageBlobDbConfigs().storageName,
  getFileStorageBlobDbConfigs().key1,
);
this.blobServiceClient = new AzureStorage.BlobServiceClient(
  'https://my.ingress.com/blobstorage/',
  AzureStorage.newPipeline(this.sharedKeyCredential),
);
// check 
try {
  await new AzureBlobStorage().checkConnection();
} catch (error) {
  Logger.error(`Connection to Azure Blob Storage failed! Reason is: ${(error as { message: string }).message}`);
}

and here is my ingress (also in this cluster works istio if it is important)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-production
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/rewrite-target: $1
    nginx.ingress.kubernetes.io/upstream-vhost: teststorage8297.blob.core.windows.net
    nginx.ingress.kubernetes.io/use-regex: "true"
  name: proxy-to-blob-storage-ingress
  namespace: default
  spec:
    ingressClassName: nginx
    rules:
    - host: my.ingress.com
      http:
        paths:
        - backend:
           service:
             name: proxy-to-blob-storage
             port:
               number: 443
             path: /blobstorage(.\*)
             pathType: Prefix
    tls:
    - hosts:
      - my.ingress.com
      secretName: proxy-to-blob-storage-tls\`
// service
apiVersion: v1
kind: Service
metadata:
  name: proxy-to-blob-storage
  namespace: default
spec:
  type: ExternalName
  externalName: teststorage8297.blob.core.windows.net
  ports:
    - port: 443

I have only been able to connect and send a file via curl so far

curl -X PUT -d "file content" -H "x-ms-blob-type: BlockBlob" "https://my.ingress.com/blobstorage/test/test.txt?SAS-Token"
EmmaZhu commented 2 months ago

Hi @rlugovyy ,

Could you share more details on the issue:

  1. How await new AzureBlobStorage().checkConnection(); works, could you share some code snippet for it?
  2. Debug.log from Azurite
  3. Version of Azurite, how you started it (with a docker, or npm package, or exe?)

Thanks Emma

rlugovyy commented 2 months ago

Hi @EmmaZhu

  1. I use the standard function from the storage-blob.d.ts library ` /**

    • The Get Account Information operation returns the sku name and account kind
    • for the specified account.
    • The Get Account Information operation is available on service versions beginning
    • with version 2018-03-28.
    • @see https://docs.microsoft.com/en-us/rest/api/storageservices/get-account-information
    • @param options - Options to the Service Get Account Info operation.
    • @returns Response data for the Service Get Account Info operation. */ getAccountInfo(options?: ServiceGetAccountInfoOptions): Promise; `
  2. Connection to Azure Blob Storage failed! Reason is: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

  3. I have blob storage running in Azure (as a service) where I get a ConnectionString DefaultEndpointsProtocol=https;AccountName=teststorage8297;AccountKey=<KEY>;BlobEndpoint=https://<AccountName>.blob.core.windows.net and then I want to replace BlobEndpoint with DefaultEndpointsProtocol=https;AccountName=teststorage8297;AccountKey=<KEY>;BlobEndpoint=https://my-ingress/blob/

I also wrote yesterday as a test, if i do not use the library from Azure but write my own CRUD, then everything works, but through the library I get an error in authentication. My assumption is that somehow inside the function AzureBlob during the generation of the SAS token it depends on the URL. although when I wrote my own function to generate a SAS token I didn't use it

EmmaZhu commented 2 months ago

Hi @rlugovyy ,

Azurite can be started with a option like: --debug c:\azurite\debug.log. Could you try to repro the issue again and share debug.log to us for investigation?

EmmaZhu commented 1 month ago

Please feel free to reopen the issue if any new updates.