Azure / azure-sdk-for-go

This repository is for active development of the Azure SDK for Go. For consumers of the SDK we recommend visiting our public developer docs at:
https://docs.microsoft.com/azure/developer/go/
MIT License
1.62k stars 822 forks source link

AzBlob SAS URL doesn't work if Azurite is run on a custom hostname. #23496

Open vikblom opened 1 week ago

vikblom commented 1 week ago

Bug Report

Hello 👋

I'm using the azblob package and had some trouble running tests against Azurite.

These are the versions I'm using:

# import (
#    "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
#    "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/sas"
# )

> go list -m github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1

> go version
go version go1.23.1 darwin/arm64

> docker image ls | grep azurite
mcr.microsoft.com/azure-storage/azurite                   3.32.0            8ab49b8b5104 

What happened?

I have some Go code that uses azblob which is backed by Azurite when testing.

This all works fine when my tests are running locally, and Azurite is available on 127.0.0.1. The full URLs are something like

http://127.0.0.1:10000/devstoreaccount1/mycontainer/...

I can create a container, create a blob, create a SAS URL for that blob, and download it OK.

However, if Azurite is on a custom hostname (for example in Gitlab CI, or when tests are also run inside docker), then GET-ing the SAS URL fails with

        <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
        <Error>
          <Code>AuthorizationFailure</Code>
          <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.
        RequestId:de7d4aa7-8755-47ce-b53b-d8b45449b50b
        Time:2024-09-25T11:00:30.694Z</Message>
        </Error>

In this case the full URL is something like:

http://azurite:10000/devstoreaccount1/mycontainer/...

Note that I can see in the Azurite logs that creating the container and blob was successful (HTTP 200).

What did you expect or want to happen?

Expected http.Get(url) to work when url was created without error.

How can we reproduce it?

go.mod ``` module reprod go 1.23.1 require github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1 require ( github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0 // indirect github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect golang.org/x/net v0.27.0 // indirect golang.org/x/text v0.16.0 // indirect ) ```
reprod_test.go ``` package azblob import ( "cmp" "context" "io" "net" "net/http" "net/url" "os" "testing" "time" "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob" "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/bloberror" "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/sas" ) const ( accountName = "devstoreaccount1" accountKey = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==" container = "mycontainer" key = "mykey" content = "lorem ipsum" ) func TestAzurite(t *testing.T) { azuriteURL := &url.URL{ Scheme: "http", Host: net.JoinHostPort( cmp.Or( os.Getenv("AZURITE_HOST"), "127.0.0.1", ), "10000", ), Path: accountName, } t.Logf("URL: %s", azuriteURL.String()) creds, err := azblob.NewSharedKeyCredential(accountName, accountKey) if err != nil { t.Fatalf("new keys") } client, err := azblob.NewClientWithSharedKeyCredential(azuriteURL.String(), creds, nil) if err != nil { t.Fatalf("new client: %s", err) } // Create container _, err = client.CreateContainer(context.Background(), container, nil) if bloberror.HasCode(err, bloberror.ContainerAlreadyExists) { err = nil } if err != nil { t.Fatalf("create container: %s", err) } // Upload. _, err = client.UploadBuffer(context.Background(), container, key, []byte(content), nil) if err != nil { t.Fatalf("upload buf: %s", err) } // Create and use SAS URL. bc := client.ServiceClient().NewContainerClient(container).NewBlobClient(key) sasURL, err := bc.GetSASURL( sas.BlobPermissions{Read: true}, time.Now().Add(time.Hour), nil, ) if err != nil { t.Fatalf("Get SAS URL: %s", err) } t.Logf("SAS URL: %s", sasURL) resp, err := http.Get(sasURL) if err != nil { bs, _ := io.ReadAll(resp.Body) t.Fatalf("get sas url: %s\n%s", err, bs) } got, _ := io.ReadAll(resp.Body) t.Logf("Got content: %s", got) } ```
docker-compose.yaml ``` name: reprod services: azurite: image: mcr.microsoft.com/azure-storage/azurite:3.32.0 ports: - 10000:10000 mytest: image: golang:1.23 environment: - AZURITE_HOST=azurite volumes: - .:/ws working_dir: /ws entrypoint: ["go", "test", ".", "-v"] ```

Run

docker compose up azurite

then compare

go test . -v
docker compose run mytest

Anything we should know about your environment.

MacOS

github-actions[bot] commented 1 week ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.