storage: signed url with STORAGE_EMULATOR_HOST get no credentials error

[rainoko]

Client

Storage

Environment

local emulator

Go Environment

$ go 1.21

Expected behavior

Storage client with emulator host should bypass credentials check

Actual behavior

When storage client is created creds will be nil (storage.co NewClient) But when asking signedUrl in bucket.go it tries to evaluate GoogleAccessId.

For example in same situation for pubsub emulator I can use fake google credentials json. But in storage implementation it skipped.

Although I can Add access id as in options. I want to create integration test that uses exact config as in prod. so I do not want ot add opts that are not used in prod.

[BrennaEpp]

Hi @rainoko, thank you for opening this issue. Just a couple questions for you:

What is the exact error message you are seeing?

What do you mean by the following? Did you attempt to use a fake google credentials JSON on the Storage client?

For example in same situation for pubsub emulator I can use fake google credentials json. But in storage implementation it skipped

Can you provide the code you are using to sign URLs? If you are not passing in a private key or a SignBytes function, iamcredentials.SignBlobRequest is called, so STORAGE_EMULATOR_HOST would need to be able to handle that call, which seems a bit out of scope.

[tomob]

So, the issue is that when STORAGE_EMULATOR_HOST is set, the client is created without credentials: https://github.com/googleapis/google-cloud-go/blob/c6711b83cb6f9f35032e69a40632b7268fcdbd0a/storage/storage.go#L161-L186

But later, when SingedURL is called, and there are no explicit credentials in options, it tries to detect them (https://github.com/googleapis/google-cloud-go/blob/c6711b83cb6f9f35032e69a40632b7268fcdbd0a/storage/bucket.go#L191), relying on bucket's client reference (https://github.com/googleapis/google-cloud-go/blob/c6711b83cb6f9f35032e69a40632b7268fcdbd0a/storage/bucket.go#L268). And when storage emulator is used, the credentials are nil.

In our production code we have something like:

    data := // read credentials file in a container

    client, err := storage.NewClient(ctx, option.WithCredentialsJSON(data))
    if err != nil {
        return nil, err
    }

    // ...

    url, err := c.client.Bucket(bucket).SignedURL(path, &storage.SignedURLOptions{
        Method:  method,
        Expires: expire,
    })

This work in production, but not in a test environment with storage emulator. So even though the test credentials are provided (through the data variable), they are not used and SignedURL fails.

How that helps fixing it.

[System-Glitch]

Same issue here. Providing GOOGLE_APPLICATION_CREDENTIALS env variable also has an effect even if STORAGE_EMULATOR_HOST is set. It shouldn't send the OAuth token with the request in that case I think.