search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.74k stars 309 forks source link

Receiving EOF error when using blob client in Go #2374

Open tforbus opened 3 months ago

tforbus commented 3 months ago

Which service(blob, file, queue, table) does this issue concern?

Blob

Which version of the Azurite was used?

Docker image 9f5f4eb775fe

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

DockerHub

What's the Node.js version?

N/A (?)

What problem was encountered?

Trying to use the client but getting EOF errors while creating a container. I copied the connection string from https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=docker-hub%2Cblob-storage#tabpanel_2_docker-hub

Steps to reproduce the issue?

compose.yaml

services:
  azurite:
    image: mcr.microsoft.com/azure-storage/azurite
    hostname: azurite
    command: "azurite -d /tmp/debug.log --blobHost 127.0.0.1 --blobPort 10000"
    ports:
      - "10000:10000"
    restart: always

go.mod

module example

go 1.21.4

require (
    github.com/Azure/azure-sdk-for-go/sdk/azcore v1.9.2 // indirect
    github.com/Azure/azure-sdk-for-go/sdk/internal v1.5.2 // indirect
    github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.3.1 // indirect
    golang.org/x/net v0.20.0 // indirect
    golang.org/x/text v0.14.0 // indirect
)

main.go

package main

import (
    "context"

    "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
)

const (
    connStr = "DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;"
)

func main() {
    client, err := azblob.NewClientFromConnectionString(connStr, nil)
    if err != nil {
        panic(err)
    }

    access := azblob.PublicAccessTypeBlob

    _, err = client.CreateContainer(context.Background(), "my-container", &azblob.CreateContainerOptions{
        Access: &access,
    })

    if err != nil {
        panic(err)
    }
}

Steps

Outputs:

panic: Put "http://127.0.0.1:10000/devstoreaccount1/my-container?restype=container": EOF

goroutine 1 [running]:
main.main()
    /.../main.go:26 +0x291

debug.log

2024-03-02T14:31:00.068Z     info: Azurite Blob service is starting on 127.0.0.1:10000
2024-03-02T14:31:00.074Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:31:00.075Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:31:00.106Z     info: BlobGCManager:start() Starting BlobGCManager. Set status to Initializing.
2024-03-02T14:31:00.107Z     info: BlobGCManager:start() Trigger mark and sweep loop. Set status to Running.
2024-03-02T14:31:00.108Z     info: BlobGCManager:markSweepLoop() Start next mark and sweep.
2024-03-02T14:31:00.108Z     info: BlobGCManager:markSweep() Get all extents.
2024-03-02T14:31:00.111Z     info: BlobGCManager:start() BlobGCManager successfully started.
2024-03-02T14:31:00.116Z     info: BlobGCManager:markSweep() Got 0 extents.
2024-03-02T14:31:00.116Z     info: BlobGCManager:markSweep() Get referred extents.
2024-03-02T14:31:00.118Z     info: BlobGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2024-03-02T14:31:00.119Z     info: BlobGCManager:markSweepLoop() Mark and sweep finished, taken 11ms.
2024-03-02T14:31:00.119Z     info: BlobGCManager:markSweepLoop() Sleep for 600000ms.
2024-03-02T14:31:00.122Z     info: Azurite Blob service successfully listens on http://127.0.0.1:10000
2024-03-02T14:31:00.123Z     info: Azurite Queue service is starting on 127.0.0.1:10001
2024-03-02T14:31:00.123Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:31:00.123Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:31:00.130Z     info: QueueGCManager:start() Starting QueueGCManager, set status to Initializing
2024-03-02T14:31:00.130Z     info: QueueGCManager:start() Trigger mark and sweep loop, set status to Running.
2024-03-02T14:31:00.131Z     info: QueueGCManager:markSweepLoop() Start new mark and sweep.
2024-03-02T14:31:00.131Z     info: QueueGCManger:markSweep() Get all extents.
2024-03-02T14:31:00.131Z     info: QueueGCManager:start() QueueGCManager successfully started.
2024-03-02T14:31:00.132Z     info: QueueGCManager:marksweep() Get 0 extents.
2024-03-02T14:31:00.132Z     info: QueueGCManager:markSweep() Get referred extents, then remove from allExtents.
2024-03-02T14:31:00.133Z     info: QueueGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2024-03-02T14:31:00.133Z     info: QueueGCManager:markSweepLoop() Mark and sweep finished, take 2ms.
2024-03-02T14:31:00.133Z     info: QueueGCManager:markSweepLoop() Sleep for 60000
2024-03-02T14:31:00.134Z     info: Azurite Queue service successfully listens on http://127.0.0.1:10001
2024-03-02T14:31:00.135Z     info: Azurite Table service is starting on 127.0.0.1:10002
2024-03-02T14:31:00.135Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:31:00.135Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:31:00.141Z     info: Azurite Table service successfully listens on http://127.0.0.1:10002
2024-03-02T14:32:00.014Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:32:00.014Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:32:00.054Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:32:00.055Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:32:00.065Z     info: QueueGCManager:markSweepLoop() Start new mark and sweep.
2024-03-02T14:32:00.065Z     info: QueueGCManger:markSweep() Get all extents.
2024-03-02T14:32:00.066Z     info: QueueGCManager:marksweep() Get 0 extents.
2024-03-02T14:32:00.066Z     info: QueueGCManager:markSweep() Get referred extents, then remove from allExtents.
2024-03-02T14:32:00.066Z     info: QueueGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2024-03-02T14:32:00.066Z     info: QueueGCManager:markSweepLoop() Mark and sweep finished, take 1ms.
2024-03-02T14:32:00.066Z     info: QueueGCManager:markSweepLoop() Sleep for 60000
2024-03-02T14:32:00.066Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:32:00.066Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:32:59.949Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:32:59.949Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:32:59.986Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:32:59.987Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:32:59.997Z     info: QueueGCManager:markSweepLoop() Start new mark and sweep.
2024-03-02T14:32:59.998Z     info: QueueGCManger:markSweep() Get all extents.
2024-03-02T14:32:59.998Z     info: QueueGCManager:marksweep() Get 0 extents.
2024-03-02T14:32:59.998Z     info: QueueGCManager:markSweep() Get referred extents, then remove from allExtents.
2024-03-02T14:32:59.998Z     info: QueueGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2024-03-02T14:32:59.998Z     info: QueueGCManager:markSweepLoop() Mark and sweep finished, take 0ms.
2024-03-02T14:32:59.998Z     info: QueueGCManager:markSweepLoop() Sleep for 60000
2024-03-02T14:32:59.998Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:32:59.998Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:33:59.883Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:33:59.884Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:33:59.921Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:33:59.922Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-03-02T14:33:59.931Z     info: QueueGCManager:markSweepLoop() Start new mark and sweep.
2024-03-02T14:33:59.931Z     info: QueueGCManger:markSweep() Get all extents.
2024-03-02T14:33:59.931Z     info: QueueGCManager:marksweep() Get 0 extents.
2024-03-02T14:33:59.931Z     info: QueueGCManager:markSweep() Get referred extents, then remove from allExtents.
2024-03-02T14:33:59.931Z     info: QueueGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2024-03-02T14:33:59.931Z     info: QueueGCManager:markSweepLoop() Mark and sweep finished, take 0ms.
2024-03-02T14:33:59.932Z     info: QueueGCManager:markSweepLoop() Sleep for 60000
2024-03-02T14:33:59.932Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-03-02T14:33:59.932Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.

Have you found a mitigation/solution?

No.

EmmaZhu commented 3 months ago

Hi @tforbus ,

From Azurite's log you shared, didn't see any request reached Azurite. Maybe you can ask GO SDK team to take a look at your code?