Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.48k stars 4.81k forks source link

[BUG] Empty files are produced when outputting results of an azure function to a storage account using blob output bindings #41182

Open ms-bemartins opened 10 months ago

ms-bemartins commented 10 months ago

Library name and version

Azure Storage Blobs v2 programming model

Describe the bug

Function app is writing to the blobs using an output binding and it’s running well most of the time but intermittently we see blobs with zero length, when this isn’t supposed to happen. This happens with more frequency when we increase the number of workers inside the function app or the amount of requests.

The function calls return successfully, even though the blobs remain empty.

Adding to this, the behaviour only happens on Premium V3 plan. Can't reproduce this on S1, S2 or even P1V2.

Expected behavior

Expected the blobs to be created with the desired content.

Actual behavior

Multiple blobs are produced with 0 B.

Reproduction Steps

Using this function app code below (4 output bindings):

@app.route(route="use_binding")`
@app.blob_output(
    arg_name="blob01",
    path="output1/v0/binding/{rand-guid}.dat",
    connection="outputBlobStorage",
)
@app.blob_output(
    arg_name="blob02",
    path="output1/v0/binding/{rand-guid}.dat",
    connection="outputBlobStorage",
)
@app.blob_output(
    arg_name="blob03",
    path="output1/v0/binding/{rand-guid}.dat",
    connection="outputBlobStorage",
)
@app.blob_output(
    arg_name="blob04",
    path="output1/v0/binding/{rand-guid}.dat",
    connection="outputBlobStorage",
)
async def write_blobs_with_binding(
    req: func.HttpRequest,
    blob01: func.Out[str],
    blob02: func.Out[str],
    blob03: func.Out[str],
    blob04: func.Out[str],
) -> func.HttpResponse:
    ts_start = time.time_ns()
    try:
        blob01.set(random.randbytes(BLOB_SIZE))
        blob02.set(random.randbytes(BLOB_SIZE))
        blob03.set(random.randbytes(BLOB_SIZE))
        blob04.set(random.randbytes(BLOB_SIZE))
    except Exception as e:
        print(f"EXCEPTION: {e}")

    time_taken = (time.time_ns() - ts_start) / 1e6
    return func.HttpResponse(f"{time_taken:.1f}ms", `status_code=200)

There were over 10K requests sent to this app and it produced multiple 0 B blobs. From these results, it seems that in scenarios where there are multiple blob output bindings or multiple .set() operations, the extension outputs empty blobs intermittently.

Environment

Azure App Service Premium V3 Plan

github-actions[bot] commented 10 months ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.