Azure / azure-storage-java

Microsoft Azure Storage Library for Java
https://docs.microsoft.com/en-us/java/api/overview/azure/storage
MIT License
189 stars 165 forks source link

upload of large files to blob storage fails #564

Open connriverdesign opened 3 years ago

connriverdesign commented 3 years ago

Which service(blob, file, queue, table) does this issue concern?

upload of file to blob storage

Which version of the SDK was used?

Please note that if your issue is with v11, we are recommending customers either move back to v11 or move to v12 (currently in preview) if at all possible. Hopefully this resolves your issue, but if there is some reason why moving away from v11 is not possible at this time, please do continue to ask your question and we will do our best to support you. The README for this SDK has been updated to point to more information on why we have made this decision.

com.azure azure-storage-blob 12.11.0

What problem was encountered?

upload is failing with files over ~5GB error message looks like this in all cases... "Caused by: reactor.core.Exceptions$ReactiveException: io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer at reactor.core.Exceptions.propagate(Exceptions.java:392) at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:97) at reactor.core.publisher.Mono.block(Mono.java:1703) at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:128) at com.azure.storage.blob.BlobContainerClient.createWithResponse(BlobContainerClient.java:266) at com.azure.storage.blob.BlobServiceClient.createBlobContainerWithResponse(BlobServiceClient.java:132) at com.azure.storage.blob.BlobServiceClient.createBlobContainer(BlobServiceClient.java:107) at com.tomtom.namgs.gsapi.product.storage.BlobClientFactory.makeUploadBlobClient(BlobClientFactory.java:23) at com.tomtom.namgs.gsapi.product.storage.BlobStorageService.uploadFromFile(BlobStorageService.java:22) ... 16 more Suppressed: java.lang.Exception: #block terminated with an error at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) ... 23 more Caused by: io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer"

We are running this application on an Azure VM instance running Ubuntu 18.04

When I attempt upload to a blob from my local machine of same size it succeeds. I do not know how to proceed with resolution.

Have you found a mitigation/solution?

no, we are locked with constant failures and have tried several approaches using the client.

Let me know what other information I can provide. We are a business partner of yours and this application is deemed important. Thanks for any support you can provide.

Just updating to see if it get's attention that way...