search

Azure / azure-storage-java

Microsoft Azure Storage Library for Java
https://docs.microsoft.com/en-us/java/api/overview/azure/storage
MIT License
189 stars 163 forks source link

Downloading files in sequence fails #493

Closed martenlindblad closed 4 years ago

martenlindblad commented 5 years ago

Which service(blob, file, queue, table) does this issue concern?

Block blob

Which version of the SDK was used?

Please note that if your issue is with v11, we are recommending customers either move back to v11 or move to v12 (currently in preview) if at all possible. Hopefully this resolves your issue, but if there is some reason why moving away from v11 is not possible at this time, please do continue to ask your question and we will do our best to support you. The README for this SDK has been updated to point to more information on why we have made this decision. 12.0.0-preview.2

What problem was encountered?

If I download a certain file manually it works, using the SDK I get the following exception

Exception in thread "main" com.azure.storage.blob.StorageException: Status code 400, "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request</h2>
<hr><p>HTTP Error 400. The request is badly formed.</p>
</BODY></HTML>
"
    at com.azure.storage.blob.Utility.lambda$1(Utility.java:272)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:118)
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1476)
    at reactor.core.publisher.MonoProcessor.subscribe(MonoProcessor.java:457)
    at reactor.core.publisher.MonoMap.subscribe(MonoMap.java:55)
    at reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
    at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
    at reactor.core.publisher.Mono.subscribe(Mono.java:3608)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:97)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:165)
    at reactor.core.publisher.FluxContextStart$ContextStartSubscriber.onError(FluxContextStart.java:117)
    at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:161)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:185)
    at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:251)
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onError(Operators.java:1718)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:185)
    at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:251)
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onError(Operators.java:1718)
    at reactor.core.publisher.Operators.error(Operators.java:181)
    at reactor.core.publisher.MonoError.subscribe(MonoError.java:52)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
    at reactor.core.publisher.Mono.subscribe(Mono.java:3608)
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:75)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onComplete(MonoFlatMap.java:174)
    at reactor.core.publisher.MonoProcessor$NextInner.onComplete(MonoProcessor.java:592)
    at reactor.core.publisher.MonoProcessor.onNext(MonoProcessor.java:380)
    at reactor.core.publisher.MonoProcessor.onComplete(MonoProcessor.java:335)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:141)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1476)
    at reactor.core.publisher.MonoProcessor.subscribe(MonoProcessor.java:457)
    at reactor.core.publisher.MonoMap.subscribe(MonoMap.java:55)
    at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
    at reactor.core.publisher.Mono.subscribe(Mono.java:3608)
    at reactor.core.publisher.MonoProcessor.add(MonoProcessor.java:531)
    at reactor.core.publisher.MonoProcessor.subscribe(MonoProcessor.java:444)
    at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
    at reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1476)
    at reactor.core.publisher.MonoProcessor.onNext(MonoProcessor.java:389)
    at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:123)
    at reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:113)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:287)
    at reactor.core.publisher.FluxUsing$UsingFuseableSubscriber.onNext(FluxUsing.java:350)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:113)
    at reactor.core.publisher.FluxPeekFuseable$PeekFuseableConditionalSubscriber.onNext(FluxPeekFuseable.java:486)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1476)
    at reactor.core.publisher.MonoReduceSeed$ReduceSeedSubscriber.onComplete(MonoReduceSeed.java:156)
    at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:136)
    at reactor.netty.channel.FluxReceive.terminateReceiver(FluxReceive.java:378)
    at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:202)
    at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:343)
    at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:325)
    at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:372)
    at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:511)
    at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:141)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1436)
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:677)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:612)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:529)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:491)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
    at java.base/java.lang.Thread.run(Thread.java:834)
    Suppressed: java.lang.Exception: #block terminated with an error
        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:93)
        at reactor.core.publisher.Mono.block(Mono.java:1493)
        at com.azure.storage.blob.Utility.blockWithOptionalTimeout(Utility.java:389)
        at com.azure.storage.blob.BlobClient.download(BlobClient.java:361)
        at com.azure.storage.blob.BlobClient.download(BlobClient.java:323)
        at xxxx.Xxx.lambda$main$0(ClimateParser.java:53)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)
        at xxxx.Xxx.main(ClimateParser.java:39)
Caused by: com.azure.storage.blob.models.StorageErrorException: Status code 400, "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request</h2>
<hr><p>HTTP Error 400. The request is badly formed.</p>
</BODY></HTML>
"
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at com.azure.core.implementation.RestProxy.instantiateUnexpectedException(RestProxy.java:315)
    at com.azure.core.implementation.RestProxy.lambda$null$2(RestProxy.java:363)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
    ... 68 more

Process finished with exit code 1

Have you found a mitigation/solution?

No

rickle-msft commented 5 years ago

@martenlindblad Thank you for reporting this issue. Can you please share which method you are calling with what parameters. If possible, a network capture of the failed request with something like Fiddler would also be very helpful. Otherwise, if you are able to set a breakpoint somewhere in the code where you can read the headers map on the outgoing request, we can likely get some information from that as well.

jaschrep-msft commented 5 years ago

The library you're reporting issue with, azure.storage.blob 12.0.0-preview.2, is developed in the https://github.com/Azure/azure-sdk-for-java/ repository. This repository hosts all pre-v12 work. We'd appreciate if you could post this issue over there.

As mentioned above, please try to include information about the request itself in that post. Thank you.

rickle-msft commented 4 years ago

@martenlindblad I am going to close this issue now as it has been inactive for a while with a request to move the discussion over to the other repo. We have also shipped a GA version of this package that we encourage you to try. Please reopen this issue or post again if you need further assistance.