Closed theobisproject closed 3 months ago
@graemerocher How can I fix it? Micronaut controller / client method can't return InputStream? Which class I need to return?
I chaged return type:
InputStream
-> HttpResponse<FileCustomizableResponseType>
But I'm not sure if this is the right solution.
@theobisproject Could you check your code with my fix in https://github.com/micronaut-projects/micronaut-openapi/pull/1704 ?
@graemerocher Is this solution right?
@altro3 Sadly the changes also do not work. Trying to download a 4.5GB file leads to the following exception.
io.micronaut.http.client.exceptions.ContentLengthExceededException: Client 'file-download': The received length exceeds the maximum allowed content length [10485760]
at io.micronaut.http.client.netty.DefaultHttpClient$BaseHttpResponseHandler.exceptionCaught(DefaultHttpClient.java:2082)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Assembly trace from producer [reactor.core.publisher.FluxCreate] :
reactor.core.publisher.Flux.create(Flux.java:650)
io.micronaut.http.client.netty.DefaultHttpClient.lambda$exchangeImpl$27(DefaultHttpClient.java:1122)
Error has been observed at the following site(s):
*_________Flux.create ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.lambda$exchangeImpl$27(DefaultHttpClient.java:1122)
*____Mono.flatMapMany ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1109)
|_ Flux.next ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.<init>(ReactorExecutionFlowImpl.java:50)
|_ Mono.map ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.map(ReactorExecutionFlowImpl.java:71)
|_ Mono.onErrorResume ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.onErrorResume(ReactorExecutionFlowImpl.java:77)
|_ Mono.map ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.map(ReactorExecutionFlowImpl.java:71)
|_ Flux.from ⇢ at io.micronaut.configuration.metrics.binder.web.WebMetricsPublisher.<init>(WebMetricsPublisher.java:124)
*__________Mono.error ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.toMono(ReactorExecutionFlowImpl.java:145)
*___________Mono.from ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.<init>(ReactorExecutionFlowImpl.java:50)
|_ Mono.map ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.map(ReactorExecutionFlowImpl.java:71)
|_ Mono.onErrorResume ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.onErrorResume(ReactorExecutionFlowImpl.java:77)
|_ Mono.flatMap ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.flatMap(ReactorExecutionFlowImpl.java:59)
|_ Flux.from ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1144)
|_ Flux.timeout ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1153)
|_ Flux.onErrorResume ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1154)
*__________Mono.error ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.toMono(ReactorExecutionFlowImpl.java:145)
*__________Flux.error ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.lambda$exchangeImpl$28(DefaultHttpClient.java:1158)
*______Flux.switchMap ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchange(DefaultHttpClient.java:834)
|_ Flux.doOnNext ⇢ at io.micronaut.http.client.netty.DefaultHttpClient$1.exchange(DefaultHttpClient.java:561)
Original Stack Trace:
at io.micronaut.http.client.netty.DefaultHttpClient$BaseHttpResponseHandler.exceptionCaught(DefaultHttpClient.java:2082)
at io.micronaut.http.client.netty.DefaultHttpClient$FullHttpResponseHandler.exceptionCaught(DefaultHttpClient.java:2318)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:333)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:455)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at io.micronaut.http.client.netty.ResettableReadTimeoutHandler$NextInterceptor.channelRead(ResettableReadTimeoutHandler.java:92)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
at io.micronaut.http.client.netty.ResettableReadTimeoutHandler.channelRead(ResettableReadTimeoutHandler.java:64)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Suppressed: java.lang.Exception: #block terminated with an error
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:104)
at reactor.core.publisher.Flux.blockFirst(Flux.java:2766)
at io.micronaut.http.client.netty.DefaultHttpClient$1.exchange(DefaultHttpClient.java:571)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.lambda$handleSynchronous$1(HttpClientIntroductionAdvice.java:233)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.handleBlockingCall(HttpClientIntroductionAdvice.java:620)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.handleSynchronous(HttpClientIntroductionAdvice.java:232)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.intercept(HttpClientIntroductionAdvice.java:195)
at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:138)
at io.micronaut.validation.ValidatingInterceptor.validateReturnMicronautValidator(ValidatingInterceptor.java:153)
at io.micronaut.validation.ValidatingInterceptor.intercept(ValidatingInterceptor.java:139)
at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:138)
Trying a smaller file which fits into the content length leads to the same exception as with the inputstream.
io.micronaut.http.client.exceptions.HttpClientResponseException: Client 'file-download': Error decoding HTTP response body: No bean introspection available for type [interface io.micronaut.http.server.types.files.FileCustomizableResponseType]. Ensure the class is annotated with io.micronaut.core.annotation.Introspected
Also it feels a bit weird to add a dependency to micronaut-http-server
when I'm generating a client.
Ok, let's try to return StreamedFile
@theobisproject try i t again, pls
Also leads to the bean introspection error
Error decoding HTTP response body: No bean introspection available for type [class io.micronaut.http.server.types.files.StreamedFile]. Ensure the class is annotated with io.micronaut.core.annotation.Introspected
@theobisproject thanks
@sdelamo @graemerocher @yawkat @dstepanov Need help with this case. I don’t understand what needs to be generated for the http client to work correctly
@yawkat Could you help?
That class is from the server package, not sure if we have something for client.
@dstepanov and what can we do with it? How to write correct client if you want to get file as InputStream?
don't think it is supported today to read a file as an input stream so this feature shouldn't even exist in the generator. Likely this is fixable though now with the byte body abstraction that @yawkat implemented
The first thing to do would be to implement the feature in core and add tests before generating code for a feature that doesn't exist.
It's not possible still because ByteBody is implemented only for the server atm
@graemerocher @dstepanov @yawkat I have only one question: It turns out that I can’t describe a method in an HTTP client that will return me the body of a binary file (in the form of a stream or a byte array or any other object)? I tried different ways to describe the client method, but I still couldn't get the file.
For example, I have this endpoint:
@Get("/{id}")
public FileCustomizableResponseType fetchData(
@PathVariable("id") @NotNull @Min(0L) Long id
) {
try {
return new StreamedFile(new FileInputStream("test.zip"), MediaType.ZIP_TYPE);
} catch (FileNotFoundException e) {
throw new RuntimeException(e);
}
}
How I need to describe HTTP client to get result of this method? I tried different vriants:
@Get("/{id}")
byte[] fetchData(
@PathVariable("id") @NotNull @Min(0L) Long id
);
@Get("/{id}")
Object fetchData(
@PathVariable("id") @NotNull @Min(0L) Long id
);
But every time I see that error:
Internal Server Error
io.micronaut.http.client.exceptions.HttpClientResponseException: Internal Server Error
at io.micronaut.http.client.netty.DefaultHttpClient$FullHttpResponseHandler.makeErrorFromRequestBody(DefaultHttpClient.java:2256)
Suppressed: The stacktrace has been enhanced by Reactor, refer to additional information below:
Assembly trace from producer [reactor.core.publisher.FluxCreate] :
reactor.core.publisher.Flux.create(Flux.java:650)
io.micronaut.http.client.netty.DefaultHttpClient.lambda$exchangeImpl$27(DefaultHttpClient.java:1112)
Error has been observed at the following site(s):
*_________Flux.create ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.lambda$exchangeImpl$27(DefaultHttpClient.java:1112)
*____Mono.flatMapMany ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1099)
|_ Flux.next ⇢ at io.micronaut.http.reactive.execution.ReactorExecutionFlowImpl.<init>(ReactorExecutionFlowImpl.java:50)
|_ Flux.from ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1134)
|_ Flux.timeout ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1144)
|_ Flux.onErrorResume ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchangeImpl(DefaultHttpClient.java:1145)
*__________Flux.error ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.lambda$exchangeImpl$30(DefaultHttpClient.java:1149)
*______Flux.switchMap ⇢ at io.micronaut.http.client.netty.DefaultHttpClient.exchange(DefaultHttpClient.java:824)
Original Stack Trace:
at io.micronaut.http.client.netty.DefaultHttpClient$FullHttpResponseHandler.makeErrorFromRequestBody(DefaultHttpClient.java:2256)
at io.micronaut.http.client.netty.DefaultHttpClient$FullHttpResponseHandler.forwardResponseToPromise(DefaultHttpClient.java:2210)
at io.micronaut.http.client.netty.DefaultHttpClient$FullHttpResponseHandler.channelReadInstrumented(DefaultHttpClient.java:2180)
at io.micronaut.http.client.netty.DefaultHttpClient$FullHttpResponseHandler.channelReadInstrumented(DefaultHttpClient.java:2148)
at io.micronaut.http.client.netty.SimpleChannelInboundHandlerInstrumented.channelRead0(SimpleChannelInboundHandlerInstrumented.java:46)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Suppressed: java.lang.Exception: #block terminated with an error
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:104)
at reactor.core.publisher.Flux.blockFirst(Flux.java:2766)
at io.micronaut.http.client.netty.DefaultHttpClient$1.exchange(DefaultHttpClient.java:566)
at io.micronaut.http.client.netty.DefaultHttpClient$1.retrieve(DefaultHttpClient.java:574)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.lambda$handleSynchronous$3(HttpClientIntroductionAdvice.java:241)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.handleBlockingCall(HttpClientIntroductionAdvice.java:619)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.handleSynchronous(HttpClientIntroductionAdvice.java:240)
at io.micronaut.http.client.interceptor.HttpClientIntroductionAdvice.intercept(HttpClientIntroductionAdvice.java:194)
at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:143)
at io.micronaut.validation.ValidatingInterceptor.validateReturnMicronautValidator(ValidatingInterceptor.java:153)
at io.micronaut.validation.ValidatingInterceptor.intercept(ValidatingInterceptor.java:139)
at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:143)
at com.example.openapi.api.DefaultApi2$Intercepted.fetchData(Unknown Source)
at com.micronaut.bug.controller.MyEntityControllerTest.myTest(MyEntityControllerTest.java:20)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.micronaut.test.extensions.junit5.MicronautJunit5Extension$2.proceed(MicronautJunit5Extension.java:142)
at io.micronaut.test.extensions.AbstractMicronautExtension.interceptEach(AbstractMicronautExtension.java:162)
at io.micronaut.test.extensions.AbstractMicronautExtension.interceptTest(AbstractMicronautExtension.java:119)
at io.micronaut.test.extensions.junit5.MicronautJunit5Extension.interceptTestMethod(MicronautJunit5Extension.java:129)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
I can't believe there is no way I can describe the client's method for getting the file in the response at the moment. Is this really true?
That error looks more like the client representing a 500, there should be an associated server error.
But yes the streaming support of the client is very limited atm, I think the best way is a publisher.
Ok, for fast fix, I think, we can replace InputSream
to ByteBuffer<?>
- I checked it locally - all works fine. About Publisher: if user want, he can enable reactive mode and return type in client will be Mono<ByteBuffer<?>>
.
About streaming support: it is different task for core, I think
@theobisproject check it, please, now all must be ok
@altro3 For files smaller than the allowed content length this works great.
For larger files I still see the error shown here. Unsure if the openapi generator can do anything there or if this is somthing fore the core project. From my point of view there should be a chunking in place for requests larger than the allowed content length.
@theobisproject you can up max content length in client configuration. Set max-content-length
property for it. But maximal value is Integer.MAX_VALUE ~ 2_147_483_647 bytes ~ 2 Gb
@theobisproject this is about files much than 2Gb: https://github.com/micronaut-projects/micronaut-core/issues/975
Yes this is the same problem. As described the API will provide files larger than 10GB. I will see what I can do :smile:
Expected Behavior
The generated client is able to download files from the API. The API is expected to deliver files >10GB.
Actual Behaviour
Calling the client fails with the following exception
I already found micronaut-projects/micronaut-core#10256 but I found no way to influence the openapi-generator to generate method signature with byte[]. It would still not be usable since the size of a byte[] is limited. The documentation for downloading a big file with StreamingHttpClient also does not help me with the generated client.
I also tried the reproduction steps with
reactive = true
for which the client sadly fails in the same way.Steps To Reproduce
Take the following OpenAPI spec
Generate a client with micronaut-openapi
Resulting class
Environment Information
Operating System: Ubuntu 22.04 Java Version: 21.0.2
Example Application
No response
Version
4.5.1