search

micronaut-projects / micronaut-core

Micronaut Application Framework
http://micronaut.io
Apache License 2.0
6.07k stars 1.07k forks source link

Issue using in memory upload with large files. #2304

Closed claykirk closed 4 years ago

claykirk commented 4 years ago

Thanks for reporting an issue for Micronaut, please review the task list below before submitting the issue. Your issue report will be closed if the issue is incomplete and the below tasks not completed.

NOTE: If you are unsure about something and the issue is more of a question a better place to ask questions is on Stack Overflow (http://stackoverflow.com/tags/micronaut) or Gitter (https://gitter.im/micronautfw/). DO NOT use the issue tracker to ask questions.

Task List

Steps to Reproduce

  1. Run the application from the provided repo on github. Note the heap space setting to simulate limited space.

    From the project root folder:

    ./gradlew clean build -x test && java -Xmx1g -jar build/libs/upload-api.jar

    If you want heap dumps:

    ./gradlew clean build -x test && java -Xmx1g -XX:+HeapDumpOnOutOfMemoryError -jar build/libs/upload-api.jar

    The server runs on http://localhost:8080

  2. Test using curl. You will need a file > 1G to reproduce the heap space error.

    curl "http://localhost:8080/upload/receive-flow-control" -H "Content-Type:multipart/form-data" -F "file=@your-test-file"

Expected Behaviour

The file should consumed without any memory issues. The heap space grows as the file is being consumed by the controller.

Actual Behaviour

Heap Space is exhausted

17:38:23.052 [nioEventLoopGroup-1-2] ERROR i.m.h.s.netty.RoutingInBoundHandler - Unexpected error occurred: Java heap space java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Arrays.java:3520) at io.netty.buffer.ByteBufUtil.getBytes(ByteBufUtil.java:891) at io.netty.buffer.ByteBufUtil.getBytes(ByteBufUtil.java:872) at io.netty.buffer.ByteBufUtil.getBytes(ByteBufUtil.java:864) at io.micronaut.http.server.netty.multipart.NettyPartData.getBytes(NettyPartData.java:69) at io.micronaut.http.server.netty.converters.NettyPartDataToArrayConverter.convert(NettyPartDataToArrayConverter.java:46) at io.micronaut.http.server.netty.converters.NettyPartDataToArrayConverter.convert(NettyPartDataToArrayConverter.java:31) at io.micronaut.core.convert.DefaultConversionService.convert(DefaultConversionService.java:118) at io.micronaut.core.convert.ConversionService.convert(ConversionService.java:105) at io.micronaut.http.server.netty.RoutingInBoundHandler$1.doOnNext(RoutingInBoundHandler.java:843) at io.micronaut.core.async.subscriber.CompletionAwareSubscriber.onNext(CompletionAwareSubscriber.java:52) at io.micronaut.http.server.netty.FormDataHttpContentProcessor$$Lambda$402/237609194.accept(Unknown Source) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at io.micronaut.http.server.netty.FormDataHttpContentProcessor.onData(FormDataHttpContentProcessor.java:159) at io.micronaut.http.server.netty.AbstractHttpContentProcessor.doOnNext(AbstractHttpContentProcessor.java:78) at io.micronaut.http.server.netty.AbstractHttpContentProcessor.doOnNext(AbstractHttpContentProcessor.java:35) at io.micronaut.core.async.subscriber.CompletionAwareSubscriber.onNext(CompletionAwareSubscriber.java:52) at io.micronaut.http.netty.reactive.HandlerPublisher.publishMessage(HandlerPublisher.java:460) at io.micronaut.http.netty.reactive.HandlerPublisher.channelRead(HandlerPublisher.java:416) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.micronaut.http.netty.stream.HttpStreamsHandler.handleReadHttpContent(HttpStreamsHandler.java:239) at io.micronaut.http.netty.stream.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:222)

Environment Information

Example Application

https://github.com/claykirk/upload-api

jameskleeh commented 4 years ago

I was able to reproduce the issue. It seems like the GC can't keep up with the inflow when the PartData is converted to the byte[] the data is removed from its original source to the new array, then it seems the array is not being GCd.

jameskleeh commented 4 years ago

So I'm having trouble discerning what is causing the byte[] to not be GCd, however if you switch out byte[] with PartData then in your method do data.getBytes().length, the issue is resolved.

claykirk commented 4 years ago

Thanks for looking into this @jameskleeh. The PartData workaround works.