search

CloudNetService / CloudNet

A modern application that can dynamically and easily deliver Minecraft oriented software
https://cloudnetservice.eu
Apache License 2.0
367 stars 115 forks source link

Netty5: OutOfMemeory and discontinued #1397

Closed JWeinelt closed 1 month ago

JWeinelt commented 2 months ago

Stacktrace

Unable to push template local:SkyWars/default to cluster
java.util.concurrent.CompletionException: io.netty5.handler.codec.EncoderException: java.lang.OutOfMemoryError: Cannot reserve 52428909 bytes of direct buffer memory (allocated: 253808145, limit: 268435456)
        at io.netty5.util.concurrent.DefaultPromise.rethrowIfFailed(DefaultPromise.java:531)
        at io.netty5.util.concurrent.DefaultPromise.sync(DefaultPromise.java:351)
        at eu.cloudnetservice.driver.network.netty.NettyUtil.awaitFuture(NettyUtil.java:223)
        at eu.cloudnetservice.driver.network.netty.NettyNetworkChannel.sendPacketSync(NettyNetworkChannel.java:114)
        at eu.cloudnetservice.driver.network.chunk.defaults.splitter.NetworkChannelsPacketSplitter.accept(NetworkChannelsPacketSplitter.java:45)
        at eu.cloudnetservice.driver.network.chunk.defaults.splitter.NetworkChannelsPacketSplitter.accept(NetworkChannelsPacketSplitter.java:30)
        at eu.cloudnetservice.driver.network.chunk.defaults.DefaultFileChunkPacketSender.lambda$transferChunkedData$0(DefaultFileChunkPacketSender.java:93)
        at eu.cloudnetservice.common.concurrent.Task.lambda$supply$1(Task.java:71)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: io.netty5.handler.codec.EncoderException: java.lang.OutOfMemoryError: Cannot reserve 52428909 bytes of direct buffer memory (allocated: 253808145, limit: 268435456)
        at io.netty5.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:95)
        at io.netty5.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:853)
        at io.netty5.channel.DefaultChannelHandlerContext$AbstractWriteTask.write(DefaultChannelHandlerContext.java:1216)
        at io.netty5.channel.DefaultChannelHandlerContext$WriteAndFlushTask.write(DefaultChannelHandlerContext.java:1267)
        at io.netty5.channel.DefaultChannelHandlerContext$AbstractWriteTask.run(DefaultChannelHandlerContext.java:1186)
        at io.netty5.util.concurrent.SingleThreadEventExecutor.runTask(SingleThreadEventExecutor.java:338)
        at io.netty5.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:361)
        at io.netty5.channel.SingleThreadEventLoop.run(SingleThreadEventLoop.java:180)
        at io.netty5.util.concurrent.SingleThreadEventExecutor.lambda$doStartThread$4(SingleThreadEventExecutor.java:774)
        at io.netty5.util.internal.ThreadExecutorMap.lambda$apply$1(ThreadExecutorMap.java:68)
        at io.netty5.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        ... 1 more
Caused by: java.lang.OutOfMemoryError: Cannot reserve 52428909 bytes of direct buffer memory (allocated: 253808145, limit: 268435456)
        at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
        at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:111)
        at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360)
        at io.netty5.buffer.bytebuffer.ByteBufferMemoryManager.allocateShared(ByteBufferMemoryManager.java:51)
        at io.netty5.buffer.pool.UnpooledUntetheredMemory.<init>(UnpooledUntetheredMemory.java:36)
        at io.netty5.buffer.pool.PoolArena.allocateHuge(PoolArena.java:226)
        at io.netty5.buffer.pool.PoolArena.allocate(PoolArena.java:126)
        at io.netty5.buffer.pool.PooledBufferAllocator.allocateUntethered(PooledBufferAllocator.java:343)
        at io.netty5.buffer.pool.PooledBufferAllocator.allocate(PooledBufferAllocator.java:319)
        at io.netty5.buffer.DefaultBufferAllocators$UncloseableBufferAllocator.allocate(DefaultBufferAllocators.java:122)
        at eu.cloudnetservice.driver.network.netty.codec.NettyPacketEncoder.allocateBuffer(NettyPacketEncoder.java:61)
        at eu.cloudnetservice.driver.network.netty.codec.NettyPacketEncoder.allocateBuffer(NettyPacketEncoder.java:40)
        at io.netty5.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:79)
        ... 11 more

According to the Netty website, https://netty.io/wiki/user-guide.html Netty5 is abandoned and unsupported. Please switch to Netty 4

Actions to reproduce

Create a Cluster and try to deploy a template using cluster push <template>. Then there will this error.

CloudNet version

CloudNet Blizzard 4.0.0-RC10 887ad7cf
[08.05 20:37:34.120] INFO: Discord: <https://discord.cloudnetservice.eu/>
[08.05 20:37:34.120] INFO:  
[08.05 20:37:34.121] INFO: ClusterId: 45767933-****-416c-****-297d644eb376
[08.05 20:37:34.121] INFO: NodeId: Node-1
[08.05 20:37:34.122] INFO: Head-NodeId: Node-1
[08.05 20:37:34.122] INFO: CPU usage: (P/S) 21.28/23.4/100%
[08.05 20:37:34.122] INFO: Node services memory allocation (U/R/M): 4096/4096/15498 MB
[08.05 20:37:34.123] INFO: Threads: 38
[08.05 20:37:34.124] INFO: Heap usage: 42/256MB
[08.05 20:37:34.124] INFO: JVM: Oracle Corporation 21 (Java HotSpot(TM) 64-Bit Server VM 21.0.3+7-LTS-152)
[08.05 20:37:34.125] INFO: Update Repo: CloudNetService/launchermeta, Update Branch: beta

Other

No response

Issue uniqueness

JWeinelt commented 2 months ago

Now I got this:

root@4.0.0-RC10-887ad7cf => cluster push staticServices Citybuild-1 
[08.05 20:40:23.965] INFO: Die Übermittlung des statischen Service in das Cluster wird gestartet...
[08.05 20:40:26.751] SEVERE: Exception in thread "pool-8-thread-4" 
[08.05 20:40:26.751] SEVERE: java.lang.OutOfMemoryError: Cannot reserve 52428873 bytes of direct buffer memory (allocated: 254043708, limit: 268435456)
[08.05 20:40:26.751] SEVERE:    at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
[08.05 20:40:26.751] SEVERE:    at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:111)
[08.05 20:40:26.751] SEVERE:    at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.bytebuffer.ByteBufferMemoryManager.allocateShared(ByteBufferMemoryManager.java:51)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.pool.UnpooledUntetheredMemory.<init>(UnpooledUntetheredMemory.java:36)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.pool.PoolArena.allocateHuge(PoolArena.java:226)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.pool.PoolArena.allocate(PoolArena.java:126)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.pool.PooledBufferAllocator.allocateUntethered(PooledBufferAllocator.java:343)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.pool.PooledBufferAllocator.allocate(PooledBufferAllocator.java:319)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.bytebuffer.NioBuffer.ensureWritable(NioBuffer.java:454)
[08.05 20:40:26.751] SEVERE:    at io.netty5.buffer.Buffer.writeBytes(Buffer.java:471)
[08.05 20:40:26.751] SEVERE:    at eu.cloudnetservice.driver.network.netty.buffer.NettyMutableDataBuf.writeByteArray(NettyMutableDataBuf.java:133)
[08.05 20:40:26.751] SEVERE:    at eu.cloudnetservice.driver.network.chunk.network.ChunkedPacket.createChunk(ChunkedPacket.java:99)
[08.05 20:40:26.751] SEVERE:    at eu.cloudnetservice.driver.network.chunk.network.ChunkedPacket.createChunk(ChunkedPacket.java:64)
[08.05 20:40:26.751] SEVERE:    at eu.cloudnetservice.driver.network.chunk.defaults.DefaultFileChunkPacketSender.lambda$transferChunkedData$0(DefaultFileChunkPacketSender.java:93)
[08.05 20:40:26.751] SEVERE:    at eu.cloudnetservice.common.concurrent.Task.lambda$supply$1(Task.java:71)
[08.05 20:40:26.751] SEVERE:    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[08.05 20:40:26.752] SEVERE:    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[08.05 20:40:26.752] SEVERE:    at java.base/java.lang.Thread.run(Thread.java:1583)
[08.05 20:40:56.042] INFO: Die Verbindung zur Node [uniqueId=Node-2] wurde unterbrochen!
derklaro commented 2 months ago

Please, feel free to switch our networking setup to netty4 real quick :)

0utplay commented 1 month ago

I think it's clear that netty 5 is not discontinued but rather the documentation was not updated