search

DarkAtra / v-rising-discord-bot

This bot allows you to display some information about your v rising server on Discord.
https://vrising.darkatra.dev
MIT License
15 stars 5 forks source link

feat(native): create native image #63

Closed DarkAtra closed 1 year ago

DarkAtra commented 1 year ago

The issue One of the biggest drawbacks of this bot is its memory consumption. With the recent upgrade to Spring Boot 3 we could try to build a native image using GraalVM.

Proposed solution Configure the spring-boot-maven-plugin so that it builds a native image via GraalVM. This would result in near instant startup times and lower memory consumption for the bot. It would also eliminate the need for a JVM.

Notes I've already attempted to build a native image when Spring Boot 3 was released and it didn't work out of the box. The main issue back then was that kotlin coroutines didn't play nicely with native image, see: https://github.com/oracle/graal/issues/5606. Also, the database is using jackson for serialization, e.g. runtime hints are required, see: https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/aot/hint/annotation/RegisterReflectionForBinding.html

DarkAtra commented 1 year ago

Related:

DarkAtra commented 1 year ago

added runtime hints:

The native image compiles and runs fine. Tested:

Note: The current next branch is using a snapshot build of Kord (see: https://github.com/kordlib/kord/pull/787).

Pre Releases:

DarkAtra commented 1 year ago

There seems to be an issue with the native build. I'm currently trying to find the root cause. What i can tell right now is that it has something to do with the async-gamequery-lib and/or netty.

Stacktrace:

2023-03-21T00:57:52.763Z ERROR 1 --- [pool-2-thread-1] d.d.v.d.ServerStatusMonitorService       : Exception while fetching the status of 0c7a7e95-ff7a-11ec-8875-199d13cc1b6e
java.util.concurrent.CompletionException: java.lang.IllegalStateException: channel not registered to an event loop
    at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.completion(SourceQueryMessenger.java:368) ~[na:na]
    at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.access$1000(SourceQueryMessenger.java:279) ~[na:na]
    at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.lambda$send$1(SourceQueryMessenger.java:240) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.spi.FailsafeFuture.completeResult(FailsafeFuture.java:99) ~[na:na]
    at dev.failsafe.AsyncExecutionImpl.complete(AsyncExecutionImpl.java:153) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.internal.RetryPolicyExecutor.lambda$null$5(RetryPolicyExecutor.java:155) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:887) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2325) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.internal.RetryPolicyExecutor.lambda$handleAsync$6(RetryPolicyExecutor.java:150) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.Functions.lambda$null$2(Functions.java:152) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[de.darkatra.vrising.discord.BotKt:na]
    at com.ibasco.agql.core.transport.pool.SimpleNettyChannelPool.lambda$notifyOnComplete$2(SimpleNettyChannelPool.java:253) ~[de.darkatra.vrising.discord.BotKt:1.1.0]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.spi.FailsafeFuture.completeResult(FailsafeFuture.java:99) ~[na:na]
    at dev.failsafe.AsyncExecutionImpl.complete(AsyncExecutionImpl.java:153) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.internal.RetryPolicyExecutor.lambda$null$5(RetryPolicyExecutor.java:155) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:887) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2325) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.internal.RetryPolicyExecutor.lambda$handleAsync$6(RetryPolicyExecutor.java:150) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.spi.PolicyExecutor.lambda$null$2(PolicyExecutor.java:109) ~[de.darkatra.vrising.discord.BotKt:3.3.0]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.Functions.lambda$null$5(Functions.java:202) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:887) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2325) ~[de.darkatra.vrising.discord.BotKt:na]
    at dev.failsafe.Functions.lambda$null$6(Functions.java:198) ~[na:na]
    at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:96) ~[de.darkatra.vrising.discord.BotKt:4.1.89.Final]
    at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[de.darkatra.vrising.discord.BotKt:4.1.89.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[de.darkatra.vrising.discord.BotKt:4.1.89.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[de.darkatra.vrising.discord.BotKt:4.1.89.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[de.darkatra.vrising.discord.BotKt:4.1.89.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403) ~[na:na]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[na:na]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[na:na]
    at java.base@17.0.6/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[de.darkatra.vrising.discord.BotKt:na]
    at java.base@17.0.6/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[na:na]
    at java.base@17.0.6/java.lang.Thread.run(Thread.java:833) ~[de.darkatra.vrising.discord.BotKt:na]
    at com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775) ~[de.darkatra.vrising.discord.BotKt:na]
    at com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203) ~[na:na]
Caused by: java.lang.IllegalStateException: channel not registered to an event loop
    at io.netty.channel.AbstractChannel.eventLoop(AbstractChannel.java:164) ~[de.darkatra.vrising.discord.BotKt:4.1.89.Final]
    at com.ibasco.agql.core.util.Netty.toCompletable(Netty.java:101) ~[na:na]
    at com.ibasco.agql.core.transport.UdpNettyChannelFactory.newChannel(UdpNettyChannelFactory.java:75) ~[na:na]
    at com.ibasco.agql.core.transport.AbstractNettyChannelFactory.create(AbstractNettyChannelFactory.java:234) ~[de.darkatra.vrising.discord.BotKt:1.1.0]
    at com.ibasco.agql.core.transport.NettyChannelFactoryDecorator.create(NettyChannelFactoryDecorator.java:55) ~[de.darkatra.vrising.discord.BotKt:1.1.0]
    at com.ibasco.agql.core.transport.FailsafeChannelFactory.access$101(FailsafeChannelFactory.java:52) ~[na:na]
    at com.ibasco.agql.core.transport.FailsafeChannelFactory$ChannelSupplier.get(FailsafeChannelFactory.java:138) ~[na:na]
    at com.ibasco.agql.core.transport.FailsafeChannelFactory$ChannelSupplier.get(FailsafeChannelFactory.java:127) ~[na:na]
    at dev.failsafe.Functions.lambda$getPromiseOfStage$3(Functions.java:136) ~[na:na]
    ... 15 common frames omitted

It seems to be specific to linux builds as both Windows and MacOS executables are working just fine.

The error can also be reproduced in WSL by compiling via (Note, that GraalVM needs to be installed in WSL):

mvn -Pnative clean native:compile

and then running the resulting exectuable with at least one server in the database.

DarkAtra commented 1 year ago

The above issue can be avoided on linux by passing -Dagql.nativeTransport=false as an argument to the docker image or the executable.

Example for docker-compose:

services:
  v-rising-discord-bot:
    image: ghcr.io/darkatra/v-rising-discord-bot:2.0.6-next.3-native
    command: -Dagql.nativeTransport=false
    mem_reservation: 256M
    mem_limit: 512M
    volumes:
      - /opt/v-rising-discord-bot:/data/v-rising-discord-bot
    environment:
      - BOT_DISCORD_BOT_TOKEN=<your-discord-bot-token>
      - BOT_DATABASE_PATH=/data/v-rising-discord-bot/bot.db
      - BOT_DATABASE_PASSWORD=<the-database-password>
    restart: unless-stopped
DarkAtra commented 1 year ago

progress is now tracked in the PR: https://github.com/DarkAtra/v-rising-discord-bot/pull/65