Closed ivangfr closed 4 years ago
Thanks we will investigate
I have managed to reproduce the issue locally. I've tweaked the NewsClient to send in batch mode - this resulted in the messages getting stored on the topic but it would complain about a bad CRC during retrieval. @graemerocher Could this be related to the following substitution: https://github.com/micronaut-projects/micronaut-kafka/blob/47aaeb59a22059393f553d908586f640ba2ca9e6/kafka/src/main/java/io/micronaut/configuration/kafka/graal/KafkaSubstitutions.java#L41 From what I've seen, this will only get triggered when using Java 9+, which would explain why it works on 8 and not on 11. It expects to get a CRC32C implementation, but this would supply CRC32.
@gradinac thanks for looking into it, it does seems to be the case let me try push a patch and hopefully @ivangfr can provide feedback on the snapshot
@gradinac what do you think about this approach? https://github.com/micronaut-projects/micronaut-kafka/pull/153
@graemerocher No problem at all! Hopefully it will solve the issue. If not, I'll dig a bit further. As for #153 I think this would be the proper approach to fixing it.
Sure @graemerocher I will try it using the snapshot and I let you guys know.
Wait a bit the snapshot is not published yet
Ok the snapshot is published
@ivangfr were you able to verify the issue is resolved with the snapshot?
@graemerocher I am having a problem to build the native image
cd producer-api && ./docker-build.sh native && cd ..
Sending build context to Docker daemon 26.78MB
Step 1/10 : FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
---> 364d0bb387bd
Step 2/10 : RUN gu install native-image
---> Running in bed8a465e0c3
Downloading: Component catalog from www.graalvm.org
Processing Component: Native Image
Downloading: Component native-image: Native Image from github.com
Installing new component: Native Image (org.graalvm.native-image, version 20.0.0)
Refreshed alternative links in /usr/bin/
Removing intermediate container bed8a465e0c3
---> 429860ef0443
Step 3/10 : COPY build/libs/producer-api-*-all.jar /home/app/producer-api/build/libs/producer-api.jar
---> 79a1d3e9b26e
Step 4/10 : WORKDIR /home/app/producer-api
---> Running in f3e7a846c390
Removing intermediate container f3e7a846c390
---> d90789be648a
Step 5/10 : RUN native-image --no-server -cp build/libs/producer-api.jar
---> Running in cfc7b062d1a2
[producer-api:28] classlist: 33,370.09 ms, 1.20 GB
[producer-api:28] (cap): 9,058.38 ms, 1.20 GB
WARNING: Could not resolve io.netty.channel.epoll.EpollChannelOption for reflection configuration.
WARNING: Could not resolve io.netty.channel.kqueue.KQueueChannelOption for reflection configuration.
[producer-api:28] setup: 27,921.67 ms, 1.20 GB
Warning: class initialization of class io.micronaut.http.bind.binders.ContinuationArgumentBinder$Companion failed with exception java.lang.NoClassDefFoundError: kotlin/TypeCastException. This class will be initialized at run time because option --allow-incomplete-classpath is used for image building. Use the option --initialize-at-run-time=io.micronaut.http.bind.binders.ContinuationArgumentBinder$Companion to explicitly request delayed initialization of this class.
[producer-api:28] (typeflow): 397,826.93 ms, 6.22 GB
[producer-api:28] (objects): 134,681.33 ms, 6.22 GB
[producer-api:28] (features): 22,714.30 ms, 6.22 GB
[producer-api:28] analysis: 573,373.93 ms, 6.22 GB
[producer-api:28] (clinit): 3,160.25 ms, 6.22 GB
[producer-api:28] universe: 14,572.85 ms, 6.22 GB
[producer-api:28] (parse): 36,707.21 ms, 6.22 GB
[producer-api:28] (inline): 20,949.72 ms, 6.22 GB
[producer-api:28] (compile): 246,946.48 ms, 6.22 GB
[producer-api:28] compile: 312,422.13 ms, 6.22 GB
[producer-api:28] image: 19,407.59 ms, 6.22 GB
Error: Image build request failed with exit status 137
The command '/bin/sh -c native-image --no-server -cp build/libs/producer-api.jar' returned a non-zero code: 137
16 minutes and 43 seconds elapsed.
That is the producer-api
...
I have in build.gradle
...
dependencies {
...
implementation("io.micronaut.kafka:micronaut-kafka:SNAPSHOT")
...
}
...
java {
sourceCompatibility = JavaVersion.toVersion('11')
targetCompatibility = JavaVersion.toVersion('11')
}
In Dockerfile.native
#FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
# For JDK 11
FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
exit status exit status 137
indicates out of memory error. You need to allocate more memory to your Docker install
Note you will need a reference to the Kafka snapshot:
implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")
And make sure you change enforcedPlatform
to platform
for the micronaut-bom
Thanks @graemerocher . I fixed the wrong reference version to micronaut-kafka
that I had and gave 16GB (instead of 8GB) to docker.
I've just compiled the producer-api
. This time, the compilation finished successfully. It also starts ok.
However, once the 1st request comes, it's throwing the following exception
» docker run --rm --name micronaut-producer-api-native \
-p 9103:8080 -e KAFKA_HOST=kafka -e ZIPKIN_HOST=zipkin \
--network producer-consumer_default \
docker.mycompany.com/micronaut-producer-api-native:1.0.0
./producer-api: /usr/lib/libstdc++.so.6: no version information available (required by ./producer-api)
12:52:20.454 [main] INFO io.micronaut.runtime.Micronaut - Startup completed in 146ms. Server Running: http://2a6f74f8e1a2:8080
12:52:42.294 [nioEventLoopGroup-1-2] INFO c.m.producerapi.rest.NewsController - Sending News message: id=d09505e6-362a-46ae-b5f4-5297a1629664, News(id=d09505e6-362a-46ae-b5f4-5297a1629664, source=Micronaut Blog, title=Micronaut Framework & GraalVM)
12:52:42.300 [nioEventLoopGroup-1-2] INFO o.a.k.c.producer.ProducerConfig - ProducerConfig values:
acks = 1
batch.size = 16384
bootstrap.servers = [kafka:9092]
buffer.memory = 33554432
client.dns.lookup = default
client.id = producer-1
compression.type = none
connections.max.idle.ms = 540000
delivery.timeout.ms = 120000
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metadata.max.idle.ms = 300000
metric.reporters = [io.micronaut.configuration.kafka.metrics.ProducerKafkaMetricsReporter]
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 2147483647
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.2
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class io.micronaut.configuration.kafka.serde.JsonSerde
12:52:42.486 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Error while fetching metadata with correlation id 1 : {micronaut.news.json=LEADER_NOT_AVAILABLE}
12:52:42.486 [kafka-producer-network-thread | producer-1] INFO org.apache.kafka.clients.Metadata - [Producer clientId=producer-1] Cluster ID: Fn1_wLG2Qq6E1EBQX2d2Xg
12:52:42.622 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Error while fetching metadata with correlation id 3 : {micronaut.news.json=LEADER_NOT_AVAILABLE}
12:52:42.728 [kafka-producer-network-thread | producer-1] ERROR o.a.k.c.producer.internals.Sender - [Producer clientId=producer-1] Uncaught error in kafka producer I/O thread:
io.micronaut.core.reflect.exception.InstantiationException: Could not instantiate type [java.util.zip.CRC32C]: No class found for name: java.util.zip.CRC32C
at io.micronaut.core.reflect.InstantiationUtils.instantiate(InstantiationUtils.java:236)
at org.apache.kafka.common.utils.Crc32C$Java9ChecksumFactory.create(KafkaSubstitutions.java:52)
at org.apache.kafka.common.utils.Crc32C.create(Crc32C.java:77)
at org.apache.kafka.common.utils.Crc32C.compute(Crc32C.java:71)
at org.apache.kafka.common.record.DefaultRecordBatch.writeHeader(DefaultRecordBatch.java:479)
at org.apache.kafka.common.record.MemoryRecordsBuilder.writeDefaultBatchHeader(MemoryRecordsBuilder.java:366)
at org.apache.kafka.common.record.MemoryRecordsBuilder.close(MemoryRecordsBuilder.java:320)
at org.apache.kafka.clients.producer.internals.ProducerBatch.close(ProducerBatch.java:407)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.drainBatchesForOneNode(RecordAccumulator.java:619)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.drain(RecordAccumulator.java:646)
at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:357)
at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:323)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239)
at java.lang.Thread.run(Thread.java:834)
at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:527)
at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
Caused by: io.micronaut.core.reflect.exception.InstantiationException: No class found for name: java.util.zip.CRC32C
at io.micronaut.core.reflect.InstantiationUtils.lambda$instantiate$8(InstantiationUtils.java:234)
at java.util.Optional.orElseThrow(Optional.java:408)
at io.micronaut.core.reflect.InstantiationUtils.instantiate(InstantiationUtils.java:234)
... 15 common frames omitted
Ok interesting. Can probably be worked around by adding @TypeHint(java.util.zip.CRC32C.class)
I wonder why https://github.com/micronaut-projects/micronaut-kafka/blob/master/kafka/src/main/java/io/micronaut/configuration/kafka/graal/KafkaSubstitutions.java#L40 is not working
Stand by let me try a few things
@ivangfr I went through the steps to reproduce the issue with your sample and https://github.com/micronaut-projects/micronaut-kafka/commit/014b1f86b05c0a72ad707b7cd37b79cf936e6f93 seems to fix it
I had to apply the following patch to get it working in your example (which assumes using mavenLocal()
otherwise you will need to add the snapshot repo):
Subject: [PATCH] Use micronaut 2.0 rc1 and kafka snapshot
---
.../consumer-api/build.gradle | 25 ++++++++++++-------
.../src/main/docker/Dockerfile.native | 4 +--
.../producer-api/build.gradle | 25 ++++++++++++-------
.../src/main/docker/Dockerfile.native | 4 +--
4 files changed, 36 insertions(+), 22 deletions(-)
diff --git a/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle b/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
index afbf4a9..5358f2d 100644
--- a/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
+++ b/producer-consumer/micronaut-producer-consumer/consumer-api/build.gradle
@@ -10,6 +10,7 @@ group "com.mycompany.consumerapi"
repositories {
jcenter()
+ mavenLocal()
}
configurations {
@@ -18,11 +19,17 @@ configurations {
}
ext {
- set('micronautVersion', '2.0.0.M3')
+ set('micronautVersion', '2.0.0.RC1')
set('lombokVersion', '1.18.12')
set('logbackVersion', '1.2.3')
}
+configurations.all {
+ resolutionStrategy {
+ force "io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT"
+ }
+}
+
dependencies {
//-- IMPORTANT!
// Lombok annotation processor must place before the Micronaut processors in the build configuration
@@ -30,31 +37,31 @@ dependencies {
compileOnly("org.projectlombok:lombok:$lombokVersion")
annotationProcessor("org.projectlombok:lombok:$lombokVersion")
//--
- annotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ annotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
annotationProcessor("io.micronaut:micronaut-inject-java")
annotationProcessor("io.micronaut:micronaut-validation")
annotationProcessor("io.micronaut:micronaut-graal")
- compileOnly(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ compileOnly(platform("io.micronaut:micronaut-bom:$micronautVersion"))
compileOnly("org.graalvm.nativeimage:svm")
- implementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ implementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
implementation("io.micronaut:micronaut-management")
- implementation("io.micronaut.configuration:micronaut-micrometer-core")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-core")
implementation("io.opentracing.brave:brave-opentracing")
implementation("io.micronaut:micronaut-tracing")
- implementation("io.micronaut.configuration:micronaut-micrometer-registry-prometheus")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-registry-prometheus")
implementation("io.micronaut:micronaut-inject")
implementation("io.micronaut:micronaut-validation")
implementation("io.micronaut:micronaut-runtime")
implementation("io.micronaut:micronaut-http-server-netty")
implementation("io.micronaut:micronaut-http-client")
- implementation("io.micronaut.kafka:micronaut-kafka")
+ implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")
implementation("io.micronaut.cache:micronaut-cache-caffeine")
runtimeOnly("io.zipkin.brave:brave-instrumentation-http")
runtimeOnly("io.zipkin.reporter2:zipkin-reporter")
runtimeOnly("ch.qos.logback:logback-classic:$logbackVersion")
- testAnnotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testAnnotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testAnnotationProcessor("io.micronaut:micronaut-inject-java")
- testImplementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testImplementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testImplementation("org.junit.jupiter:junit-jupiter-api")
testImplementation("io.micronaut.test:micronaut-test-junit5")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
diff --git a/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native b/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
index 08c605b..83bb4d9 100644
--- a/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
+++ b/producer-consumer/micronaut-producer-consumer/consumer-api/src/main/docker/Dockerfile.native
@@ -1,6 +1,6 @@
-FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
+#FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
# For JDK 11
-#FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
+FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
RUN gu install native-image
COPY build/libs/consumer-api-*-all.jar /home/app/consumer-api/build/libs/consumer-api.jar
diff --git a/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle b/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
index fa86250..3d0bd23 100644
--- a/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
+++ b/producer-consumer/micronaut-producer-consumer/producer-api/build.gradle
@@ -10,6 +10,7 @@ group "com.mycompany.producerapi"
repositories {
jcenter()
+ mavenLocal()
}
configurations {
@@ -18,11 +19,17 @@ configurations {
}
ext {
- set('micronautVersion', '2.0.0.M3')
+ set('micronautVersion', '2.0.0.RC1')
set('lombokVersion', '1.18.12')
set('logbackVersion', '1.2.3')
}
+configurations.all {
+ resolutionStrategy {
+ force "io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT"
+ }
+}
+
dependencies {
//-- IMPORTANT!
// Lombok annotation processor must place before the Micronaut processors in the build configuration
@@ -30,31 +37,31 @@ dependencies {
compileOnly("org.projectlombok:lombok:$lombokVersion")
annotationProcessor("org.projectlombok:lombok:$lombokVersion")
//--
- annotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ annotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
annotationProcessor("io.micronaut:micronaut-inject-java")
annotationProcessor("io.micronaut:micronaut-validation")
annotationProcessor("io.micronaut:micronaut-graal")
- compileOnly(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ compileOnly(platform("io.micronaut:micronaut-bom:$micronautVersion"))
compileOnly("org.graalvm.nativeimage:svm")
- implementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ implementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
implementation("io.micronaut:micronaut-management")
- implementation("io.micronaut.configuration:micronaut-micrometer-core")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-core")
implementation("io.opentracing.brave:brave-opentracing")
implementation("io.micronaut:micronaut-tracing")
- implementation("io.micronaut.configuration:micronaut-micrometer-registry-prometheus")
+ implementation("io.micronaut.micrometer:micronaut-micrometer-registry-prometheus")
implementation("io.micronaut:micronaut-inject")
implementation("io.micronaut:micronaut-validation")
implementation("io.micronaut:micronaut-runtime")
implementation("io.micronaut:micronaut-http-server-netty")
implementation("io.micronaut:micronaut-http-client")
- implementation("io.micronaut.kafka:micronaut-kafka")
+ implementation("io.micronaut.kafka:micronaut-kafka:2.1.0.BUILD-SNAPSHOT")
implementation("io.micronaut.cache:micronaut-cache-caffeine")
runtimeOnly("io.zipkin.brave:brave-instrumentation-http")
runtimeOnly("io.zipkin.reporter2:zipkin-reporter")
runtimeOnly("ch.qos.logback:logback-classic:$logbackVersion")
- testAnnotationProcessor(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testAnnotationProcessor(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testAnnotationProcessor("io.micronaut:micronaut-inject-java")
- testImplementation(enforcedPlatform("io.micronaut:micronaut-bom:$micronautVersion"))
+ testImplementation(platform("io.micronaut:micronaut-bom:$micronautVersion"))
testImplementation("org.junit.jupiter:junit-jupiter-api")
testImplementation("io.micronaut.test:micronaut-test-junit5")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
diff --git a/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native b/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
index c05986d..16b43d7 100644
--- a/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
+++ b/producer-consumer/micronaut-producer-consumer/producer-api/src/main/docker/Dockerfile.native
@@ -1,6 +1,6 @@
-FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
+#FROM oracle/graalvm-ce:20.0.0-java8 as graalvm
# For JDK 11
-#FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
+FROM oracle/graalvm-ce:20.0.0-java11 as graalvm
RUN gu install native-image
COPY build/libs/producer-api-*-all.jar /home/app/producer-api/build/libs/producer-api.jar
--
2.24.3 (Apple Git-128)
Perfect @graemerocher ! I am looking forward to 2.0.0
final release.
Task List
Actual Behaviour
Hi, I've migrated my
graalvm-quarkus-micronaut-springboot
toJava 11
. All seems to work fine except for the Micronaut Producer-Consumer example.Before, both Producer and Consumer were using Java version
1.8
(inbuild.gradle
) andoracle/graalvm-ce:20.0.0-java8
as base image for the native images. This configuration was working fine.Then, I updated Producer and Consumer to Java version
11
andoracle/graalvm-ce:20.0.0-java11
as base image. I am able to build the application's native docker images but, at runtime, I have the errors below. It seems that the messages are been corrupted somehow.Logs
Producer
Consumer
Expected Behaviour
The message should be produced and consumed correctly as done using Java 8
Steps to Reproduce
Clone project
You will need Kafka & Zookeeper running. In thie README explains how to start them by running a
docker-compose.yml
file present ingraalvm-quarkus-micronaut-springboot/producer-consumer
folder;Go to Micronaut Procucer
build.gradle
and change the Java Version to 11Go to Micronaut Producer
Dockerfile.native
. Comment the Java8 line and uncomment the Java11Do the 2 steps described above, but for the Micronaut Consumer.
Environment Information
Example Application
graalvm-quarkus-micronaut-springboot