Global change in Dockerfiles overwrote some pertinent AWT related hints

[Karm]

Hello,

I noticed this large change in Docker files and their comments: https://github.com/quarkusio/quarkus-quickstarts/commit/1a61b2318404f6f8bcf0f510b2615d3d8863556e

I am not sure you meant it as it erases libfreetype deps etc. E.g. Docker file before the change: microdnf and now: nothing

Could you please share more context so as we can fix it, please?

[Karm]

Hello @gsmet, could you give me more context on the mass change?

[gsmet]

This was introduced in the release script by @Sgitario to keep the Dockerfiles in sync with the ones generated.

We missed the fact that this particular one was actually tuned.

@Sgitario could you have a look? Thanks!

[Sgitario]

This change was done to synchronize all the quickstarts with the latest Dockerfile files. I'm trying to see whether there is a way to install these dependencies that are required for the Quarkus AWT extension and at the same time to get the benefit of automatically updating the Dockerfile after every release. Otherwise, the only solution I see is to exclude this quickstart example from the script, but the Dockerfile will be outdated later than sooner.

[Sgitario]

@Karm I tried to reproduce this issue and it worked fine without RUN microdnf install freetype fontconfig && microdnf clean all.

See full stack:

[jcarvaja@localhost awt-graphics-rest-quickstart]$ mvn clean verify -Pnative
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------------< org.acme:awt-graphics-rest-quickstart >----------------
[INFO] Building awt-graphics-rest-quickstart 1.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ awt-graphics-rest-quickstart ---
[INFO] Deleting /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ awt-graphics-rest-quickstart ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 5 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ awt-graphics-rest-quickstart ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ awt-graphics-rest-quickstart ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 6 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ awt-graphics-rest-quickstart ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:3.0.0-M7:test (default-test) @ awt-graphics-rest-quickstart ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.acme.awt.rest.ImageResourceTest
2023-01-09 10:11:47,638 INFO  [io.quarkus] (main) awt-graphics-rest-quickstart 1.0.0-SNAPSHOT on JVM (powered by Quarkus 2.15.2.Final) started in 2.544s. Listening on: http://localhost:8081
2023-01-09 10:11:47,639 INFO  [io.quarkus] (main) Profile test activated. 
2023-01-09 10:11:47,640 INFO  [io.quarkus] (main) Installed features: [awt, cdi, resteasy, resteasy-multipart, smallrye-context-propagation, vertx]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.433 s - in org.acme.awt.rest.ImageResourceTest
2023-01-09 10:11:49,921 INFO  [io.quarkus] (main) awt-graphics-rest-quickstart stopped in 0.037s
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ awt-graphics-rest-quickstart ---
[INFO] Building jar: /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT.jar
[INFO] 
[INFO] --- quarkus-maven-plugin:2.15.2.Final:build (default) @ awt-graphics-rest-quickstart ---
[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building native image source jar: /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-native-image-source-jar/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-native-image-source-jar/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner.jar
[WARNING] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Cannot find the `native-image` in the GRAALVM_HOME, JAVA_HOME and System PATH. Install it using `gu install native-image` Attempting to fall back to container build.
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildContainerRunner] Using docker to run the native image builder
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildContainerRunner] Checking image status quay.io/quarkus/ubi-quarkus-mandrel-builder-image:22.3-java17
22.3-java17: Pulling from quarkus/ubi-quarkus-mandrel-builder-image
Digest: sha256:618b058c24533d73d555c0e3f30fbe42467aa4c25861fe636b24ac86c8bd250b
Status: Image is up to date for quay.io/quarkus/ubi-quarkus-mandrel-builder-image:22.3-java17
quay.io/quarkus/ubi-quarkus-mandrel-builder-image:22.3-java17
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on native-image 22.3.0.1-Final Mandrel Distribution (Java Version 17.0.5+8)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] docker run --env LANG=C --rm --user 1000:1000 -v /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-native-image-source-jar:/project:z --name build-native-Jkpob quay.io/quarkus/ubi-quarkus-mandrel-builder-image:22.3-java17 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dlogging.initial-configurator.min-level=500 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Duser.language=es -J-Duser.country=ES -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.ResourcesFeature,io.quarkus.runtime.graal.DisableLoggingFeature,io.quarkus.awt.runtime.graal.AwtFeature,io.quarkus.awt.runtime.graal.DarwinAwtFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:+CollectImageBuildStatistics -H:ImageBuildStatisticsFile=awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner-timing-stats.json -H:BuildOutputJSONFile=awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner-build-output-stats.json -H:+AllowFoldMethods -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+ReportExceptionStackTraces -H:-AddAllCharsets --enable-url-protocols=http -H:NativeLinkerOption=-no-pie -H:-UseServiceLoaderFeature -H:+StackTrace -J--add-exports=org.graalvm.sdk/org.graalvm.nativeimage.impl=ALL-UNNAMED -J--add-exports=org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner -jar awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner.jar
========================================================================================================================
GraalVM Native Image: Generating 'awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                    (7,7s @ 0,29GB)
 Version info: 'GraalVM 22.3.0.1-Final Java 17 Mandrel Distribution'
 Java version info: '17.0.5+8'
 C compiler: gcc (redhat, x86_64, 8.5.0)
 Garbage collector: Serial GC
 5 user-specific feature(s)
 - io.quarkus.awt.runtime.graal.AwtFeature
 - io.quarkus.awt.runtime.graal.DarwinAwtFeature
 - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
 - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.jboss.threads] categories
 - io.quarkus.runtime.graal.ResourcesFeature: Register each line in META-INF/quarkus-native-resources.txt as a resource on Substrate VM
[2/7] Performing analysis...  [******]                                                                  (84,1s @ 3,57GB)
  12.169 (87,87%) of 13.849 classes reachable
  20.258 (62,05%) of 32.647 fields reachable
  62.381 (56,81%) of 109.813 methods reachable
     541 classes,   123 fields, and 2.497 methods registered for reflection
     178 classes, 1.535 fields, and 2.092 methods registered for JNI access
       7 native libraries: dl, freetype, m, pthread, rt, stdc++, z
[3/7] Building universe...                                                                              (12,6s @ 2,09GB)
[4/7] Parsing methods...      [***]                                                                     (10,2s @ 2,13GB)
[5/7] Inlining methods...     [***]                                                                      (6,0s @ 4,27GB)
[6/7] Compiling methods...    [*********]                                                               (78,6s @ 2,44GB)
[7/7] Creating image...                                                                                  (8,0s @ 4,48GB)
  26,06MB (47,31%) for code area:    41.429 compilation units
  26,09MB (47,37%) for image heap:  323.694 objects and 19 resources
   2,93MB ( 5,32%) for other data
  55,09MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area:                               Top 10 object types in image heap:
   1,62MB sun.security.ssl                                     5,68MB byte[] for code metadata
   1,00MB java.util                                            3,05MB java.lang.String
 873,39KB com.oracle.svm.core.jni                              2,89MB java.lang.Class
 750,96KB sun.font                                             2,68MB byte[] for general heap data
 730,08KB java.lang.invoke                                     2,33MB byte[] for java.lang.String
 717,70KB com.sun.crypto.provider                              1,28MB byte[] for embedded resources
 487,39KB java.awt.image                                       1,02MB com.oracle.svm.core.hub.DynamicHubCompanion
 457,48KB java.lang                                          661,40KB byte[] for reflection metadata
 450,92KB com.sun.imageio.plugins.tiff                       632,48KB java.util.HashMap$Node
 450,58KB sun.security.x509                                  578,08KB java.lang.String[]
  18,33MB for 447 more packages                                5,10MB for 2693 more object types
------------------------------------------------------------------------------------------------------------------------
                        6,6s (3,0% of total time) in 32 GCs | Peak RSS: 6,21GB | CPU load: 6,21
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /project/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner (executable)
 /project/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner-build-output-stats.json (json)
 /project/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner-timing-stats.json (raw)
 /project/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner.build_artifacts.txt (txt)
========================================================================================================================
Finished generating 'awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner' in 3m 36s.
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] docker run --env LANG=C --rm --user 1000:1000 -v /home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-native-image-source-jar:/project:z --entrypoint /bin/bash quay.io/quarkus/ubi-quarkus-mandrel-builder-image:22.3-java17 -c objcopy --strip-debug awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner
[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 227000ms
[INFO] 
[INFO] --- maven-failsafe-plugin:3.0.0-M7:integration-test (default) @ awt-graphics-rest-quickstart ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.acme.awt.rest.ImageResourceIT
Executing "/home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/awt-graphics-rest-quickstart-1.0.0-SNAPSHOT-runner -Dquarkus.http.port=8081 -Dquarkus.http.ssl-port=8444 -Dtest.url=http://localhost:8081 -Dquarkus.log.file.path=/home/jcarvaja/sources/quarkus/quarkus-quickstarts/awt-graphics-rest-quickstart/target/quarkus.log -Dquarkus.log.file.enable=true"
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-01-09 10:15:41,149 INFO  [io.quarkus] (main) awt-graphics-rest-quickstart 1.0.0-SNAPSHOT native (powered by Quarkus 2.15.2.Final) started in 0.095s. Listening on: http://0.0.0.0:8081
2023-01-09 10:15:41,150 INFO  [io.quarkus] (main) Profile prod activated. 
2023-01-09 10:15:41,150 INFO  [io.quarkus] (main) Installed features: [awt, cdi, resteasy, resteasy-multipart, smallrye-context-propagation, vertx]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.074 s - in org.acme.awt.rest.ImageResourceIT
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-failsafe-plugin:3.0.0-M7:verify (default) @ awt-graphics-rest-quickstart ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:06 min
[INFO] Finished at: 2023-01-09T10:15:44+01:00
[INFO] ------------------------------------------------------------------------

[Karm]

@Sgitario We are talking runtime image here. Your aforementioned run did not use either Dockerfile.native or Dockerfile.native-micro.

[Sgitario]

@Sgitario We are talking runtime image here. Your aforementioned run did not use either Dockerfile.native or Dockerfile.native-micro.

Yes, I noticed that. Though, the README.md file seems to mean that those files are used when running the maven package (this should be addressed).

I think the simplest solution is to exclude this project from the release script that updates the Dockerfile resources.

[Karm]

@Sgitario The problem is both build time and runtime bound. You need those libraries at build time and then you need them at runtime, both as dynamically linked shared objects .so and their file structure e.g. /usr/share/fonts, /usr/share/fontconfig etc.

I am fine with excluding this quickstart from the Dockerfile automation. What concerns me is that it could have slipped in the first place. Is there any kind of GH action that chews through the quickstarts' Dockerfile.jvm, Dockerfile.legacy-jar, Dockerfile.native, Dockerfile.native-micro after the autogenerated update is done?

I would appreciate if the automation keeps bumping ubi-minimal and quarkus-micro-image version tags as time goes on, as long as there is some gate testing it...

Happy to hear your thoughts on the topic

[Sgitario]

I've already provided a pull request to exclude the awt quickstart from the Dockerfile update: https://github.com/quarkusio/quarkus-release/pull/19

[gsmet]

I restored the original Dockerfiles in main.