Adapt to `--strict-image-heap` being enabled by default

lread commented 5 months ago

Starting with GraalVM 22, --strict-image-heap is enabled by default.

Here's the error we now get if I run bb native-image-test using GraalVM 22.0.1:

Error: An object of type 'single_segment_example$dummy' was found in the image heap. This type, however, is marked for initialization at image run time for the following reason: classes are initialized at run time by default.
This is not allowed for correctness reasons: All objects that are stored in the image heap must be initialized at build time.

You now have two options to resolve this:

1) If it is intended that objects of type 'single_segment_example$dummy' are persisted in the image heap, add 

    '--initialize-at-build-time=single_segment_example$dummy'

to the native-image arguments. Note that initializing new types can store additional objects to the heap. It is advised to check the static fields of 'single_segment_example$dummy' to see if they are safe for build-time initialization,  and that they do not contain any sensitive data that should not become part of the image.

2) If these objects should not be stored in the image heap, you can use 

    '--trace-object-instantiation=single_segment_example$dummy'

to find classes that instantiate these objects. Once you found such a class, you can mark it explicitly for run time initialization with 

    '--initialize-at-run-time=<culprit>'

to prevent the instantiation of the object.

If you are seeing this message after upgrading to a new GraalVM release, this means that some objects ended up in the image heap without their type being marked with --initialize-at-build-time.
To fix this, include '--initialize-at-build-time=single_segment_example$dummy' in your configuration. If the classes do not originate from your code, it is advised to update all library or framework dependencies to the latest version before addressing this error.

The following detailed trace displays from which field in the code the object was reached.
Detailed message:
Trace: Object was reached by
  reading field clojure.lang.Var.root of constant 
    clojure.lang.Var@721ec3a5: #'single-segment-example/dummy
  reading static field hello_world.main$_main.const__0
    at <unknown-location>
  registered as read because: null

Congrats to the Graal team for the comprehensive error message!

So what's happening is that we skip registering the single-segment namespace for build-time initialization because it has no package, but with --strict-image-heap now enabled by default, because the single-segment namespace is not initialized at build time, we get the error above.

This breaks our native image test but seems like reasonable behaviour to me.

Options to fix our native-image-test:

Add in -H:-StrictImageHeap option to native-image to disable strict image heap
Add in the --initialize-at-build-time=single_segment_example$dummy option to native-image
Stop testing single-segment namespaces
Isolate testing single-segment namespaces and expect native-image to fail

I think I like option 2. It gives some hint to the dogged how they might actually init a single segment namespace while not encouraging turning off the now default strict image heap.

borkdude commented 4 months ago

Stop testing single-segment namespaces

I vote for that one

lread commented 4 months ago

@borkdude and I chatted. I shared that the --initialize-at-build-time for the single segment namespace worked in my local testing. He found that surprising because he did not have success with this kind of thing in the past:

If you compile that namespace, you'll see many more classes than just this one. previously I got errors when not explicitly mentioning them

I will:

[x] double-check my local testing to verify that it indeed works
[x] try an experiment with something more complicated, like the single segment namespace in digest.

lread commented 4 months ago

I experimented with digest by using its (now deprecated) single-segement digest namespace.

Setup

added org.clj-commons/digest {:mvn/version "1.4.100"} dep to test-hello-world/deps.edn
in test-hello-world/src/hello_word/main.clj
- added [digest] under to :require
- used it from -main like so: (println "md5 digest for clojure" (digest/md5 "clojure"))

Sanity test using JVM

cd test-hello-world
bb clean
bb build
java -jar target/hello-world.jar

Resulted in, as expected:

md5 digest for clojure 32c0d97f82a20e67c6d184620f6bd322
Hello world

Initial Run

From project root:

bb clean
bb native-image-test

Yielded native-image error reported in this issue, but instead for digest$digest.

With init at build time for digest$digest

Edited test-hello-world/bb.edn to include --initialize-at-build-time=digest$digest for native-image calls.

From project root:

bb clean
bb native-image-test

This run shows probably more of what @borkdude experienced. The native-image error matched the error reported in this issue, but instead for each of: digest$fn__193, digest$fn__191, digest$fn__189,digest$fn__187,digest$fn__179,digest$fn__177, and digest$fn__164.

Conclusion

Single-segment namespaces cannot be handled easily by via --initialize-at-build-time. Perhaps there is some way to initialize them properly with an agent? As single-segment namespaces are an anti-pattern in Clojure, I'm happy not to invest any time in finding a way to make them work in conjunction with graal-build-time and to continue to consider them out of scope and unsupported.

So option 3, Stop testing single-segment namespaces, it is!

lread commented 4 months ago

I think I'll leave our single-segment warning message in, it could be a helpful hint as to why the build has failed.

clj-easy / graal-build-time