search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.58k stars 2.63k forks source link

How should I handle classes that shouldn't be loaded in native #40713

Open flynndi opened 4 months ago

flynndi commented 4 months ago

Description

I'm developing a Quarkus extension for an ORM framework. However, I'm encountering some issues when testing with native mode.

The error message I'm receiving is:

Error: Discovered unresolved type during parsing: org.postgresql.util.PGobject. This error is reported at image build time because class xxx.xxx.xxx.sql.dialect.PostgresDialect is registered for linking at image build time by command line and command line.
Error encountered while parsing xxx.xxx.xxx.sql.dialect.PostgresDialect.lambda$unknownReader$0(PostgresDialect.java:136) 
Parsing context:
   at static root method.(Unknown Source)

Detailed message:

com.oracle.svm.core.util.UserError$UserException: Discovered unresolved type during parsing: org.postgresql.util.PGobject. This error is reported at image build time because class xxx.xxx.xxx.sql.dialect.PostgresDialect is registered for linking at image build time by command line and command line.
Error encountered while parsing xxx.xxx.xxx.sql.dialect.PostgresDialect.lambda$unknownReader$0(PostgresDialect.java:136) 
Parsing context:
   at static root method.(Unknown Source)

Detailed message:

    at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.UserError.abort(UserError.java:85)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FallbackFeature.reportAsFallback(FallbackFeature.java:248)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:814)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:592)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
------------------------------------------------------------------------------------------------------------------------
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)

This seems to be related to the parsing process during the GraalVM build, particularly with the usage of PGobject in my codebase. However, I'm not sure how to resolve this issue.

Here's a brief overview of my setup:

I'm using Quarkus framework for my application. I have a class PostgresDialect which includes a method unknownReader that uses PGobject. I've tried various approaches such as replacing lambda expressions with anonymous classes and registering substitutions with GraalVM's Feature interface, but the issue persists.

Could anyone provide insights into why GraalVM is encountering this "Unresolved Type" issue and how to resolve it? Any help or suggestions would be greatly appreciated.

The following is the orm framework code

public class PostgresDialect extends DefaultDialect {

    @Override
    public Reader<?> unknownReader(Class<?> sqlType) {
        if (sqlType == PGobject.class) {
            return (rs, col) -> rs.getObject(col.col(), PGobject.class);
        }
        return null;
    }

}

Here's what I did with the quarkus extension

public class Substitutions {

    @TargetClass(className = "xxx.xxx.sql.dialect.PostgresDialect")
    public static final class PostgresDialectSubstitutions {

        // This code is the cause of the error
        @Substitute(polymorphicSignature = true)
        public Reader<?> unknownReader(Class<?> sqlType) {
            return null;
        }
    }
}

Even when my dialect is not Postgres, I still need to include the Postgres dependency.

Although I can include the Postgres dependency when not using Postgres to ensure the native package builds successfully, this doesn't seem like the correct approach

So, I referred to Quarkus's code, but it didn't work.

It seems that no matter how I replace the unknownReader method, GraalVM still tries to find PGobject.

Thank you in advance!

Repository name

quarkus-jimmer-extension

Short description

jimmer-extension

Repository Homepage URL

https://github.com/flynndi/quarkus-jimmer-extension

Repository Topics

Team Members

Additional context

No response

quarkus-bot[bot] commented 4 months ago

/cc @zakkak (native-image)

zakkak commented 4 months ago

Hi @flynndi, if I get this right you have already gone through https://quarkus.io/guides/native-reference#i-get-a-analysiserrorparsingerror-when-building-a-native-executable-due-to-an-unresolvedelementexception-what-can-i-do and decided to proceed with substitutions.

I am a bit confused with the obfuscated (xxx.xxx) package names in your issue descriptions while the code is available in https://github.com/flynndi/quarkus-jimmer-extension. Do you indeed see the failure with https://github.com/flynndi/quarkus-jimmer-extension or is it just another extension of yours?

IIRC what's happening here is that GraalVM tries to load the original code in order to substitute it which results in the observed error. So you would need to entirely avoid using org.babyfish.jimmer.sql.dialect.PostgresDialect (by substituting the methods reaching it).

Can you provide a reproducer?

flynndi commented 4 months ago

Hi @flynndi, if I get this right you have already gone through https://quarkus.io/guides/native-reference#i-get-a-analysiserrorparsingerror-when-building-a-native-executable-due-to-an-unresolvedelementexception-what-can-i-do and decided to proceed with substitutions.

I am a bit confused with the obfuscated (xxx.xxx) package names in your issue descriptions while the code is available in https://github.com/flynndi/quarkus-jimmer-extension. Do you indeed see the failure with https://github.com/flynndi/quarkus-jimmer-extension or is it just another extension of yours?

IIRC what's happening here is that GraalVM tries to load the original code in order to substitute it which results in the observed error. So you would need to entirely avoid using org.babyfish.jimmer.sql.dialect.PostgresDialect (by substituting the methods reaching it).

Can you provide a reproducer?

reproducer: https://github.com/flynndi/quarkus-jimmer-extension-example

https://github.com/flynndi/quarkus-jimmer-extension This is an extension I am developing. When I use this extension for integration testing in native mode, I need to include the PostgreSQL dependency even if I am not using PostgreSQL. I tried using @Substitute to replace the method, but the aforementioned problem occurred.

The relevant code is at: https://github.com/flynndi/quarkus-jimmer-extension/blob/main/runtime/src/main/java/io/quarkiverse/jimmer/runtime/graal/JimmerSubstitutions.java

Thank you for your help.

flynndi commented 4 months ago

Hi @flynndi, if I get this right you have already gone through https://quarkus.io/guides/native-reference#i-get-a-analysiserrorparsingerror-when-building-a-native-executable-due-to-an-unresolvedelementexception-what-can-i-do and decided to proceed with substitutions.

I am a bit confused with the obfuscated (xxx.xxx) package names in your issue descriptions while the code is available in https://github.com/flynndi/quarkus-jimmer-extension. Do you indeed see the failure with https://github.com/flynndi/quarkus-jimmer-extension or is it just another extension of yours?

IIRC what's happening here is that GraalVM tries to load the original code in order to substitute it which results in the observed error. So you would need to entirely avoid using org.babyfish.jimmer.sql.dialect.PostgresDialect (by substituting the methods reaching it).

Can you provide a reproducer?

Hi @zakkak, Is modifying the bytecode a feasible solution? I tried modifying the bytecode but received the same error.

zakkak commented 4 months ago

Hello @flynndi, I had a look at the reproducer and it looks like the class is being loaded because Quarkus registers the type Hierarchy of https://github.com/flynndi/quarkus-jimmer-extension-example/blob/main/src/main/kotlin/io/quarkiverse/jimmer/it/entity/UserRole.kt for reflection.

Is modifying the bytecode a feasible solution? I tried modifying the bytecode but received the same error.

How? I would expect if you modify the bytecode in the jar before passing it to native-image to work. But that's probably not the best approach as it seems too fragile.

flynndi commented 4 months ago

Hello @flynndi, I had a look at the reproducer and it looks like the class is being loaded because Quarkus registers the type Hierarchy of https://github.com/flynndi/quarkus-jimmer-extension-example/blob/main/src/main/kotlin/io/quarkiverse/jimmer/it/entity/UserRole.kt for reflection.

Is modifying the bytecode a feasible solution? I tried modifying the bytecode but received the same error.

How? I would expect if you modify the bytecode in the jar before passing it to native-image to work. But that's probably not the best approach as it seems too fragile.

Hello @zakkak , Is there any way to solve this problem?
I tried the following methods but did not achieve the desired results:

Using the @Substitute annotation to replace methods Implementing the org.graalvm.nativeimage.hosted.Feature interface Bytecode replacement Could you provide some ideas or suggestions?

zakkak commented 4 months ago

Hello @zakkak , Is there any way to solve this problem?

The most trivial way is to include the dependency. It will result in more code being packed in the native executable and might prevent some dead code elimination, but should work other than that.

I tried the following methods but did not achieve the desired results:

Using the @substitute annotation to replace methods

As you already found out unfortunately that won't work as the class is being loaded without the substitutions applied.

I wonder if this is something that could be resolved on the GraalVM side though, needs more thought and investigation...

Implementing the org.graalvm.nativeimage.hosted.Feature interface

I don't see how that could help here. What did you do in that implementation?

Bytecode replacement

It's still not clear to me how and at what stage you did the bytecode replacement. If you do it before the GraalVM loads the class it should work (but you need to find all the paths that might be reaching the class).

flynndi commented 4 months ago

Hello @zakkak , Is there any way to solve this problem?

The most trivial way is to include the dependency. It will result in more code being packed in the native executable and might prevent some dead code elimination, but should work other than that.

I tried the following methods but did not achieve the desired results: Using the @substitute annotation to replace methods

As you already found out unfortunately that won't work as the class is being loaded without the substitutions applied.

I wonder if this is something that could be resolved on the GraalVM side though, needs more thought and investigation...

Implementing the org.graalvm.nativeimage.hosted.Feature interface

I don't see how that could help here. What did you do in that implementation?

Bytecode replacement

It's still not clear to me how and at what stage you did the bytecode replacement. If you do it before the GraalVM loads the class it should work (but you need to find all the paths that might be reaching the class).

Regarding Bytecode replacement, This is just a simple experiment, just to see if it works, here is my code: https://github.com/flynndi/quarkus-jimmer-extension/blob/FeatureByteBuddy/runtime/src/main/java/io/quarkiverse/jimmer/runtime/graal/JimmerFeature.java

So for now, it can only include dependencies. Thank you very much for clarifying.