GraalVM issue with ArrayList serialization

vsevel commented 3 years ago

Describe the bug

When trying to serialize an ArrayList in native, we get the following error:

2021-08-27 07:00:01,618 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-0) HTTP Request to /hello/serlist failed, error id: 4735770b-c272-499f-ad87-5b70a6a2ca6c-1: org.jboss.resteasy.spi.UnhandledException: java.io.InvalidClassException: java.util.ArrayList; no valid constructor
        at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:106)
        at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:372)
        at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:218)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:519)
        at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:261)
        at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:161)
        at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
        at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:164)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:247)
        at io.quarkus.resteasy.runtime.standalone.RequestDispatcher.service(RequestDispatcher.java:73)
        at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.dispatch(VertxRequestHandler.java:138)
        at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler$1.run(VertxRequestHandler.java:93)
        at io.quarkus.vertx.core.runtime.VertxCoreRecorder$13.runWith(VertxCoreRecorder.java:536)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
        at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
        at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:829)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:553)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
Caused by: java.io.InvalidClassException: java.util.ArrayList; no valid constructor
        at java.io.ObjectStreamClass$ExceptionInfo.newInvalidClassException(ObjectStreamClass.java:159)
        at java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:875)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2170)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:493)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:451)
        at org.acme.GreetingResource.serlist(GreetingResource.java:69)
        at java.lang.reflect.Method.invoke(Method.java:566)
        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:170)
        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
        at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:660)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:524)
        at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:474)
        at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:476)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:434)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:408)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:69)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:492)
        ... 17 more

Expected behavior

The list should be able to be serialized and un-serialized.

Actual behavior

It fails with the above exception.

How to Reproduce?

Reproducer:

Create a sample JAXRS application
Add the following code:

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    @Path("/serstring")
    public String serstring() throws IOException, ClassNotFoundException {

        byte[] bytes = null;

        try (
                ByteArrayOutputStream baos = new ByteArrayOutputStream();
                ObjectOutputStream oos = new ObjectOutputStream(baos)) {

            oos.writeObject("Hello RESTEasy");
            oos.flush();
            bytes = baos.toByteArray();
        }

        try (ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
                ObjectInputStream ois = new ObjectInputStream(bais)) {
            return (String) ois.readObject();
        }
    }

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    @Path("/serlist")
    public String serlist() throws IOException, ClassNotFoundException {

        byte[] bytes = null;

        try (
                ByteArrayOutputStream baos = new ByteArrayOutputStream();
                ObjectOutputStream oos = new ObjectOutputStream(baos)) {

            List<String> list = new ArrayList<>();
            list.add("Hello RESTEasy");

            oos.writeObject(list);
            oos.flush();
            bytes = baos.toByteArray();
        }

        try (ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
                ObjectInputStream ois = new ObjectInputStream(bais)) {

            return ois.readObject().toString();
        }
    }

do a native build
start the server
execute curl http://localhost:8080/hello/serstring => Hello RESTEasy
execute curl http://localhost:8080/hello/serlist => exception

Output of `uname -a` or `ver`

windows

Output of `java -version`

openjdk 11.0.1-redhat 2018-10-16 LTS

GraalVM version (if different from Java)

Running Quarkus native-image plugin on GraalVM 21.1.0 Java 11 CE (Java Version 11.0.11+8-jvmci-21.1-b05)

Quarkus version or git rev

2.1.4.Final

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)

Additional information

<quarkus.native.container-build>true</quarkus.native.container-build>

geoand commented 3 years ago

You can likely get around the problem, by using @RegisterForReflection(targets=ArrayList.class). @zakkak shouldn't GraalVM be able to handle serialization out of the box?

vsevel commented 3 years ago

I forgot to mention that I had already:

@RegisterForReflection(serialization = true, targets = { ArrayList.class, String.class })
public class ReflectionConfig {
}

vsevel commented 3 years ago

hello @zakkak what is the proper way to progress on this issue? should I create an issue directly on the graalvm project?

jaikiran commented 3 years ago

I think this is a bug in Quarkus. It looks like when serialization is set to true on the RegisterForReflection annotation, the RegisterForReflectionBuildStep only registers it for serialization and explicitly disables reflection registration. Specifically, this line https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/deployment/steps/RegisterForReflectionBuildStep.java#L68. Notice that if serialization is true, it will call ReflectiveClassBuildItem.serializationClass(className) which looks like:

public static ReflectiveClassBuildItem serializationClass(String... className) {
        return new ReflectiveClassBuildItem(false, false, false, false, false, true, className);
    }

zakkak commented 3 years ago

Hi @vsevel sorry for the delay, I was on PTO. I 'll have a look at it (and #19942) this week and come back to you.

vsevel commented 3 years ago

HI @zakkak np. hope you had some good time. note that @matthyx tested a pure java native serialization example, which works fine: https://github.com/matthyx/graalvm-serialization (with no reflection or serialization specific configuration). it appears that something is getting the array list constructors to get dropped. hope it rings some bells.

zakkak commented 3 years ago

(with no reflection or serialization specific configuration)

FWIW that's not entirely true. https://github.com/matthyx/graalvm-serialization/blob/f0941a80c8fbca8c81a24c0b81799258a2223535/Dockerfile#L5 essentially generates the necessary configuration.

$ /opt/jvms/graalvm-ce-java11-21.2.0/bin/java -agentlib:native-image-agent=config-output-dir=META-INF/native-image HelloWorld
[toto, tutu, titi]
$ more META-INF/native-image/serialization-config.json
[
  {
  "name":"java.util.ImmutableCollections$ListN"
  },
  {
  "name":"java.lang.String"
  },
  {
  "name":"java.util.CollSer"
  }
]

Without this the generated binary fails with:

$ rm -rf META-INF
$ /opt/jvms/graalvm-ce-java11-21.2.0/bin/native-image --no-fallback HelloWorld
...
$ ./helloworld 
Exception in thread "main" java.lang.ClassNotFoundException: java.util.ImmutableCollections$ListN
    at com.oracle.svm.core.hub.ClassForNameSupport.forName(ClassForNameSupport.java:71)
    at java.lang.Class.forName(DynamicHub.java:1319)
    at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:756)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1995)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1862)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2169)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:493)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:451)
    at HelloWorld.main(HelloWorld.java:19)

matthyx commented 3 years ago

FWIW that's not entirely true.

Yes, I never said there wasn't one... but at least that's not something I had to write (or figure out).

vsevel commented 3 years ago

I never said there wasn't one

Well I did ;) thanks for the precision @zakkak

quarkusio / quarkus