FST fail to deserialize Java records

[ykotikov]

Can FST properly do deserialization of Java Records?

I'm using the latest version fst available and trying to emulate serialization/deserialization process Serialization works fine, but when I'm trying to deserialize the data I'm getting the following error:

Caused by: java.lang.IllegalAccessException: Can not set final int field Test.value to (int)1
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwFinalFieldIllegalAccessException(UnsafeFieldAccessorImpl.java:76)
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwFinalFieldIllegalAccessException(UnsafeFieldAccessorImpl.java:100)
    at java.base/jdk.internal.reflect.UnsafeQualifiedIntegerFieldAccessorImpl.setInt(UnsafeQualifiedIntegerFieldAccessorImpl.java:132)
    at java.base/java.lang.reflect.Field.setInt(Field.java:984)
    at org.nustaq.serialization.FSTClazzInfo$FSTFieldInfo.setIntValue(FSTClazzInfo.java:942)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:698)
    ... 7 more

Code Example

import org.nustaq.serialization.FSTConfiguration;

public class Main {

    private static final FSTConfiguration fstConfig = FSTConfiguration.createUnsafeBinaryConfiguration();

    static {
        // treat unserializable classes same as if they would be serializable.
        fstConfig.setForceSerializable(true);
    }

    // modules added as open ones via jvm args
    // --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.math=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.sql/java.sql=ALL-UNNAMED
    public static void main(String[] args) {

        var test = new Test("test", 1);

        // bi-directional check
        byte[] serialized = fstConfig.asByteArray(test);
        var res = (Test) fstConfig.asObject(serialized);

        System.out.println(res);
    }

    public record Test(
        String type,
        int value
    ) {
    }
}

Java version: bellsoft/liberica-openjdk-debian:17.0.2 FST version: de.ruedigermoeller:fst:3.0.4-jdk17

[istinnstudio]

...deleted

[AlexisDrogoul]

Hi, same remark and request here. We are thinking about using FST in a rather large project (https://github.com/gama-platform/gama) for both binary (saving large simulations) and JSON (inter-simulations communication) serialisation protocols, but the fact that it can't handle the deserialisation of records is really blocking. Any hint on when/how this can be addressed ?

[AlexisDrogoul]

A workaround consists in declaring a custom serialiser for the record classes. However, it defeats a bit the purpose of the library (zero effort configuration !).

[anttikerola]

I am succesfully running FST on Java 17 on OpenJDK. Also java records appear to be working. This was done with some additional reflection in the FSTUtil in order to get the record field offsets. Then the FST does not fall back to the reflection based setters and the IllegalAccessException is not thrown. I also tried my changes on Java 21 and it worked. I do need to use some --add-opens arguments when I start the jvm. So it is not pretty but it works.

I could provide a patch if anyone with maintainer privileges is interested.

[AlexisDrogoul]

Hi,

I don't have maintainer privileges, but I'm certainly interested in your patch. Do you think you could make it available as a pull request ?

Cheers Alexis

[renoth]

I am succesfully running FST on Java 17 on OpenJDK. Also java records appear to be working. This was done with some additional reflection in the FSTUtil in order to get the record field offsets. Then the FST does not fall back to the reflection based setters and the IllegalAccessException is not thrown. I also tried my changes on Java 21 and it worked. I do need to use some --add-opens arguments when I start the jvm. So it is not pretty but it works.

I could provide a patch if anyone with maintainer privileges is interested.

Could you provide a patch / PR regardless., so we can see your implementation?

[4shael]

I am succesfully running FST on Java 17 on OpenJDK. Also java records appear to be working. This was done with some additional reflection in the FSTUtil in order to get the record field offsets. Then the FST does not fall back to the reflection based setters and the IllegalAccessException is not thrown. I also tried my changes on Java 21 and it worked. I do need to use some --add-opens arguments when I start the jvm. So it is not pretty but it works.

I could provide a patch if anyone with maintainer privileges is interested.

@anttikerola so did you provide it somewhere?

[istinnstudio]

I do not use records but a patch like this one will give the project the state of "liveliness" even if no-one will merge it to main. It can be a fork and then a pull request.