search

FasterXML / jackson-dataformats-binary

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile
Apache License 2.0
310 stars 133 forks source link

[avro] Can not serialize UUID to string #218

Closed lukas-krecan closed 3 years ago

lukas-krecan commented 4 years ago

Due to this change in Jackson Avro it's not possible to serialize UUID to String.

public class AvroTest {
    private final String schemaStr = "{\n" +
            "  \"name\": \"Bean\",\n" +
            "  \"type\": \"record\",\n" +
            "  \"namespace\": \"example\",\n" +
            "  \"fields\": [\n" +
            "    {\n" +
            "      \"name\": \"uuid\",\n" +
            "      \"type\": \"string\"\n" +
            "    }\n" +
            "  ]\n" +
            "}";

    @Test
    void testAvro() throws JsonProcessingException {
        AvroMapper mapper = new AvroMapper();
        Schema schema =new Schema.Parser().parse(schemaStr);
        mapper.writer(new AvroSchema(schema)).writeValueAsBytes(new Bean());
    }

    private static class Bean {
        public UUID getUUID() {
            return UUID.randomUUID();
        }
    }
}

fails with 

com.fasterxml.jackson.databind.JsonMappingException: class [B cannot be cast to class java.lang.CharSequence ([B and java.lang.CharSequence are in module java.base of loader 'bootstrap')

    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._wrapAsIOE(DefaultSerializerProvider.java:509)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:482)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
    at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1513)
    at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1215)
    at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1109)
    at AvroTest.testAvro(AvroTest.java:26)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
    at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
    at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:212)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:208)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1507)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1507)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
    at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
    at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
Caused by: java.lang.ClassCastException: class [B cannot be cast to class java.lang.CharSequence ([B and java.lang.CharSequence are in module java.base of loader 'bootstrap')
    at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:323)
    at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:315)
    at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:150)
    at com.fasterxml.jackson.dataformat.avro.ser.NonBSGenericDatumWriter.write(NonBSGenericDatumWriter.java:123)
    at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:206)
    at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:195)
    at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:130)
    at com.fasterxml.jackson.dataformat.avro.ser.NonBSGenericDatumWriter.write(NonBSGenericDatumWriter.java:123)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
    at com.fasterxml.jackson.dataformat.avro.ser.RootContext.complete(RootContext.java:122)
    at com.fasterxml.jackson.dataformat.avro.AvroGenerator._complete(AvroGenerator.java:621)
    at com.fasterxml.jackson.dataformat.avro.AvroGenerator.writeEndObject(AvroGenerator.java:410)
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:180)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
cowtowncoder commented 4 years ago

Does

@JsonFormat(shape = JsonFormat.Shape.STRING)
public UUID getUUID() { }

work?

Or conversely use of "config overrides" for all instances of java.util.UUID, with something like

mapper.configOverride(UUID.class)
    .setFormat(JsonFormat.Value.forShape(JsonFormat.Shape.STRING))
lukas-krecan commented 4 years ago

@cowtowncoder Setting shape does not help

cowtowncoder commented 4 years ago

@lukas-krecan Ok. Thank you for verifying. Yes, looking at UUIDSerializer, it currently only considers whether backend can write binary natively (which formerly Avro backend claimed it couldn't), and bases decision on that.

I'll create a ticket for jackson-databind to make UUIDSerializer respect JsonFormat.Shape overrides, to allow resolving this problem.

cowtowncoder commented 4 years ago

Created https://github.com/FasterXML/jackson-databind/issues/2815 for tracing fix for this issue.

In hindsight I should have made sure shape overrides were working when finishing 2.11.

cowtowncoder commented 3 years ago

So; fixed in 2.11.3 for jackson-databind. To force serialization as String with Avro, use either per-property annotation:

@JsonFormat(shape = JsonFormat.Shape.STRING)
UUID id;

or configure mapper with config override:

        ObjectMapper m = new AvroMapper();
        m.configOverride(UUID.class)
            .setFormat(JsonFormat.Value.forShape(JsonFormat.Shape.STRING));
lukas-krecan commented 3 years ago

Thanks a lot