Livedoc generates malformed JSON

jpeffer commented 6 years ago

Below is an exception that is thrown by Jackson while attempting to generate the API documentation. Customer is a POJO with getter / setter methods and the following attribute declaration:

public class Customer
{
    private String id;
    private Map<EstablishmentAssociationType, Establishment> associatedEstablishments;

    ... (getters and setters)
}

EstablishmentAssociationType is an Enum and Establishment is another POJO.

Stacktrace:

Caused by: com.fasterxml.jackson.databind.JsonMappingException: Null key for a Map not allowed in JSON (use a converting NullKeySerializer?) (through reference chain: org.hildan.livedoc.core.model.doc.Livedoc["types"]->java.util.ArrayList[0]->org.hildan.livedoc.core.model.groups.Group["elements"]->java.util.ArrayList[10]->org.hildan.livedoc.core.model.doc.types.ApiTypeDoc["template"]->java.util.LinkedHashMap["customer"]->java.util.LinkedHashMap["associatedEstablishments"]->java.util.Collections$SingletonMap["null"])
    at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:284)
    at com.fasterxml.jackson.databind.SerializerProvider.mappingException(SerializerProvider.java:1251)
    at com.fasterxml.jackson.databind.SerializerProvider.reportMappingProblem(SerializerProvider.java:1145)
    at com.fasterxml.jackson.databind.ser.impl.FailingSerializer.serialize(FailingSerializer.java:35)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeOptionalFields(MapSerializer.java:777)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:635)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeOptionalFields(MapSerializer.java:778)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:635)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeOptionalFields(MapSerializer.java:778)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:635)
    at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
    at com.fasterxml.jackson.module.afterburner.ser.ObjectMethodPropertyWriter.serializeAsField(ObjectMethodPropertyWriter.java:87)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
    at com.fasterxml.jackson.module.afterburner.ser.ObjectMethodPropertyWriter.serializeAsField(ObjectMethodPropertyWriter.java:87)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
    at com.fasterxml.jackson.module.afterburner.ser.ObjectMethodPropertyWriter.serializeAsField(ObjectMethodPropertyWriter.java:87)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
    at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
    at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913)
    at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:267)
    ... 62 common frames omitted

While the error is thrown, JSON is still produced, but is malformed due to the unhandled exception that is thrown.

joffrey-bion commented 6 years ago

Hi @jpeffer,

Thanks a lot for opening this issue. I suspect this is due to the template provider generating an example Customer with a null key in the Map attribute. I'll look into it ASAP.

joffrey-bion commented 6 years ago

For enums, the TemplateProvider gets the first enum value as default, so if the type of the keys of the map is a non-empty enum, this problem should not occur. Does your enum have any value?

jpeffer commented 6 years ago

This issue occurs with a HashMap as well. The associatedEstablishments variable is not initialized, though I fail to see why it would be populated when defining the spec.

joffrey-bion commented 6 years ago

@jpeffer I'm not talking about the associatedEstablishments field, but the enum declaration of EstablishmentAssociationType. The only way I found to reproduce the issue was to define an empty enum, i.e. an enum class with no constants declared in it, like this:

public enum EmtpyEnum {
}

If that is the case for your current EstablishmentAssociationType enum, then you just need to add some values for it to work. The next version of livedoc will handle empty enums properly so that Jackson does not fail.

If that is not the case of your enum, then please let me know and I'll reopen this issue. Also, in this case, could you please provide a bit more information so that I can reproduce the issue?

jpeffer commented 6 years ago

Ahh I see, that makes more sense. Unfortunately, it is not empty though. Here is the definition:

public enum EstablishmentAssociationType
{

    BILL_TO("billTo"),
    INSTALL("install"),
    SOLD_TO("soldTo");

    private final String type;

    /**
     * type
     *
     * @param type
     */
    EstablishmentAssociationType(final String type)
    {
        this.type = type;
    }

    /**
     * getValue
     *
     * @return String
     */
    public String getValue()
    {
        return type;
    }

}

joffrey-bion commented 6 years ago

Mmh, strange then, I'm reopening the issue.

I need some more information to be able to reproduce:

do you have a custom ObjectMapper configuration? or is the error resulting from the default Spring mapper?
if it's the mapper from Spring, could you please give me the Spring version you are using?
what is the content of the Establishment class and potential other children?
do you have computed getters (without corresponding field) that are hidden in your "getters and setters" mention?

If your project is open-source, then a link to your repo would be even easier for both of us :)

Thanks a lot!

jpeffer commented 6 years ago

I have tried with a custom object mapper and with the default Spring mapper, both result in the same error.
4.3.3.RELEASE through the spring-framework-bom dependency
Will need to get back to you

jpeffer commented 6 years ago

Not sure if it matters, but here are the Jackson dependencies in the classpath as well for jackson.version 2.9.2


        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.module</groupId>
            <artifactId>jackson-module-afterburner</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.datatype</groupId>
            <artifactId>jackson-datatype-joda</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>org.msgpack</groupId>
            <artifactId>jackson-dataformat-msgpack</artifactId>
            <version>0.8.13</version>
        </dependency>

joffrey-bion commented 6 years ago

Thanks for providing this information.

I tried to reproduce with both:

Spring 4.3's ObjectMapper
jackson-databind v2.6.5 and v2.9.2's basic ObjectMapper (with pretty printer on)

I used various maps with key and value types being enums and custom POJOs in different combinations and still can't reproduce the problem.

In any case, my fix from 2 days ago should prevent null keys in maps from any kind of templates, so you should not be facing this issue anymore with livedoc 4.2.0+. Could you please give it a try? If this does not fix the issue, then please comment here and I'll re-open again.

jpeffer commented 6 years ago

Looks like the most recent version 4.2.0 now requires Java 8? We're unfortunately stuck on Java 7 for a few more months, so I will have to come back and retest then, unless you plan on supporting an EOL version (which I wouldn't blame anyone for not supporting).

joffrey-bion commented 6 years ago

Interesting, Livedoc has been compiled with JDK 8 from the start, so I don't understand why you're having problems now. FYI, the latest version is in fact 4.3.1. What might have happened is that 4.2.0 or one of the later versions was compiled with JDK9 instead, which would indeed be a problem since I intend compatibility with Java 8+.

What's the exact error "major.minor version" you are getting?

On a side note, I understand how you feel being stuck with Java 7: at work, I'm still stuck with Java 6. And I'm sorry but I indeed don't plan on supporting Java 7, since I made heavy use of Java 8 constructs all over the code. Thanks for understanding :)

jpeffer commented 6 years ago

Here's the error I was seeing:

Caused by: java.lang.UnsupportedClassVersionError: org/hildan/livedoc/core/annotations/Api has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0 (unable to load class org.hildan.livedoc.core.annotations.Api)

It appears locally I am actually using Java 8:

java -version java version "1.8.0_40-ea" Java(TM) SE Runtime Environment (build 1.8.0_40-ea-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.40-b16, mixed mode)

Our Maven target is 1.7, so I assume that is why things go poof. I will change my local setup so I can at least try and verify if the new version resolves the originally reported issue.

joffrey-bion commented 6 years ago

Thanks for your answer. Indeed, class file version 53.0 is Java 9. My mistake, I have been careless in building v4.2.0. The latest Livedoc version (4.3.2) should fix the issue.

I will change my local setup so I can at least try and verify if the new version resolves the originally reported issue.

Thanks a lot in advance for that! I'd be very grateful.

I'm sorry that you won't be able to use Livedoc in your Java 7.

jpeffer commented 6 years ago

Making progress it seems. 4.3.2 fixed the class file version issue. I am now seeing the following error:

TypeError: t.sections is undefined Stack trace: s@http://localhost:8080/livedoc/static/js/main.64939410.js:1:446610 beginWork@http://localhost:8080/livedoc/static/js/main.64939410.js:1:322093 a@http://localhost:8080/livedoc/static/js/main.64939410.js:1:334715 i@http://localhost:8080/livedoc/static/js/main.64939410.js:1:335036 s@http://localhost:8080/livedoc/static/js/main.64939410.js:1:335274 E@http://localhost:8080/livedoc/static/js/main.64939410.js:1:338818 _@http://localhost:8080/livedoc/static/js/main.64939410.js:1:338555 m@http://localhost:8080/livedoc/static/js/main.64939410.js:1:337683 f@http://localhost:8080/livedoc/static/js/main.64939410.js:1:337073 enqueueSetState@http://localhost:8080/livedoc/static/js/main.64939410.js:1:311598 a.prototype.setState@http://localhost:8080/livedoc/static/js/main.64939410.js:1:290867 t/o.handlePersistorState@http://localhost:8080/livedoc/static/js/main.64939410.js:1:406169 u@http://localhost:8080/livedoc/static/js/main.64939410.js:1:191682 u@http://localhost:8080/livedoc/static/js/main.64939410.js:1:1417819 a/</</<@http://localhost:8080/livedoc/static/js/main.64939410.js:1:175960

Along with:

Failed to register/update a ServiceWorker for scope ‘http://localhost:8080/livedoc/’: Load failed with status 404 for script ‘http://localhost:8080/livedoc/service-worker.js’.

The generated JSON appears to at least be valid now for the API. Should I open a new issue to track this separately?

No worries on Java 7. It's EOL and eventually you just have to make a decision to support newer versions to improve the codebase.

joffrey-bion commented 6 years ago

Great that the Java version problem is now solved.

Regarding the UI error, it might be that your local storage contains outdated data from the previous version. I will work on this issue in the context of https://github.com/joffrey-bion/livedoc/issues/89.

Cleaning your local storage should solve the issue for now. If not, please open a separate issue.

joffrey-bion / livedoc

Livedoc generates malformed JSON #79

Stacktrace: