FasterXML / jackson-dataformats-text

Uber-project for (some) standard Jackson textual format backends: csv, properties, yaml (xml to be added in future)
Apache License 2.0
404 stars 148 forks source link

Could please add JsonAnyGetter and JsonAnySetter annotations support? #47

Open C-h-e-r-r-y opened 6 years ago

C-h-e-r-r-y commented 6 years ago

The idea

Original Jackson has \@JsonAnySetter annotation which allow you deserializae any unknow fields into Map. Why not add this possibility for unknow columns? The setter could be Collection<String> with rest of column values or for Jackson compatibility Map<String, Object> for nested types (I do not know whether you support nested types during deseriablization) or simple Map<String, String> with keys as column order number. (just as idea)

The implementation should write all column as they described in JsonPropertyOrder and all other put into annotation setter.

The code

I have tried to run the following:

package com.json.jackson;

import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;

import java.util.List;

public class JacksonCsvAnyGetter {
    public static void main (String[] args) throws Exception {
        String csv =    "myPackageName,lang,packageTag1,packageValue1\n"
                +       "myFileName,lang,FileTag1,FileValue1";

        CsvMapper mapper = new CsvMapper();
        CsvSchema schema = mapper.schemaFor(Dto.class); // schema from 'Dto' definition
        MappingIterator<Dto> it = mapper.readerFor(Dto.class).with(schema)
                .readValues(csv);
        while (it.hasNextValue()) {
            Dto value = it.nextValue();
            System.out.println(value);
        }
        List<Dto> all = it.readAll();
    }
}
package com.json.jackson;

import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;

import java.util.HashMap;
import java.util.Map;

@JsonPropertyOrder({"name", "lang"})
public class Dto {
    private String name;
    private String lang;
    private Map<String, Object> all = new HashMap<>();

    public String getName() {
        return name;
    }

    public void setName(final String name) {
        this.name = name;
    }

    public String getLang() {
        return lang;
    }

    public void setLang(final String lang) {
        this.lang = lang;
    }

    @JsonAnySetter
    public void set(String name, Object value) {
        all.put(name, value);
    }

    @Override
    public String toString() {
        return "Dto{" +
                "name='" + name + '\'' +
                ", lang='" + lang + '\'' +
                ", all=" + all +
                '}';
    }
}

pom.xml snippet

        <properties>
            <jackson.version>2.9.1</jackson.version>
        </properties>

        <dependency>
            <groupId>com.fasterxml.jackson</groupId>
            <artifactId>jackson-bom</artifactId>
            <version>${jackson.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-csv</artifactId>
            <version>${jackson.version}</version>
        </dependency>

The exception

Exception in thread "main" com.fasterxml.jackson.dataformat.csv.CsvMappingException: Too many entries: expected at most 2 (value #2 (11 chars) "packageTag1")
 at [Source: (StringReader); line: 1, column: 20]
    at com.fasterxml.jackson.dataformat.csv.CsvMappingException.from(CsvMappingException.java:23)
    at com.fasterxml.jackson.dataformat.csv.CsvParser._reportCsvMappingError(CsvParser.java:1223)
    at com.fasterxml.jackson.dataformat.csv.CsvParser._handleExtraColumn(CsvParser.java:978)
    at com.fasterxml.jackson.dataformat.csv.CsvParser._handleNextEntry(CsvParser.java:839)
    at com.fasterxml.jackson.dataformat.csv.CsvParser.nextFieldName(CsvParser.java:649)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
    at com.fasterxml.jackson.databind.MappingIterator.nextValue(MappingIterator.java:277)
    at com.json.jackson.JacksonCsvAnyGetter.main(JacksonCsvAnyGetter.java:20)
cowtowncoder commented 6 years ago

Ok, at high level I think this makes sense, and I thought there was an issue for it, but apparently not. It is something I have thought a bit about, although without coming to a conclusion.

Challenge here is that any-properties are really handled at databind level, without any knowledge of underlying format. So as-is, there is no way to really customize this for CSV use case. This means that:

  1. It is possible to read/write any properties for cases where POJO has fewer entries than CSV schema -- but names have to map
  2. There is no way to handle truly unknown columns.

So question would be that of how to map such extra data. Answer may have to vary between serialization, deserialization.

For deserialization one would either have to use synthetic names ("1", "2", templating, something), and that might work quite easily. Use of Collection would seem to make sense, but in practice could only support Collection<String>; and would be incompatible with existing logic of calling method once per extra property.

For serialization things might be easier, come to think of that... if there was a way to indicate special arrangement. Perhaps Jackson 3.0 could have capability introspection which would indicate alternate call for such extra data, for formats that do not have true name/value pairing at low level (CSV is positional and name binding is extra layer).

amr commented 5 years ago

Isn't this already supported? https://github.com/FasterXML/jackson-dataformat-csv/blob/master/src/test/java/com/fasterxml/jackson/dataformat/csv/deser/AnySetterTest.java

cowtowncoder commented 5 years ago

@amr Good point -- so it'd be due to issue 109 (from old repository) having been implemented.