DynamoDB Enhanced Client: Provide JSON Attribute Converter Out of the Box

[helloworldless]

It would be nice if the DynamoDB Enhanced Client provided a JSON AttributeConverter, something like the SDK v1's @DynamoDBTypeConvertedJson. This seems like such a common use case that it would make sense for the SDK to provide it rather than every consumer of the SDK who needs it having to implement it themselves.

I did take a look at implementing it, e.g. class JsonAttributeConverter<T> implements AttributeConverter<T>, but it's proving to be a challenge! I wondered how the SDK v2 was handling generic AttributeConverters internally, and I found class SetAttributeConverter<T extends Collection<?>> implements AttributeConverter<T> (link) which, to be honest, I'm still trying to wrap my head around 😄 . That being said, if you think the SetAttributeConverter<T> is a good template for how this proposed JsonAttributeConverter<T> would work, and you think this would be good for a first time contribution, I'd be happy to take a shot at it myself!

I suppose one major drawback to providing this out of the box is that I don't see a way for a consumer of the SDK to customize the ObjectMapper. Everywhere in the SDK, I just see this is a static field: private static final ObjectMapper MAPPER = new ObjectMapper();.

For now, as a workaround, I'm just using a non-parameterized, single-use AttributeConverter e.g. class MyCustomEntityJsonAttributeConverter implements AttributeConverter, the downside being that I need to create one for each custom entity that I want to be JSON converted.

[bigunyak]

I'd also vote for such functionality coming out of the box as probably being really widely used. Writing my own custom converters also solves it but one problem with it is that I'm not able to pass already existing ObjectMapper. Instead, I need to create a new ObjectMapper in every converter because converters only work with NoArgs constructor.

[0bx]

Coupling SDK with particular JSON implementation doesn't make much sense IMO as projects may use different JSON libraries (Jackson, Gson, e.t.c) to tackle that problem. Another drawback is exception handling. Users (not library) should decide what kind of exception should be thrown on JSON parsing/serializing error.

You can easily implement generic attribute converter with static ObjectMapper like this.

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import software.amazon.awssdk.enhanced.dynamodb.AttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.AttributeValueType;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.io.UncheckedIOException;

public class JacksonAttributeConverter<T> implements AttributeConverter<T> {

    private final Class<T> clazz;
    private static final ObjectMapper mapper = new ObjectMapper();

    static {
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true);
    }

    public JacksonAttributeConverter(Class<T> clazz) {
        this.clazz = clazz;
    }

    @Override
    public AttributeValue transformFrom(T input) {
        try {
            return AttributeValue
                    .builder()
                    .s(mapper.writeValueAsString(input))
                    .build();
        } catch (JsonProcessingException e) {
            throw new UncheckedIOException("Unable to serialize object", e);
        }
    }

    @Override
    public T transformTo(AttributeValue input) {
        try {
            return mapper.readValue(input.s(), this.clazz);
        } catch (JsonProcessingException e) {
            throw new UncheckedIOException("Unable to parse object", e);
        }
    }

    @Override
    public EnhancedType type() {
        return EnhancedType.of(this.clazz);
    }

    @Override
    public AttributeValueType attributeValueType() {
        return AttributeValueType.S;
    }
}

Then you create subclass per attribute type you want to convert

public class SnapshotConverter extends JacksonAttributeConverter<Snapshot> {

    public SnapshotConverter() {
        super(Snapshot.class);
    }
}

You can also modify JacksonAttributeConverter to take ObjectMapper as second argument of the constructor and fallback to static if one hasn't been provided

Finally, you can use it as annotation on your model:

    @Getter(onMethod = @__({
            @DynamoDbAttribute("snapshot"),
            @DynamoDbConvertedBy(SnapshotConverter.class)
    }))
    private Snapshot snapshot;

[evankozliner]

A default implementation for jackson users could save devs a considerable amount of time. mapper.readValue(<your-table>, <your-mapper-bean>) worked in the past and was remarkably handy. Cannot seem to make this work without default constructors from what I can tell, but maybe I am missing something?

[helenaperdigueirovg]

Coupling SDK with particular JSON implementation doesn't make much sense IMO as projects may use different JSON libraries (Jackson, Gson, e.t.c) to tackle that problem. Another drawback is exception handling. Users (not library) should decide what kind of exception should be thrown on JSON parsing/serializing error.

You can easily implement generic attribute converter with static ObjectMapper like this.

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import software.amazon.awssdk.enhanced.dynamodb.AttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.AttributeValueType;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.io.UncheckedIOException;

public class JacksonAttributeConverter<T> implements AttributeConverter<T> {

    private final Class<T> clazz;
    private static final ObjectMapper mapper = new ObjectMapper();

    static {
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true);
    }

    public JacksonAttributeConverter(Class<T> clazz) {
        this.clazz = clazz;
    }

    @Override
    public AttributeValue transformFrom(T input) {
        try {
            return AttributeValue
                    .builder()
                    .s(mapper.writeValueAsString(input))
                    .build();
        } catch (JsonProcessingException e) {
            throw new UncheckedIOException("Unable to serialize object", e);
        }
    }

    @Override
    public T transformTo(AttributeValue input) {
        try {
            return mapper.readValue(input.s(), this.clazz);
        } catch (JsonProcessingException e) {
            throw new UncheckedIOException("Unable to parse object", e);
        }
    }

    @Override
    public EnhancedType type() {
        return EnhancedType.of(this.clazz);
    }

    @Override
    public AttributeValueType attributeValueType() {
        return AttributeValueType.S;
    }
}

Then you create subclass per attribute type you want to convert

public class SnapshotConverter extends JacksonAttributeConverter<Snapshot> {

    public SnapshotConverter() {
        super(Snapshot.class);
    }
}

You can also modify JacksonAttributeConverter to take ObjectMapper as second argument of the constructor and fallback to static if one hasn't been provided

Finally, you can use it as annotation on your model:

    @Getter(onMethod = @__({
            @DynamoDbAttribute("snapshot"),
            @DynamoDbConvertedBy(SnapshotConverter.class)
    }))
    private Snapshot snapshot;

Thank you very much for this! It worked!

[pospears]

Any guidance on how to make work with generic object types? For example Snapshot<?> snapshot;
Currently get exception stating "Type variable type T is not supported."

[khayouge]

Coupling SDK with particular JSON implementation doesn't make much sense IMO as projects may use different JSON libraries (Jackson, Gson, e.t.c) to tackle that problem. Another drawback is exception handling. Users (not library) should decide what kind of exception should be thrown on JSON parsing/serializing error.

You can easily implement generic attribute converter with static ObjectMapper like this.

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import software.amazon.awssdk.enhanced.dynamodb.AttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.AttributeValueType;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.io.UncheckedIOException;

public class JacksonAttributeConverter<T> implements AttributeConverter<T> {

    private final Class<T> clazz;
    private static final ObjectMapper mapper = new ObjectMapper();

    static {
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true);
    }

    public JacksonAttributeConverter(Class<T> clazz) {
        this.clazz = clazz;
    }

    @Override
    public AttributeValue transformFrom(T input) {
        try {
            return AttributeValue
                    .builder()
                    .s(mapper.writeValueAsString(input))
                    .build();
        } catch (JsonProcessingException e) {
            throw new UncheckedIOException("Unable to serialize object", e);
        }
    }

    @Override
    public T transformTo(AttributeValue input) {
        try {
            return mapper.readValue(input.s(), this.clazz);
        } catch (JsonProcessingException e) {
            throw new UncheckedIOException("Unable to parse object", e);
        }
    }

    @Override
    public EnhancedType type() {
        return EnhancedType.of(this.clazz);
    }

    @Override
    public AttributeValueType attributeValueType() {
        return AttributeValueType.S;
    }
}

Then you create subclass per attribute type you want to convert

public class SnapshotConverter extends JacksonAttributeConverter<Snapshot> {

    public SnapshotConverter() {
        super(Snapshot.class);
    }
}

You can also modify JacksonAttributeConverter to take ObjectMapper as second argument of the constructor and fallback to static if one hasn't been provided

Finally, you can use it as annotation on your model:

    @Getter(onMethod = @__({
            @DynamoDbAttribute("snapshot"),
            @DynamoDbConvertedBy(SnapshotConverter.class)
    }))
    private Snapshot snapshot;

Is it possible to use or extract few properties from json? I mean I have lot of properties, but in the first queriyes I just need to two properties from from the json. Is that possible?

[erezweiss-b]

Hi, is there any update on when the feature will be implemented? also, I see poor performance on query table when using the attribute converter for JSON field compared to dynamodb mapper in version 1

[vidyarajsv]

We are encountering the same problem. Is there a solution or an alternative available for us to implement in Java2?