Missing fields in CloudEvent deserialisation

[olitomlinson]

Raising this on behalf of a colleague who is on vacation

---

I have started using DAPR pub/sub and am creating a Subscription in a Springboot application, so I am using the JAVA sdk.

The subscriber receives the messages correctly, but I noticed that the CloudEvent class, used to deserialize the incoming event, is missing these fields tracid,traceparent,tracestate,topic,pubsubname.

I'm using these libraries

implementation 'io.dapr:dapr-sdk:1.5.0'
implementation 'io.dapr:dapr-sdk-actors:1.5.0'
implementation 'io.dapr:dapr-sdk-springboot:1.5.0'

I know that the last version is 1.6.0, but from the javadoc I saw that the CloudEvent class of that version miss the fields too.

cc @fvitolo

[yaron2]

cc @mukundansundar

[jakesmolka]

I have a similar issue. When creating a message in an app and then accessing it in another app I can't deserialize the CloudEvent as excepted.

Roughly illustrating it:

App A creating the message:

val data = DaprPayload(input, singletonMap("templateId", templateId)). // Custom data type with added metadata

DaprClientBuilder().build().use { client ->
    client.publishEvent(
        daprConfig.pubsub.name,
        daprConfig.pubsub.topics.openehrInput,
        data,
        singletonMap(io.dapr.client.domain.Metadata.TTL_IN_SECONDS, "\${dapr.pubsub.ttl}")
    ).toFuture().get()
}

App B accessing it fails:

jsonMapper().readValue(body, CloudEvent::class.java)

It throws and error like Method threw 'com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException' exception. with this additional message Unrecognized field "pubsubname" (class io.dapr.client.domain.CloudEvent), not marked as ignorable.

The CloudEvent looks like this (on the pubsub component and how it gets into App B):

{
    "data": {
        "metadata": {
            "templateId": "..."
        },
        "payload": "..."
    },
    "datacontenttype": "application/json",
    "id": "9694d729-1373-4f21-9bd2-1f05681e04f9",
    "pubsubname": "pubsub",
    "source": "client",
    "specversion": "1.0",
    "topic": "openehr-input",
    "traceid": "00-668be055cfb8f2954b67c90f5fab0013-f85bc3e91ca83947-01",
    "traceparent": "00-668be055cfb8f2954b67c90f5fab0013-f85bc3e91ca83947-01",
    "tracestate": "",
    "type": "com.dapr.event.sent"
}

[olitomlinson]

@jakesmolka yep, looks like pubsubname is missing from the CloudEvent model too

https://github.com/dapr/java-sdk/blob/master/sdk/src/main/java/io/dapr/client/domain/CloudEvent.java

[mukundansundar]

Based on https://github.com/dapr/components-contrib/blob/master/pubsub/envelope.go, the fields that can be in CloudEvent are not completely there in Java SDK model.

@artursouza @yaron2 Dapr has additional fields on top of the CloudEvents spect v1.0, for that I think publishing a proto file will be good. WDYT? Since this will be a moving target as and when new fields are included that are optional, mandating a proto definition would potentially help in having the model auto generated.

[yaron2]

Based on https://github.com/dapr/components-contrib/blob/master/pubsub/envelope.go, the fields that can be in CloudEvent are not completely there in Java SDK model.

@artursouza @yaron2 Dapr has additional fields on top of the CloudEvents spect v1.0, for that I think publishing a proto file will be good. WDYT? Since this will be a moving target as and when new fields are included that are optional, mandating a proto definition would potentially help in having the model auto generated.

Makes sense.

[artursouza]

Until this fix is in place, please, use your own CloudEvent class, maybe the official CloudEvent's SDK: https://cloudevents.github.io/sdk-java/json-jackson.html

[artursouza]

I have a similar issue. When creating a message in an app and then accessing it in another app I can't deserialize the CloudEvent as excepted.

Roughly illustrating it:

App A creating the message:

val data = DaprPayload(input, singletonMap("templateId", templateId)). // Custom data type with added metadata

DaprClientBuilder().build().use { client ->
    client.publishEvent(
        daprConfig.pubsub.name,
        daprConfig.pubsub.topics.openehrInput,
        data,
        singletonMap(io.dapr.client.domain.Metadata.TTL_IN_SECONDS, "\${dapr.pubsub.ttl}")
    ).toFuture().get()
}

App B accessing it fails:

jsonMapper().readValue(body, CloudEvent::class.java)

It throws and error like Method threw 'com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException' exception. with this additional message Unrecognized field "pubsubname" (class io.dapr.client.domain.CloudEvent), not marked as ignorable.

The CloudEvent looks like this (on the pubsub component and how it gets into App B):

{
    "data": {
        "metadata": {
            "templateId": "..."
        },
        "payload": "..."
    },
    "datacontenttype": "application/json",
    "id": "9694d729-1373-4f21-9bd2-1f05681e04f9",
    "pubsubname": "pubsub",
    "source": "client",
    "specversion": "1.0",
    "topic": "openehr-input",
    "traceid": "00-668be055cfb8f2954b67c90f5fab0013-f85bc3e91ca83947-01",
    "traceparent": "00-668be055cfb8f2954b67c90f5fab0013-f85bc3e91ca83947-01",
    "tracestate": "",
    "type": "com.dapr.event.sent"
}

The mapper is not configured to ignore unknown attributes: mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)

Now, the fix here is to serialize the "unknown" attributes into a dictionary since it is not possible to hardcode every possible attribute in cloud event since the spec allows it to be extensible.

[artursouza]

Users can now extend the CloudEvent class as per #773 before we support the dictionary fallback solution.

[artursouza]

Lowering priority since CloudEvent class can now be extended.