search

microsoft / botbuilder-java

The Microsoft Bot Framework provides what you need to build and connect intelligent bots that interact naturally wherever your users are talking, from text/sms to Skype, Slack, Office 365 mail and other popular services.
http://botframework.com
MIT License
180 stars 117 forks source link

com.microsoft.bot.schema.Serialization ObjectMapper is unable to serialize Java8 Date/Time with newest jackson. #1454

Open kreigiron opened 2 years ago

kreigiron commented 2 years ago

Is your feature request related to a problem? Please describe. Problem: newer Jackson libraries (e.g. v 2.12.x) are unable to deserialize Java 8 time and date objects because a JDK8 Module is not added in ObjectMapper at com.microsoft.bot.schema.Serialization, that ObjectMapper is tightly coupled and as a client of the library we're unable to add additional modules in order to avoid this issue,

On jackson 2.12 the serialization throws a

Specific warnings thrown by jackson libs <2.12

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by com.fasterxml.jackson.databind.util.ClassUtil (jar:file:/{ommited}!/BOOT-INF/lib/jackson-databind-2.11.3.jar!/) to field java.time.OffsetDateTime.offset
WARNING: Please consider reporting this to the maintainers of com.fasterxml.jackson.databind.util.ClassUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Describe the solution you'd like Register the modules into the object mapper with



objectMapper.registerModule(new Jdk8Module());
objectMapper.registerModule(new JavaTimeModule());
``
by either:
* Register these modules into com.microsoft.bot.schema.Serialization by the sdk
* exposing the objectmapper and allow sdk users to add the modules by their own by .

**Describe alternatives you've considered**
See above

**Additional context**
Tested with the following versions:
JDK: adoptopenddk 13 from public docker image adoptopenjdk/openjdk13:alpine-jre
Spring Version: 2.4.0
Jackson libraries: 2.11.3, 2.12.6. 2.13.1
bot-integration-spring && bot-dialogs: 4.14.1
Related stackoverflow: https://stackoverflow.com/questions/27952472/serialize-deserialize-java-8-java-time-with-jackson-json-mapper
tracyboehrer commented 2 years ago

@kreigiron My first thought is that the additional modules are registered by way of inclusion in the POM's. For date and time, this is specifically the jackson-datatype-jsr310 module.

Of note also is that we've only fully tested on Java8. Though I do believe I've run it on 11 without issue. That particular "illegal reflective access" is a 9+ warning.

Have you tried this with JDK 8?

kreigiron commented 2 years ago

@tracyboehrer actually the JDK8Module is part of jackson-datatype-jsr310 so yes, we need to include that artifact, but it seems that we may need to manually add the JDK8Module into the mapper otherwise it won't register (at least that did not work for me).

I tried with JDK8 but due to GC constraints we need to run on java 13+, my current workaround is to downgrade jackson when running on that jvm version, but if we try to use jackson provided within newest Spring boot (2.12+) it won't work.

Thanks.

tracyboehrer commented 2 years ago

@kreigiron Let us dig into it. With 8, those modules are just available after a call to objectMapper.findAndRegisterModules();.

TheBudgetMan commented 2 years ago

We have also run into this issue. Steps to reproduce:

We get the following exception

java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: Java 8 date/time type `java.time.OffsetDateTime` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (through reference chain: com.microsoft.bot.schema.Activity["timestamp"])
        at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:683) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094) ~[na:na]
        at com.microsoft.bot.dialogs.DialogSet.createContext(DialogSet.java:189) ~[bot-dialogs-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.dialogs.Dialog.run(Dialog.java:325) ~[bot-dialogs-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.sample.teamsauth.DialogBot.onMessageActivity(DialogBot.java:57) ~[classes!/:sample]
        at com.microsoft.bot.builder.ActivityHandler.onTurn(ActivityHandler.java:78) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.sample.teamsauth.DialogBot.onTurn(DialogBot.java:44) ~[classes!/:sample]
        at com.microsoft.bot.builder.MiddlewareSet.receiveActivityInternal(MiddlewareSet.java:99) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.MiddlewareSet.lambda$receiveActivityInternal$1(MiddlewareSet.java:110) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.BotFrameworkAdapter$TenantIdWorkaroundForTeamsMiddleware.onTurn(BotFrameworkAdapter.java:1430) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.MiddlewareSet.receiveActivityInternal(MiddlewareSet.java:109) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.MiddlewareSet.receiveActivityInternal(MiddlewareSet.java:74) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.MiddlewareSet.receiveActivityWithStatus(MiddlewareSet.java:67) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.BotAdapter.runPipeline(BotAdapter.java:206) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.BotFrameworkAdapter.lambda$processActivity$2(BotFrameworkAdapter.java:478) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235) ~[na:na]
        at com.microsoft.bot.builder.BotFrameworkAdapter.processActivity(BotFrameworkAdapter.java:476) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.builder.BotFrameworkAdapter.lambda$processActivity$1(BotFrameworkAdapter.java:433) ~[bot-builder-4.14.1.jar!/:4.14.1]
        at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1705) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692) ~[na:na]
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ~[na:na]
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[na:na]
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ~[na:na]
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) ~[na:na]
Caused by: java.lang.IllegalArgumentException: Java 8 date/time type `java.time.OffsetDateTime` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (through reference chain: com.microsoft.bot.schema.Activity["timestamp"])
        at com.fasterxml.jackson.databind.ObjectMapper.valueToTree(ObjectMapper.java:3312) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.microsoft.bot.schema.Serialization.safeGetAs(Serialization.java:69) ~[bot-schema-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.schema.Serialization.getAs(Serialization.java:49) ~[bot-schema-4.14.1.jar!/:4.14.1]
        at com.microsoft.bot.dialogs.ObjectPath.clone(ObjectPath.java:293) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at com.microsoft.bot.dialogs.ObjectPath.getNormalizedValue(ObjectPath.java:804) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at com.microsoft.bot.dialogs.ObjectPath.setObjectSegment(ObjectPath.java:721) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at com.microsoft.bot.dialogs.ObjectPath.setPathValue(ObjectPath.java:177) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at com.microsoft.bot.dialogs.ObjectPath.setPathValue(ObjectPath.java:131) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at com.microsoft.bot.dialogs.DialogContext.init(DialogContext.java:82) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at com.microsoft.bot.dialogs.DialogContext.<init>(DialogContext.java:43) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at com.microsoft.bot.dialogs.DialogSet.lambda$createContext$0(DialogSet.java:189) ~[bot-dialogs-4.15.0-SNAPSHOT.jar!/:4.15.0-SNAPSHOT]
        at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680) ~[na:na]
        ... 28 common frames omitted
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 date/time type `java.time.OffsetDateTime` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (through reference chain: com.microsoft.bot.schema.Activity["timestamp"])
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1276) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ser.impl.UnsupportedTypeSerializer.serialize(UnsupportedTypeSerializer.java:35) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:770) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3126) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        at com.fasterxml.jackson.databind.ObjectMapper.valueToTree(ObjectMapper.java:3307) ~[jackson-databind-2.12.3.jar!/:2.12.3]
        ... 39 common frames omitted

Downgrading jackson or java is not an option for us unfortunately.

Have you got any updates on this issue, or some suggested workarounds?

TheBudgetMan commented 2 years ago

@tracyboehrer we did a little experimentation trying to correct the issue. We made it go away by instantiating ObjectMapper like this:

JsonMapper.builder()
            .findAndAddModules()
            .addModule(new JavaTimeModule())
            .build();

This might be a superficial solution, as findAndAddModules should pick it up (it did not). We did it first in com.microsoft.bot.schema.Serialization(to solve the error above), but it turns out there are a lot of places where ObjectMapper is not instantiated like this, so other exceptions like this start appearing.

tracyboehrer commented 2 years ago

Thanks all. The warning initially reported happens when using Java 9+.

But the root problem is the inconsistent ObjectMapper initialization. Apparently, we neglected to resolve that. Without either the call to ObjectMapper.findAndRegisterModules, or explicitly adding the Java 8 Date/Time module, there would be problems with any payload that has a date/time property.

Or are you saying that findAndRegisterModules is also no longer working?

TheBudgetMan commented 2 years ago

Or are you saying that findAndRegisterModules is also no longer working?

It seems so. When we tried using only findAndAddModules, it did not work. Also, if I recall correctly, Serialization has the findAndRegisterModules call in it.

RitaFer commented 1 year ago

I'm with this error in my teams bot, i've no ideia about what to do for fix. Even those samples don't work in my server.