2.7.1 skill invocation is not backward compatible

vgoldman commented 5 years ago

I'm submitting a...

[X] Regression (a behavior that used to work and stopped working in a new release)
[X] Bug report  
[ ] Performance issue
[ ] Feature request
[ ] Documentation issue or request
[ ] Other... Please describe:

Expected Behavior

When upgrading from 2.6.0 to 2.7.1 existing standard skill builder is supposed to work in a same way (as documented).

Current Behavior

After upgrade skill invocation breaks on LaunchRequest. New code makes use of "AbstractSkill.execute" method which tries to de-serialize request into "Request" and fails (probably because input object is a RequestEnvelope as it was in 2.6.0 and not Request). It would probably also fail once it gets to "invoke" part Response response = invoke(deserializedRequest.get(), context); since return type from invoke is a ResponseEnvelope and not Response.

com.fasterxml.jackson.datatype.jsr310.deser.JSR310DateTimeDeserializerBase.findFormatOverrides(Lcom/fasterxml/jackson/databind/DeserializationContext;Lcom/fasterxml/jackson/databind/BeanProperty;Ljava/lang/Class;)Lcom/fasterxml/jackson/annotation/JsonFormat$Value;: java.lang.NoSuchMethodError java.lang.NoSuchMethodError: com.fasterxml.jackson.datatype.jsr310.deser.JSR310DateTimeDeserializerBase.findFormatOverrides(Lcom/fasterxml/jackson/databind/DeserializationContext;Lcom/fasterxml/jackson/databind/BeanProperty;Ljava/lang/Class;)Lcom/fasterxml/jackson/annotation/JsonFormat$Value; at com.fasterxml.jackson.datatype.jsr310.deser.JSR310DateTimeDeserializerBase.createContextual(JSR310DateTimeDeserializerBase.java:34) at com.fasterxml.jackson.datatype.jsr310.deser.InstantDeserializer.createContextual(InstantDeserializer.java:235) at com.fasterxml.jackson.databind.DeserializationContext.handleSecondaryContextualization(DeserializationContext.java:669) at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:430) at com.fasterxml.jackson.databind.deser.std.StdDeserializer.findDeserializer(StdDeserializer.java:947) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:439) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:296) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:428) at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:179) at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:108) at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:93) at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:131) at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:518) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:107) at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:255) at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:145) at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3708) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2005) at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:2476) at com.amazon.ask.util.impl.JacksonJsonUnmarshaller.unmarshall(JacksonJsonUnmarshaller.java:48) at com.amazon.ask.impl.AbstractSkill.execute(AbstractSkill.java:44) at com.amazon.ask.AlexaSkill.execute(AlexaSkill.java:22) at com.amazon.ask.SkillStreamHandler.handleRequest(SkillStreamHandler.java:71)

Steps to Reproduce (for bugs)

Build a basic sample (with implementation for handlers) and try to invoke with 2.6.0 and 2.7.1.
2.6.0 works as expected, 2.7.1 fails with exception above.

public class MySkillStreamHandler extends SkillStreamHandler { public MySkillStreamHandler () { super(getSkill()); } private static Skill getSkill() { return Skills.standard() .addRequestHandlers(new LaunchRequestHandler(), new FallbackIntentHandler()).build(); } }

Possible Solution

// Not required, but suggest a fix/reason for the bug,
// or ideas how to implement the addition or change

Context

Your Environment

ASK Java SDK version used: 2.7.1
Operating System and version: Lambda

Java Info

Java version used for development: 1.8

breedloj commented 5 years ago

Hey,

Do you mind sharing your pom.xml so I can see if anything jumps out at me? This appears to be a classpath issue wherein the JSR310 module is not present. You're getting the error as soon as deserialization is triggered since the module is required to deserialize into Java 8 style date/time fields.

The new abstraction still takes a RequestEnvelope and returns a ResponseEnvelope. You may be looking here and thinking otherwise; this is basically just a binding that tells the SDK to look for the presence of this field in order to determine if the input JSON is actually a RequestEnvelope. All RequestEnvelopes contain a 'request' field at the top level.

I was not able to repro this issue when I upgraded our sample skills from 2.6.0 to 2.7.1. However, I am wondering if the introduction of the runtime layer (which now contains the Jackson dependencies) somehow messed up the transitive dependency resolution in your project. It's also odd that the other Jackson classes appear to be present (or else you'd be seeing an error earlier on). Another helpful thing you can do is run mvn dependency:tree -Dverbose in your project and paste the output here. That'll basically give me the entire dependency tree of your project.

Lastly, can you confirm the command you're using the produce the JAR for Lambda?

vgoldman commented 5 years ago

See below (can't produce a tree right now). I used Eclipse AWS Lambda Plugin to prepare and upload jar - not sure what it uses for build - either maven compiler or Eclipse Java builder.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.skilltest</groupId>
  <artifactId>mysampleskill</artifactId>
  <version>1.0.0</version>
  <packaging>jar</packaging>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.6.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
          <encoding>UTF-8</encoding>
          <forceJavacCompilerUse>true</forceJavacCompilerUse>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-bom</artifactId>
        <version>1.11.415</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
  <dependency>
      <groupId>com.amazon.alexa</groupId>
      <artifactId>ask-sdk</artifactId>
      <version>2.7.1</version>
    </dependency>
    <dependency>
  <groupId>com.amazon.alexa</groupId>
  <artifactId>ask-sdk-dynamodb-persistence-adapter</artifactId>
  <version>2.7.1</version>
</dependency>
   <dependency>
    <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk-ses</artifactId>
       <!-- version>1.11.400</version-->
  </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-lambda-java-log4j</artifactId>
        <version>1.0.0</version>
    </dependency> 
    <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-nop</artifactId>
    <version>1.7.25</version>
    </dependency> 
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>com.amazonaws</groupId>
      <artifactId>aws-lambda-java-events</artifactId>
      <version>1.3.0</version>
    </dependency>
    <dependency>
      <groupId>com.amazonaws</groupId>
      <artifactId>aws-lambda-java-core</artifactId>
      <version>1.1.0</version>
    </dependency>
  </dependencies>
</project>

breedloj commented 5 years ago

Instead of generating the JAR through Eclipse, can you instead try running this inside of your workspace?

mvn org.apache.maven.plugins:maven-assembly-plugin:2.6:assembly -DdescriptorId=jar-with-dependencies package

This will help us to narrow down the cause of the issue. I don't know that the Eclipse plugin pulls dependencies from Maven or through explicitly registered JARs.

vgoldman commented 5 years ago

Same result. See tree below. Looks like a couple of previous version jackson dependencies are pulled by ws-java-sdk-core. Could that be the issue? How can I work around it? On the same note - do you guys publish somewhere change log between ask-sdk releases? I could not find it anywhere. Would be really helpful.

[INFO] +- com.amazon.alexa:ask-sdk:jar:2.7.1:compile [INFO] | +- com.amazon.alexa:ask-sdk-core:jar:2.7.1:compile [INFO] | | +- com.amazon.alexa:ask-sdk-runtime:jar:2.7.1:compile [INFO] | | | +- com.fasterxml.jackson.core:jackson-core:jar:2.9.4:compile [INFO] | | | +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.9.4:compile [INFO] | | | - org.apache.logging.log4j:log4j-slf4j-impl:jar:2.8.2:compile [INFO] | | | - org.apache.logging.log4j:log4j-api:jar:2.8.2:compile [INFO] | | - com.amazon.alexa:ask-sdk-model:jar:1.7.0:compile [INFO] | | - com.fasterxml.jackson.core:jackson-annotations:jar:2.9.4:compile [INFO] | +- com.amazon.alexa:ask-sdk-lambda-support:jar:2.7.1:compile [INFO] | | - commons-io:commons-io:jar:2.6:compile [INFO] | +- com.amazon.alexa:ask-sdk-servlet-support:jar:2.7.1:compile [INFO] | | - commons-codec:commons-codec:jar:1.6:compile [INFO] | - com.amazon.alexa:ask-sdk-apache-client:jar:2.7.1:compile [INFO] | - org.apache.httpcomponents:httpclient:jar:4.5.5:compile [INFO] | - org.apache.httpcomponents:httpcore:jar:4.4.9:compile [INFO] +- com.amazon.alexa:ask-sdk-dynamodb-persistence-adapter:jar:2.7.1:compile [INFO] | - com.amazonaws:aws-java-sdk-dynamodb:jar:1.11.415:compile [INFO] +- com.amazonaws:aws-java-sdk-ses:jar:1.11.415:compile [INFO] | +- com.amazonaws:aws-java-sdk-core:jar:1.11.415:compile [INFO] | | +- commons-logging:commons-logging:jar:1.1.3:compile [INFO] | | +- software.amazon.ion:ion-java:jar:1.0.2:compile [INFO] | | +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.7.1:compile [INFO] | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.6.7:compile [INFO] | | - joda-time:joda-time:jar:2.8.1:compile [INFO] | - com.amazonaws:jmespath-java:jar:1.11.415:compile [INFO] +- com.amazonaws:aws-lambda-java-log4j:jar:1.0.0:compile [INFO] | - log4j:log4j:jar:1.2.17:compile [INFO] +- org.slf4j:slf4j-nop:jar:1.7.25:compile [INFO] | - org.slf4j:slf4j-api:jar:1.7.25:compile [INFO] +- junit:junit:jar:4.12:test [INFO] | - org.hamcrest:hamcrest-core:jar:1.3:test [INFO] +- com.amazonaws:aws-lambda-java-events:jar:1.3.0:compile [INFO] | +- com.amazonaws:aws-java-sdk-s3:jar:1.11.415:compile [INFO] | | - com.amazonaws:aws-java-sdk-kms:jar:1.11.415:compile [INFO] | +- com.amazonaws:aws-java-sdk-sns:jar:1.11.415:compile [INFO] | | - com.amazonaws:aws-java-sdk-sqs:jar:1.11.415:compile [INFO] | +- com.amazonaws:aws-java-sdk-cognitoidentity:jar:1.11.415:compile [INFO] | - com.amazonaws:aws-java-sdk-kinesis:jar:1.11.415:compile [INFO] - com.amazonaws:aws-lambda-java-core:jar:1.1.0:compile

javier-ochoa commented 5 years ago

i got similar problem when using "aws-java-sdk-s3" dependency.

In my case, the s3 package dependency was overwriting several jackson package versions and it end up making it incompatible on runtime, so alexa sdk will break when unmarshalling.

Try overriding the jackson versions on your pom file. Here is an example of how i fixed it on my pom file:

 <dependencies>
    <dependency>
      <groupId>com.amazon.alexa</groupId>
      <artifactId>ask-sdk</artifactId>
      <version>2.7.1</version>
    </dependency>

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-annotations</artifactId>
      <version>2.9.4</version>
    </dependency>

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.9.4</version>
    </dependency>

    <dependency>
      <groupId>com.fasterxml.jackson.dataformat</groupId>
      <artifactId>jackson-dataformat-cbor</artifactId>
      <version>2.9.4</version>
    </dependency>

    <dependency>
      <groupId>com.amazonaws</groupId>
      <artifactId>aws-java-sdk-s3</artifactId>
      <version>1.11.402</version>
    </dependency>

   ... other dependencies here

</dependencies>

vgoldman commented 5 years ago

Yep, that worked. Thank you @javier-ochoa

tianrenz commented 5 years ago

Hi all,

Closing this thread due to inactivity. Please feel free to reopen the issue if problem persists.

Regards

alexa / alexa-skills-kit-sdk-for-java