MicronautLambdaRuntime to support Micronaut function, too

[SchulteMarkus]

In my project, I am using Micronaut for processing SQS events. I have created an example project at https://github.com/SchulteMarkus/micronaut-creating-first-graal-app/tree/master/complete-sqs The project "complete-sqs" works fine, I can deploy build/libs/complete-sqs-0.1-all.jar to AWS Lambda using Java8 runtime and it works as intended.

I am interested in using this project not as a .jar in AWS Lambda Java8 runtime, but using GraalVM for building a binary and using this binary in AWS Lambda custom runtime.

Well, this works, too (awesome!). Maybe you want to have a look at https://github.com/SchulteMarkus/micronaut-creating-first-graal-app/blob/master/complete-sqs/Makefile for the used commands.

In my current setup, I am using a bootstrap file as given by AWS, my result is https://github.com/SchulteMarkus/micronaut-creating-first-graal-app/blob/master/complete-sqs/bootstrap

I have seen there is MicronautLambdaRuntime available. I want to update my project using MicronautLambdaRuntime instead of my ugly and big bootstrap file, you can see the necessary changes in https://github.com/SchulteMarkus/micronaut-creating-first-graal-app/pull/5/files

This approach does not work :-( I am able to upload my build/awsfunction.zip to AWS Lambda custom runtime, it is triggerad on a new SQS message, but does not do anything at all

12:16:28
+ ./micronaut-graal-sqs-function
12:16:28
WARNING: The sunec native library, required by the SunEC provider, could not be loaded. This library is usually shipped as part of the JDK and can be found under <JAVA_HOME>/jre/lib/<platform>/libsunec.so. It is loaded at run time via System.loadLibrary("sunec"), the first time services from SunEC are accessed. To use this provider's services the java.library.path system property needs to be set a
12:16:28
START RequestId: 99347906-9a8e-511d-9cdf-2f2cb028a8f5 Version: $LATEST
12:16:28
END RequestId: 99347906-9a8e-511d-9cdf-2f2cb028a8f5
12:16:28
REPORT RequestId: 99347906-9a8e-511d-9cdf-2f2cb028a8f5 Duration: 47.77 ms Billed Duration: 400 ms Memory Size: 512 MB Max Memory Used: 97 MB Init Duration: 296.89 ms

Even worse, deploying build/libs/complete-sqs-0.1-all.jar to AWS Lambda Java8 runtime stops working, too.

[graemerocher]

Currently the custom runtime requires the API proxy approach

[SchulteMarkus]

Yes, this is mentioned on https://micronaut-projects.github.io/micronaut-aws/latest/guide/#customRuntimes But maybe at some day in the future, MicronautLambdaRuntime supports Micronaut function, too :-)

[charlie-harvey]

This issue confuses me when I look at the code. The default Runtime created by the project generator looks like this:

public class BookLambdaRuntime extends AbstractMicronautLambdaRuntime<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent, BookRequest, Book> {

Which clearly translates the Body of an APIGatewayProxyReqeustEvent and hands your code a Book. Great.

But I changed my Runtime to look for SNSEvents instead. And I am not asking to translate the Message, I'll do it myself.

public class BookLambdaRuntime extends AbstractMicronautLambdaRuntime<SNSEvent, String, SNSEvent, String> {

Taking a look at the code in AbstractMicronautLambdaRuntime I can see that it should be detecting that the types are the same and it should just hand it over.

protected HandlerRequestType createHandlerRequest(RequestType request) throws JsonProcessingException, JsonMappingException {
    if (this.requestType == this.handlerRequestType) {
        return request;
    } else if (request instanceof APIGatewayProxyRequestEvent) {
        String content = ((APIGatewayProxyRequestEvent)request).getBody();
        return this.valueFromContent(content, this.handlerRequestType);
    } else {
        return null;
    }
}

But what happens is that I get an empty event. Not Null, somehow. Just empty. For the life of me I can't figure out why. What am I missing? It really looks like it should work the way it is written.

[alvarosanchez]

@charlie-harvey I think you are experiencing what is described at #493

[alvarosanchez]

@charlie-harvey actually SNSEvent should work: https://github.com/micronaut-projects/micronaut-aws/blob/master/function-aws-custom-runtime/src/main/java/io/micronaut/function/aws/runtime/AbstractMicronautLambdaRuntime.java#L93

If you share an example that reproduces the issue, we can investigate

[hchang-chwy]

@alvarosanchez I had similar issue as @charlie-harvey when using

public class BookLambdaRuntime extends AbstractMicronautLambdaRuntime<SQSEvent, String, SQSEvent, String> {

and I was getting Null SQSEvent at https://github.com/micronaut-projects/micronaut-aws/blob/master/function-aws-custom-runtime/src/main/java/io/micronaut/function/aws/runtime/AbstractMicronautLambdaRuntime.java#L331

The issue seems related to https://github.com/aws/aws-sdk-java/issues/1888 https://github.com/aws/aws-lambda-java-libs/issues/45

where SQSEvent class has field records while the actual SQS JSON has field Records the mismatch was causing ObjectMapper unable to deserialize the request into SQSEvent object and somewhere in DefaultHttpClient swallowed the parsing error without any additional logs. To solve the issue, AWS suggested making ObjectMapper accept case insensitive properties as below

ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true);

but doesn't look like there's a way to achieve the same in AbstractMicronautLambdaRuntime. Further more the inner classes(SQSMessage,MessageAttribute) of SQSEvent are not added to @TypeHint in AbstractMicronautLambdaRuntime and was causing

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of xxxxxxxx (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)

when running in graalvm.

I had to create my own SQSEvent annotated @Introspected as below to bypass the issue.

@Data
@Introspected
public class SQSEvent {

    @JsonProperty("Records")
    private List<SQSMessage> records;

    @Data
    @Introspected
    public static class SQSMessage{

....

    }

    @Data
    @Introspected
    public static class MessageAttribute{
...
    }
}

[geertvanheusden]

With the pointers from above and some additional trial and error, I was finally able to get SNSEvent to work in a native GraalVM image:

The ObjectMapper can be registered using a BeanCreatedEventListener (described on https://micronaut-projects.github.io/micronaut-aws/2.0.0.M1/guide/index.html#apiProxy if you look for ObjectMapper):

@Singleton
public class ObjectMapperBeanEventListener implements BeanCreatedEventListener<ObjectMapper> {

    @Override
    public ObjectMapper onCreated(BeanCreatedEvent<ObjectMapper> event) {
        final ObjectMapper mapper = event.getBean();
        mapper.configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true);
        mapper.registerModule(new JodaModule());
        return mapper;
    }
}

As you can see I also had to register the JodaModule to the ObjectMapper and add the required dependency (because this one is still not merged https://github.com/aws/aws-lambda-java-libs/pull/141/files):

<dependency>
  <groupId>com.fasterxml.jackson.datatype</groupId>
  <artifactId>jackson-datatype-joda</artifactId>
  <version>2.12.3</version>
  <scope>compile</scope>
</dependency>

The Introspected issue was resolved by using the following class instead of rewriting the existing ones:

@Introspected(classes = {SNSEvent.SNSRecord.class, SNSEvent.SNS.class, SNSEvent.MessageAttribute.class})
public class SNSEventConfiguration {
}

The last step was including the org.joda.time package to the micronaut-maven-plugin configuration:

<plugin>
  <groupId>io.micronaut.build</groupId>
  <artifactId>micronaut-maven-plugin</artifactId>
  <configuration>
    <nativeImageBuildArgs>
      --initialize-at-build-time=org.joda.time
    </nativeImageBuildArgs>
  </configuration>
</plugin>

That did the trick (at least for me ;-) )

[IgorEulalio]

Im having the same problem that @hchang-chwy as said.

When i receive the SQSEvent, the deserialize is ok, but the event stays null.

@Introspected
public class BookRequestHandler extends MicronautRequestHandler<SQSEvent, String> {

    @Override
    public String execute(SQSEvent input) {

        System.out.println("Input received:" + input);

        List<SQSEvent.SQSMessage> records = input.getRecords();

        if (records.isEmpty()) {
            System.out.println("No records received");
        }
        else {
            records.forEach(rcr -> {
                String body = rcr.getBody();
                System.out.println("BODY: " + body);
            });
        }

        return "Sucess";
    }
}

Follow the implementation off AbstractMicronautLambdaRuntime:

public class BookLambdaRuntime extends AbstractMicronautLambdaRuntime<SQSEvent, String, SQSEvent, String> {

    public static void main(String[] args) {
        try {
            new BookLambdaRuntime().run(args);

        } catch (MalformedURLException e) {
            e.printStackTrace();
        }
    }

    @Override
    @Nullable
    protected RequestHandler<SQSEvent, String> createRequestHandler(String... args) {
        return new BookRequestHandler();
    }
}

When the lambda is called, the print of input is:

START RequestId: 3d6b6e2c-ea8f-4c20-8134-60f95b73497f Version: $LATEST
Input received:{}

[jigneshkhatri]

I made micronaut function working in lambda and get triggered by the SQS by compiling above mentioned solutions and some other. Check my answer here - https://stackoverflow.com/a/73556660/3709922

[storytime]

I have the same issue.

Worked in localhost with native build, but not in AWS. Lambda has an admin role.

[charlie-harvey]

@geertvanheusden

The ObjectMapperListener and the SNSEventConfiguration was it! You don't need the JodaTime stuff anymore - that's finally been removed from the aws jar.

Thanks so much. Been waiting a long time for a resolution to this.

[storytime]

@geertvanheusden

The ObjectMapperListener and the SNSEventConfiguration was it! You don't need the JodaTime stuff anymore - that's finally been removed from the aws jar.

Thanks so much. Been waiting a long time for a resolution to this.

can you please provide more information