GraalVM Native Image compatibility

[shomub]

Type: Feature

Is your feature request related to a problem? Please describe. spring-cloud-aws support for spring-boot 3 running on spring native. See attached reproducer to see issue on running the bootBuildImage Task

Describe the solution you'd like spring-clould-aws support for spring boot 3 on spring native

[maciejwalkowiak]

@shomub i think you missed the attachment

[shomub]

complete.zip uploaded file

[maciejwalkowiak]

Version 2.x will not be compatible with Spring Native. 3.0 on the other hand will since it is using GraalVM ready AWS SDK v2. I will take a look what's the current state of native support for 3.x codebase.

[shomub]

Thank you Maciej!

On Wed, Jul 27, 2022, 10:52 AM Maciej Walkowiak @.***> wrote:

Version 2.x will not be compatible with Spring Native. 3.0 on the other hand will since it is using GraalVM ready AWS SDK v2. I will take a look what's the current state of native support for 3.x codebase.

— Reply to this email directly, view it on GitHub https://github.com/awspring/spring-cloud-aws/issues/468#issuecomment-1196864303, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIANQTI5SVZI44VJRKRU2ZDVWFELNANCNFSM54ZR6NJA . You are receiving this because you were mentioned.Message ID: @.***>

[maciejwalkowiak]

DynamoDB Enhanced Client BeanTableSchema is not compatible with GraalVM Native Image (https://github.com/aws/aws-sdk-java-v2/issues/2445). Either this issue has to be fixed, or we need to advice users to use StaticTableSchema in docs for building native images.

[davidmelia]

Hi, related to this I noticed the AWS parameter store integration (3.0.0-M3) is stripped out when doing a native build such that it is invoked during the native build step but then never invoked when running that executable. This should work similar to the spring cloud config client.

[jamfor352]

I've also noticed my @SqsListener methods not being set up properly as well when running via GraalVM on a Spring Boot 3.0/AOT app. Can tell this as apparently the SqsListener isn't registered when I look up the DefaultContainerRegistry - apparently no listeners exist. Is this known about?

[tomazfernandes]

I've also noticed my @SqsListener methods not being set up properly as well when running via GraalVM on a Spring Boot 3.0/AOT app. Can tell this as apparently the SqsListener isn't registered when I look up the DefaultContainerRegistry - apparently no listeners exist. Is this known about?

I haven't had the chance to look into native yet, so I can't tell exactly what's going on.

What we do in SQS is we look for the @SqsListener annotations on the beans here. I'd guess this introspection method we use isn't supported OOTB.

Looking at the Spring Kafka project which uses a similar approach, I see they have this hints file, so we will probably need something similar.

[jamfor352]

I've also noticed my @SqsListener methods not being set up properly as well when running via GraalVM on a Spring Boot 3.0/AOT app. Can tell this as apparently the SqsListener isn't registered when I look up the DefaultContainerRegistry - apparently no listeners exist. Is this known about?

I haven't had the chance to look into native yet, so I can't tell exactly what's going on.

What we do in SQS is we look for the @SqsListener annotations on the beans here. I'd guess this introspection method we use isn't supported OOTB.

Looking at the Spring Kafka project which uses a similar approach, I see they have this hints file, so we will probably need something similar.

I was actually just playing around with this and I got it to work (in a horribly hacky way) by adding into my reflect-config.json file all the classes that the native-image-agent picked up under the io.awspring package (and setting them to have full reflection capability), eg:

    "name": "io.awspring.cloud.sqs.annotation.SqsListener",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredFields": true,
    "allPublicFields": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true
  }

This could probably be far more granular on a class-by-class basis, but this worked for me (I also added equivalent entries for my own project's classes)

[MatejNedic]

Hey @jamfor352 , sorry for late response. Could you make a demo and reproduce it? I am writing fine grained hints for SQS integration and I cannot reproduce an issue. Only thing I need to add is reflection hints for POJO I want to serialise and Class that uses SqsListener

[mmuniz75]

Spring Boot 3.2 Spring Cloud 2021.0.6

I have this simple projetct with AWS Parameter store value to be load in a rest end point :

curl --location 'http://localhost:8080/param'

It works fine in JVM version, it compiles as well with nativeCompile but the native code not starts.

please set the following credentials to execute the server : export AWS_ACCESS_KEY_ID=AKIAXMDC5B4B3IYMKFE6 export AWS_SECRET_ACCESS_KEY=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

This a temp user to access AWS Parameter store

parameter-store-native.zip

[mmuniz75]

Using Spring Cloud 2023.0.0 and spring-cloud-aws 3.1 now when I starts the compiled version I have :

---

APPLICATION FAILED TO START

---

Description:

Config data resource '[SecretsManagerConfigDataResource@2f04477c context = '/xx/xxx-xxxxxxx', optional = false]' via location 'aws-secretsmanager:/xx/xxx-xxxxxxx' does not exist

I have the same message when I execute the jar version without the AWS token

[mmuniz75]

Looks like it is not able to get the AWS Token

[MatejNedic]

Hey @mmuniz75 could you provide sample please?

[mmuniz75]

parameter-native.zip

I have this one without token but still not starts in the compiled version.

You can test setting the properties :

export AWS_ACCESS_KEY_ID=AKIAXMDC5B4B3IYMKFE6 export AWS_SECRET_ACCESS_KEY=SkBFAJ7VRDmGOYMjD1iOaGfpyRPPJaUhLynVfQyI

and then call : curl --location 'http://localhost:8080/param'

[mmuniz75]

parameter-check.zip

Here is one more example that shows AWS SDK is ok to get the paremeter store in compiled version, the issue is just for spring-cloud-aws to get it

credentials : export AWS_ACCESS_KEY_ID=AKIAXMDC5B4B3IYMKFE6 export AWS_SECRET_ACCESS_KEY=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

[MatejNedic]

Hey @mmuniz75 add the following and it will work\

hints.reflection().registerType(TypeReference.of(ParameterStorePropertySources.class),
                hint -> hint.withMembers(MemberCategory.INVOKE_DECLARED_CONSTRUCTORS,
                        MemberCategory.INTROSPECT_DECLARED_METHODS, MemberCategory.DECLARED_FIELDS));

[mmuniz75]

I added this hints and still not works.

To workaroud this issue I create a Util to load all parameters and secrets using directly the AWS api before starts Spring Boot

AWSParamUtil.zip

In the project I replace the spring cloud version fors this ones : implementation 'software.amazon.awssdk:ssm:2.22.9' implementation 'software.amazon.awssdk:secretsmanager:2.22.9'

If I just have : implementation 'io.awspring.cloud:spring-cloud-aws-starter-secrets-manager:3.1.0' implementation 'io.awspring.cloud:spring-cloud-aws-starter-parameter-store:3.1.0'

The compilated version not execute

[mmuniz75]

Under version 3.2 now at least has error message :

11:02:36.003 [main] ERROR org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter --

***************************
APPLICATION FAILED TO START
***************************

Description:

Config data resource '[SecretsManagerConfigDataResource@3f469038 context = '/sc/cfc/gcp_homolog?prefix=gcp.', optional = false, enabled = true]' via location 'aws-secretsmanager:/sc/cfc/gcp_homolog?prefix=gcp.' does not exist

Action:

Check that the value 'aws-secretsmanager:/sc/cfc/gcp_homolog?prefix=gcp.' at class path resource [application.yaml] - 556:11 is correct, or prefix it with 'optional:'

For parameter store I got the same error

[MatejNedic]

@mmuniz75 Are you sure it can locate secrets and parameters?

I have hint above and for me I am able now and in projects which I have retrive parameter from paramstore when launching executable.

[mmuniz75]

what hits did you use ?

I tested without success with the following :

hints.reflection().registerType(TypeReference.of(ParameterStorePropertySources.class),
                                                            hint -> hint.withMembers(MemberCategory.INVOKE_DECLARED_CONSTRUCTORS,
                                                                                                      MemberCategory.INTROSPECT_DECLARED_METHODS,
                                                                                                      MemberCategory.DECLARED_FIELDS));

hints.reflection().registerType(TypeReference.of(SecretsManagerPropertySources.class),
                                        hint -> hint.withMembers(MemberCategory.INVOKE_DECLARED_CONSTRUCTORS,
                                                                                  MemberCategory.INTROSPECT_DECLARED_METHODS,
                                                                                  MemberCategory.DECLARED_FIELDS));