micronaut-projects / micronaut-aws

Projects specific to integrating Micronaut and Amazon Web Services (AWS)
Apache License 2.0
87 stars 80 forks source link

Parameter store as Cloud Config not working in native image on AWS Lambda #1042

Open kieranjen opened 3 years ago

kieranjen commented 3 years ago

I am trying to use the parameter store feature explained in the documentation here. When building the application as a native image and trying to run on AWS Lambda, I get an error on startup.

Task List

Steps to Reproduce

  1. Build the example application with ./gradlew buildNativeLambda
  2. Create an SSM parameter called /config/application/example/awsTest with any value e.g. Test
  3. Upload the built native image zip to an AWS Lambda custom runtime
  4. Run the lambda function using the following JSON as the lambda input:
{
  "body": "eyJ0ZXN0IjoiYm9keSJ9",
  "resource": "/",
  "path": "/",
  "httpMethod": "POST",
  "isBase64Encoded": true,
  "queryStringParameters": {
    "foo": "bar"
  },
  "multiValueQueryStringParameters": {
    "foo": [
      "bar"
    ]
  },
  "pathParameters": {
    "proxy": "/"
  },
  "stageVariables": {
    "baz": "qux"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "US",
    "Host": "1234567890.execute-api.eu-west-2.amazonaws.com",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Custom User Agent String",
    "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==",
    "X-Forwarded-For": "127.0.0.1, 127.0.0.2",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "multiValueHeaders": {
    "Accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
    ],
    "Accept-Encoding": [
      "gzip, deflate, sdch"
    ],
    "Accept-Language": [
      "en-US,en;q=0.8"
    ],
    "Cache-Control": [
      "max-age=0"
    ],
    "CloudFront-Forwarded-Proto": [
      "https"
    ],
    "CloudFront-Is-Desktop-Viewer": [
      "true"
    ],
    "CloudFront-Is-Mobile-Viewer": [
      "false"
    ],
    "CloudFront-Is-SmartTV-Viewer": [
      "false"
    ],
    "CloudFront-Is-Tablet-Viewer": [
      "false"
    ],
    "CloudFront-Viewer-Country": [
      "US"
    ],
    "Host": [
      "0123456789.execute-api.eu-west-2.amazonaws.com"
    ],
    "Upgrade-Insecure-Requests": [
      "1"
    ],
    "User-Agent": [
      "Custom User Agent String"
    ],
    "Via": [
      "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)"
    ],
    "X-Amz-Cf-Id": [
      "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA=="
    ],
    "X-Forwarded-For": [
      "127.0.0.1, 127.0.0.2"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ]
  },
  "requestContext": {
    "accountId": "123456789012",
    "resourceId": "123456",
    "stage": "prod",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "requestTime": "09/Apr/2015:12:34:56 +0000",
    "requestTimeEpoch": 1428582896000,
    "identity": {
      "cognitoIdentityPoolId": null,
      "accountId": null,
      "cognitoIdentityId": null,
      "caller": null,
      "accessKey": null,
      "sourceIp": "127.0.0.1",
      "cognitoAuthenticationType": null,
      "cognitoAuthenticationProvider": null,
      "userArn": null,
      "userAgent": "Custom User Agent String",
      "user": null
    },
    "path": "/",
    "resourcePath": "/",
    "httpMethod": "POST",
    "apiId": "1234567890",
    "protocol": "HTTP/1.1"
  }
}

Expected Behaviour

When the application starts it should pick up the configuration from the SSM parameter store and print and return it from the API call.

Actual Behaviour

The following exception is raised when the application is starting up:


io.micronaut.context.exceptions.ConfigurationException: Exception thrown instantiating MicronautLambdaRuntimeHandler
--
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.createRequestHandler(MicronautLambdaRuntime.java:40)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.createHandler(AbstractMicronautLambdaRuntime.java:192)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.startRuntimeApiEventLoop(AbstractMicronautLambdaRuntime.java:313)
at io.micronaut.function.aws.runtime.AbstractMicronautLambdaRuntime.run(AbstractMicronautLambdaRuntime.java:129)
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.main(MicronautLambdaRuntime.java:50)
Caused by: com.amazonaws.serverless.exceptions.ContainerInitializationException: Error starting Micronaut container: Bean definition [io.micronaut.aws.sdk.v1.AWSClientConfiguration] could not be loaded: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.initialize(MicronautLambdaContainerHandler.java:252)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.<init>(MicronautLambdaContainerHandler.java:162)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.<init>(MicronautLambdaContainerHandler.java:115)
at io.micronaut.function.aws.proxy.MicronautLambdaHandler.<init>(MicronautLambdaHandler.java:54)
at io.micronaut.function.aws.runtime.MicronautLambdaRuntime.createRequestHandler(MicronautLambdaRuntime.java:38)
... 4 more
Caused by: io.micronaut.context.exceptions.BeanInstantiationException: Bean definition [io.micronaut.aws.sdk.v1.AWSClientConfiguration] could not be loaded: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1565)
at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:219)
at io.micronaut.context.DefaultBeanContext.readAllBeanDefinitionClasses(DefaultBeanContext.java:2857)
at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:231)
at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:165)
at io.micronaut.function.aws.proxy.MicronautLambdaContainerHandler.initialize(MicronautLambdaContainerHandler.java:248)
... 8 more
Caused by: io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type [io.micronaut.aws.sdk.v1.AWSClientConfiguration]: null
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1972)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2724)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2710)
at io.micronaut.context.DefaultBeanContext.loadContextScopeBean(DefaultBeanContext.java:2249)
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1559)
... 13 more
Caused by: java.lang.ExceptionInInitializerError
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:315)
at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:143)
at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:137)
at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:100)
at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:80)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:375)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:295)
at io.micronaut.aws.sdk.v1.AWSClientConfiguration.<init>(AWSClientConfiguration.java:36)
at io.micronaut.aws.sdk.v1.$AWSClientConfigurationDefinition.build(Unknown Source)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1943)
... 17 more
Caused by: java.lang.IllegalArgumentException
at com.amazonaws.internal.config.InternalConfig.loadfrom(InternalConfig.java:249)
at com.amazonaws.internal.config.InternalConfig.load(InternalConfig.java:263)
at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:336)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:375)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:295)
... 26 more
Request loop failed with: Exception thrown instantiating MicronautLambdaRuntimeHandler

Environment Information

Example Application

https://github.com/kieranjen/micronaut-graal-ssm-params

hchang-chwy commented 3 years ago

Seems to be related to AWSParameterStoreConfiguration where it extends AWSClientConfiguration from aws sdk v1 lib which is by nature not compatible with native image.

graemerocher commented 3 years ago

There is a PR waiting on @sdelamo https://github.com/micronaut-projects/micronaut-aws/pull/1060

hchang-chwy commented 3 years ago

Per https://arnoldgalovics.com/tackling-java-cold-startup-times-on-aws-lambda-with-graalvm/, I was finally able to fix the issue by adding below files under src/main/graal

resource-config.json

{
  "resources": [
    {
      "pattern": "com/amazonaws/internal/config/awssdk_config_default.json"
    },
    {
      "pattern": "com/amazonaws/partitions/endpoints.json"
    },
    {
      "pattern": "com/amazonaws/sdk/versionInfo.properties"
    }
  ]
}

reflect.json

[
  {
    "name": "com.amazonaws.internal.config.InternalConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.SignerConfig",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.SignerConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HttpClientConfig",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HttpClientConfigJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.HostRegexToRegionMappingJsonHelper",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  },
  {
    "name": "com.amazonaws.internal.config.JsonIndex",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true,
    "allDeclaredClasses": true
  }
]
graemerocher commented 3 years ago

@ilopmar maybe we should include the above config until the PR @sdelamo has assigned to him is dealt with?

ilopmar commented 3 years ago

We do have support for ParameterStore but using SDK v2 (which includes it out of the box). This is our test application: https://github.com/micronaut-graal-tests/micronaut-aws-sdk2-graal/tree/2.5.x_paramstore

graemerocher commented 3 years ago

So what is #1060 for?

ilopmar commented 3 years ago

@sdelamo can confirm it but I think that PR adds support for use Parameter Store as a Distributed Config provider in Micronaut.

sdelamo commented 3 years ago

1060 is share logic for both distributed configuration solutions AWS Parameter store and AWS Secret Manager.

graemerocher commented 3 years ago

@sdelamo can this be closed since in Micronaut 3 it now uses the v2 SDK?