lambci / docker-lambda

Docker images and test runners that replicate the live AWS Lambda environment
MIT License
5.83k stars 431 forks source link

System properties injection from LambdaJavaRTEntry-1.0.jar to My lambda handler #194

Closed cosmir17 closed 5 years ago

cosmir17 commented 5 years ago

Hi,

I made a docker file and managed to run 'LambdaJavaRTEntry-1.0.jar' manually with my own Java options (-D...... -D...... -D...... properties). https://github.com/lambci/docker-lambda/blob/master/java8/run/lambda-runtime-mock/src/main/java/lambdainternal/LambdaRuntime.java

Nonetheless, as shown on the picture, the java opts were provided to LambdaJavaRTEntry-1.0.jar but it does not get supplied to my lambda handler (mylambdaclass)

Screenshot 2019-06-15 at 07 24 53

Can I ask how I can supply JAVA OPTS necessary for 'mylambdaclass' handler? My handler needs to connect to different DBs and JAVA OPTS contain addresses of them (generated dynamically from Mavan) but LambdaJavaRTEntry-1.0.jar doesn't supply when it invoke my lambda handler.

Thank you, Sean

mhart commented 5 years ago

What are you using to pass these options in your production Lambda?

cosmir17 commented 5 years ago

It's fabric8 docker plugin. https://github.com/fabric8io/docker-maven-plugin/blob/master/samples/dockerfile/pom.xml [this example doesn't supply -D's to the docker file]

Nonetheless, I don't think it matters. Because 'ENTRYPOINT java $JAVA_OPTS_MAVEN -jar /var/runtime/lib/LambdaJavaRTEntry-1.0.jar' is basically same as the following if '$JAVA_OPTS_MAVEN' contains all the -D's below

ENTRYPOINT ["/usr/bin/java", "-XX:MaxHeapSize=2834432k", "-XX:MaxMetaspaceSize=163840k", "-XX:ReservedCodeCacheSize=81920k", \
  "-XX:+UseSerialGC", "-Xshare:on", "-XX:-TieredCompilation", "-Djava.net.preferIPv4Stack=true", \
"-jar", "/var/runtime/lib/LambdaJavaRTEntry-1.0.jar"]

https://github.com/lambci/docker-lambda/blob/master/java8/run/Dockerfile

My question is, how can I pass -D's specified in ENTRYPOINT to my handler(org.com.mylambdaclass) that is executed by '/var/runtime/lib/LambdaJavaRTEntry-1.0.jar'?

mhart commented 5 years ago

You'll probably need to write your own bootstrap. I really don't think AWS production Lambdas support what you're trying to do, which is why I was asking how you're doing it there.

cosmir17 commented 5 years ago

My apologies, I should have mentioned the following after you first commented. I only use maven/Docker only for my integration local test environment so that I can run cucumber test locally.

mhart commented 5 years ago

The best way to change things in the Lambda environment is to use environment variables – that way you're not diverging from the way that Lambda/SAM work live too.

Is there a reason you can't just use environment variables? What do you have to pass using -D?

cosmir17 commented 5 years ago

for the stage/production environment, the team supplies those environment variables from 'application.properties' files. My team uses an in-house cloud/jenkins management system provided from other team built on top of AWS so that we don't touch those AWS lambda variables directly..

For the local integration environment, it's what I am supplying.. (please ignore '-Xdebug'). ds

*Note that 'postgres' & 'redis' keywords will be placed with values from the fabric8 Maven plugin.

When you said, 'Is there a reason you can't just use environment variables? What do you have to pass using -D?', can I ask what you meant? My understanding is, java applications needs -D options when it gets executed for config properties.

mhart commented 5 years ago

I mean pulling whatever config you need from environment variables – so using System.getenv() for example.

That way you can also configure your live production Lambdas using the same variables: https://docs.aws.amazon.com/lambda/latest/dg/env_variables.html

When you run your docker container, you can just pass env variables using, eg, -e REDIS_PORT=6379. Or you can specify them in your Dockerfile if you're building your own image

mhart commented 5 years ago

BTW, I think your Dockerfile is the problem with why your -D options are not being picked up. You're using an environment variable that's not defined: $JAVA_OPTS_MAVEN

If you pass your -D options manually you'll see that they work just fine.

mhart commented 5 years ago

You can try something like this: https://stackoverflow.com/a/56324100/58173

cosmir17 commented 5 years ago
  1. I believe '-e' option is same as ENV in a docker file. I need to somehow, generate ENV lines with dynamically generated values in my dockerfile.
  2. I have just discovered that the fabric8 maven plugin support ENV feature. Screenshot 2019-06-15 at 18 59 42

Nonetheless, can I get back to you after some try? I've just read the source code in my team's project. This might acutally some other trivial issue. The lambda system I am trying to BDD gets config settings only from 'application.properties'. So I can just change the source code to get it from 'system properties' or using the following, https://github.com/lightbend/config for BDD (local) test environment. My apologies if it's the case, I should have checked more thoroughly..

Either case, I shall get back to you and leave feedback or asking for more help :)

cosmir17 commented 5 years ago

I am closing this ticket as it was caused by config logic in the source code. It was only getting configs from "application-{env}.properties" file. Thank you, @mhart for your kind help. My apologies that I was not thorough enough.