Open ivoanjo opened 5 years ago
jfuss
commented
5 years ago @ivoanjo This looks like an issue with your configuration on gradle. Running ./gradlew build, I see relatively the same output. That is I see the derpservice.jar in lib and the all the .class files. SAM CLI more or less shells out to gradle for the builder. I am not expert in gradle but this seems like something you would have control over in gradle no?
ivoanjo
commented
5 years ago Thanks for the quick feedback, @jfuss! 👍 It seems the issue is triggered from my use of the gradle application plugin.
If I remove it from my build script...
diff --git a/build.gradle.kts b/build.gradle.kts
index 4748d99..3877ace 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -1,7 +1,7 @@
plugins {
id("org.jetbrains.kotlin.jvm").version("1.3.31")
// id("com.github.johnrengelman.shadow").version("5.0.0")
- application
+ // application
}
repositories {
@@ -18,6 +18,6 @@ dependencies {
//testImplementation("org.http4k:http4k-client-okhttp:3.154.0")
}
-application {
- mainClassName = "derpservice.AppKt"
-}
+// application {
+// mainClassName = "derpservice.AppKt"
+// }... then the result of sam build is as expected:
.aws-sam/build/DerpServiceLambda/
├── derpservice
│ ├── AppKt.class
│ ├── DerpService$app$1.class
│ ├── DerpService$app$2.class
│ ├── DerpService$app$3.class
│ ├── DerpService$timingFilter$1$1.class
│ ├── DerpService$timingFilter$1.class
│ ├── DerpService.class
│ └── DerpServiceLambda.class
├── lib
│ ├── annotations-13.0.jar
│ ├── http4k-core-3.154.0.jar
│ ├── http4k-serverless-lambda-3.154.0.jar
│ ├── javax.servlet-api-4.0.1.jar
│ ├── kotlin-reflect-1.3.31.jar
│ ├── kotlin-stdlib-1.3.31.jar
│ ├── kotlin-stdlib-common-1.3.31.jar
│ ├── kotlin-stdlib-jdk7-1.3.31.jar
│ └── kotlin-stdlib-jdk8-1.3.31.jar
└── META-INF
├── derpservice.kotlin_module
└── MANIFEST.MF
3 directories, 19 filesSo it seems like the SAM CLI is having issues around copying the output when this plugin is enabled. The plugin itself is built-in to the default gradle distribution and was bootstrapped for me when I created the project with gradle init --type kotlin-application --dsl kotlin.
I'm not sure where/how the SAM CLI is deciding to copy all those files. Could you help me track down what command does the SAM CLI shell out to, and what options is it using?
jfuss
commented
5 years ago @ivoanjo All of our build gradle logic lives in here. Taking a quick look at the DESIGN.md, coping the libs is expected, as this is where the dependencies of the library are placed.
This build was built for the Java runtime, but noticing you are using Koltin. Maybe that has something to do with it? I am not familiar with how Koltin and Java are different in some of the build aspects.
ivoanjo
commented
5 years ago Apologies for the delay, and thanks @jfuss for pointing me in the right direction.
The issue here seems to be in the sam's gradle build logic, specifically this line (reproduced here):
def artifactJars = t.project.configurations.archives.artifacts.files.fileswith gradle's application plugin disabled, this list contains:
/tmp/tmp08iioaae/aca5d9a94f5c7674c2a5ca3b4aeb36b04f81d177/build/libs/derpservice.jarwhereas in my setup the list is:
/tmp/tmpuqfthfqf/aca5d9a94f5c7674c2a5ca3b4aeb36b04f81d177/build/libs/derpservice.jar
/tmp/tmpuqfthfqf/aca5d9a94f5c7674c2a5ca3b4aeb36b04f81d177/build/distributions/derpservice.zip
/tmp/tmpuqfthfqf/aca5d9a94f5c7674c2a5ca3b4aeb36b04f81d177/build/distributions/derpservice.tarwhich explains it. The next step is
copyToArtifactDir(artifactDir, artifactJars, runtimeCpJars)which does
def copyToArtifactDir(artifactDir, artifactJars, classPathJars) {
artifactJars.each {
it.withInputStream({ jis ->
def zipIs = new ZipInputStream(jis)
for (def e = zipIs.getNextEntry(); e != null; e = zipIs.getNextEntry()) {
def entryPath = artifactDir.resolve(e.name)
if (e.isDirectory()) {
makeDirs(entryPath)
} else {
copyToFile(zipIs, entryPath)
}
zipIs.closeEntry()
}
zipIs.close()
})
}
// ...Aka the problem is that as part of its work, the application plugin produces both a .zip and a .tar which include scripts to start the application, as well as all of the dependencies (e.g. if every gradle setup used the application plugin, SAM's life would be easy as telling gradle to build the distribution and uploading that zip to S3).
The SAM gradle build logic is picking up those zip/tar artifacts as if it was part of the application, and unzipping them into the build folder, thus we end up with the extra META-INF/, derpservice/bin and derpservice/lib.
This is somewhat where my gradle-foo starts to fail me. I believe the application plugin delegates to the distribution plugin, so I believe the fix here would be for SAM's gradle build logic to ignore any artifacts produced by this plugin (or just anything in a distributions/ folder?).
Let me know if I can provide any more info! 😃 🚀👍
Description
I'm working on a very simple (public) example service, using lambda and sam (also gradle 5 and kotlin).
When I run
sam buildand latersam package,samincludes in the deployedjartwo copies of my dependencies.Steps to reproduce
Checkout example service at https://gitlab.com/ivoanjo/derpservice, checkout
sam-issuesbranch or just download https://gitlab.com/ivoanjo/derpservice/-/archive/sam-issues/derpservice-sam-issues.tar.Run
sam build.Observed result
Output of
sam build:Resulting build folder (notice the two copies of the dependencies):
Also, there seems to be an extra copy of the service as a jar (
derpservice.jar) as well as thebin/launchers which are also useless for lambda.Expected result
Inside the build folder (and the zip file deployed), there should only be a single copy of every dependency, as well as a single copy of my service (either inside, or outside a jar, but not both).
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
sam --version: SAM CLI, version 0.17.0