search

cagataygurturk / lambadaframework

Build serverless REST API's with JAVA. It implements the JAX-RS API and deploys your application easily to AWS Lambda and API Gateway
MIT License
244 stars 48 forks source link

Adding own dependency #30

Closed anderjo4 closed 7 years ago

anderjo4 commented 7 years ago

Hey,

I'm trying to add a dependency with maven.

My dependencies in the pom.xml looks like this:

<dependencies>
        <dependency>
            <groupId>org.lambadaframework</groupId>
            <artifactId>runtime</artifactId>
            <version>${lambada.version}</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-log4j</artifactId>
            <version>1.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>1.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20160810</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

It's the dependency json.org that won't work.

The Error I receive is : 

 A required class was missing while executing org.lambadaframework:lambada-maven-plugin:0.0.6:deploy: org/json/JSONObject
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import  from realm ClassRealm[project>se.some.project:project-test:0.0.6, parent: ClassRealm[maven.api, parent: null]]]
[ERROR] 
[ERROR] -----------------------------------------------------: org.json.JSONObject

I did suspect that it's the maven-shade-pluginthat causes the problem but the log clearly states that it's included in the shaded jar:

Including org.json:json:jar:20160810 in the shaded jar.

Local maven set up

Maven home: /opt/apache-maven-3.3.9
Java version: 1.8.0_102, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12", arch: "x86_64", family: "mac"

Is it possible to add dependencies?

Thanks!

//Jonathan

anderjo4 commented 7 years ago

Tried with gson instead, the problem persists.

cagataygurturk commented 7 years ago

Jackson is included in Lambada, maybe you can consider using it.

anderjo4 commented 7 years ago

@cagataygurturk Yep, looks like i have to go with that. The questions is, what will happen when I have to include other dependencies? Will they crash and burn too?

cagataygurturk commented 7 years ago

No, of course it won't crash with other dependencies. This is obviously a JSON library specific error, I'll investigate the root cause. With other dependencies you should not experience any issue in theory.

anderjo4 commented 7 years ago

@cagataygurturk Thanks!

cagataygurturk commented 7 years ago

Closing the issue right now. Please open again if you experience another problem.

anderjo4 commented 7 years ago

Ok, Could you reproduce the problem?

anderjo4 commented 7 years ago

@cagataygurturk I switched to using jacksson. But it produces same error :/

Number of foreign imports: 1
import: Entry[import  from realm ClassRealm[project>se.some.project:project-test:0.0.6, parent: ClassRealm[maven.api, parent: null]]]

-----------------------------------------------------

    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:166)
    ... 21 more
Caused by: java.lang.NoClassDefFoundError: com/oracle/javafx/jmx/json/JSONDocument
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
    at java.lang.Class.privateGetPublicMethods(Class.java:2902)
    at java.lang.Class.getMethods(Class.java:1615)
    at org.glassfish.jersey.server.model.MethodList.getMethods(MethodList.java:109)
    at org.glassfish.jersey.server.model.MethodList.<init>(MethodList.java:96)
    at org.glassfish.jersey.server.model.MethodList.<init>(MethodList.java:77)
    at org.glassfish.jersey.server.model.IntrospectionModeller.doCreateResourceBuilder(IntrospectionModeller.java:135)
    at org.glassfish.jersey.server.model.IntrospectionModeller.access$000(IntrospectionModeller.java:80)
    at org.glassfish.jersey.server.model.IntrospectionModeller$1.call(IntrospectionModeller.java:112)
    at org.glassfish.jersey.server.model.IntrospectionModeller$1.call(IntrospectionModeller.java:109)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:255)
    at org.glassfish.jersey.server.model.IntrospectionModeller.createResourceBuilder(IntrospectionModeller.java:109)
    at org.glassfish.jersey.server.model.Resource.from(Resource.java:797)
    at org.lambadaframework.jaxrs.JAXRSParser.getResourcesFromClassRecursive(JAXRSParser.java:191)
    at org.lambadaframework.jaxrs.JAXRSParser.scan(JAXRSParser.java:180)
    at org.lambadaframework.aws.ApiGateway.getResources(ApiGateway.java:279)
    at org.lambadaframework.aws.ApiGateway.deployEndpoints(ApiGateway.java:132)
    at org.lambadaframework.deployer.LambadaDeployer.execute(LambadaDeployer.java:81)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
    ... 21 more
Caused by: java.lang.ClassNotFoundException: com.oracle.javafx.jmx.json.JSONDocument
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 43 more

Looks like i can't reopen this Issue.

cagataygurturk commented 7 years ago

What's the packageName value in your pom.xml? Set it to the package where your JAX-RS annotations exist and let's see what'll happen. Like se.some.project.controllers or something like that.

anderjo4 commented 7 years ago

Ok I've tried change my package name but got the same error. My JAX-RS Annotations is in : se.some.project.rest

PackageName Before: se.some.project PackageName After: se.some.project.rest

I've also have this package where JSONDocument is used frequently: se.some.project.repo

This is not a know error? Don't want to waste your time on my screw up.

anderjo4 commented 7 years ago

Opps. JSONDocument is not Jacksson. It belongs to: com.oracle.javafx.jmx.json.JSONDocument;

anderjo4 commented 7 years ago

Well this have boiled down to not being about JSON.

I've removed everything that where JSON specific. Now I get the same error but with my other not standard import which is:

        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>1.3.0</version>
        </dependency>

This package has the API to communicate with AWS dynamoDB. When I try to deploy it I get the error:

[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import  from realm ClassRealm[project>se.simonsoft.review.rest:review-rest:0.0.6, parent: ClassRealm[maven.api, parent: null]]]
[ERROR] 
[ERROR] -----------------------------------------------------: com.amazonaws.services.dynamodbv2.model.ConditionalCheckFailedException

Same as before.

First I suspected that it was a maven problem. I've tried:

Changes that differ from original lambadaframework-boilerplate pom.xml:

My project structure: se.some.project.rest: Contains one class which has the JAX-RS annotations. se.some.project.repo: Contains classes needed to integrate with DynamoDB. test se.some.project.repo: Tests for dynamoDB integration.

When does it work: If I remove the whole package se.some.project.repo with it's classes it works even if I add a dependency such as json.org and using it inside my class with JAX-RS annotations.

I do still think this is a Maven problem, could it be that my own classes has to be included in the Über jar in some way? I can't find that it adding my class that contains my JAX-RS annotations in the Über jar, should it add it?

cagataygurturk commented 7 years ago

If you post your full pom.xml it will be more helpful.

cagataygurturk commented 7 years ago 
I do still think this is a Maven problem, could it be that my own classes has to be included in the Über jar in some way? I can't find that it adding my class that contains my JAX-RS annotations in the Über jar, should it add it?

Everything, your classes and dependency classes should be added to Über JAR. Maven Shade Plugin configuration in lambada-boilerplate project is configured correctly for that purpose.

anderjo4 commented 7 years ago 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>se.some.project</groupId>
    <artifactId>project-lamabada</artifactId>
    <packaging>jar</packaging>
    <version>0.0.6</version>
    <properties>
        <lambada.version>0.0.6</lambada.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <deployment.bucket>some-lambada-test</deployment.bucket>
        <deployment.package>se.some.project</deployment.package>
        <!-- Region to deploy-->
        <deployment.region>eu-west-1</deployment.region>
        <!-- Stage to deploy -->
        <deployment.stage>production</deployment.stage>
        <!-- Maximum execution time for lambda -->
        <deployment.maximumExecutionTime>10</deployment.maximumExecutionTime>
        <!-- Maximum allowed memory usage for lambda -->
        <deployment.lambdaMemorySize>1024</deployment.lambdaMemorySize>

    </properties>
    <dependencies>
        <dependency>
            <groupId>org.lambadaframework</groupId>
            <artifactId>runtime</artifactId>
            <version>${lambada.version}</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-log4j</artifactId>
            <version>1.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>1.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20160810</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.lambadaframework</groupId>
                <artifactId>lambada-maven-plugin</artifactId>
                <version>${lambada.version}</version>
                <configuration>
                    <packageName>${deployment.package}</packageName>
                    <regionToDeploy>${deployment.region}</regionToDeploy>
                    <stageToDeploy>${deployment.stage}</stageToDeploy>
                    <lambdaMaximumExecutionTime>${deployment.maximumExecutionTime}</lambdaMaximumExecutionTime>
                    <lambdaMemorySize>${deployment.lambdaMemorySize}</lambdaMemorySize>
                    <lambdaExecutionRolePolicies>
                        <param>arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess</param>
                    </lambdaExecutionRolePolicies>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>prepare</goal>
                            <goal>deploy</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <!--
            Maven Shade plugin packages the project to a fat JAR with all required dependencies.
            With minimum dependencies it should create a 5 MB jar file.
            -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.4.3</version>
                <configuration>
                    <!-- It is important to NOT to activate minimizeJar option -->
                    <minimizeJar>false</minimizeJar>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
        <extensions>
            <extension>
                <groupId>org.lambadaframework</groupId>
                <artifactId>wagon</artifactId>
                <version>${lambada.version}</version>
            </extension>
        </extensions>
    </build>
    <distributionManagement>
        <repository>
            <id>aws-releases</id>
            <name>AWS Release Repository</name>
            <url>s3://${deployment.bucket}/releases</url>
        </repository>
        <snapshotRepository>
            <id>aws-snapshots</id>
            <name>AWS Snapshot Repository</name>
            <url>s3://${deployment.bucket}/snapshots</url>
        </snapshotRepository>
    </distributionManagement>
</project>
cagataygurturk commented 7 years ago

Try to change deployment.package to se.some.project.rest and let's see. And if you can paste the whole maven output it'll also be helpful so I can see at which stage it breaks.

az3 commented 7 years ago

@anderjo4 Did you try using dependencyManagement? Instead of adding direct dependency under project.dependencies.dependency; try adding hints via project.dependencyManagement.dependencies.dependency. That will force maven (and indirect dependencies) to use your version of libraries.

anderjo4 commented 7 years ago

@az3 Wrapped <dependencies> in < dependencyManagement> But now it can't find any symbols at all.

anderjo4 commented 7 years ago

I've unzipped the generated jar in target. In my case it is named review-rest-0.0.6, It contains all the jars it should contain. In other words, my own classes and JSON is included in the uber jar.

cagataygurturk commented 7 years ago

I really did not understand the problem but I saw some users on stackoverflow complaining about the same error message: http://stackoverflow.com/questions/12364296/maven-build-error-failure

Maybe you can consider giving a try with maven 2

arran4 commented 7 years ago

In my opinion it's an issue with the JaxRsParser, it doesn't seem to be acknowledging classes outside the scan package regardless of their usage. I got around this temporally by using a ServiceLoader, it's not the best way of doing it, but you have to make sure that the exposed class is completely clean of dependencies... I suspect there is a way of doing this with an interface but I haven't played around with it.

http://docs.oracle.com/javase/7/docs/api/java/util/ServiceLoader.html

your.program.package1

^ Neither above refer to anything outside your.program.package, all classes return JaxRs "responses" and consume primitives, or classes within the package1 package.

your.program.package2

And this worked fine. It will require more digging to fix this issue. Let me know if you test using an interface with annotations.

Ping me here if more issues.

arran4 commented 7 years ago

https://docs.oracle.com/javase/tutorial/ext/basics/spi.html

anderjo4 commented 7 years ago

@cagataygurturk

same error message: http://stackoverflow.com/questions/12364296/maven-build-error-failure Yeah I've found that one to, and tried it. None of the proposed solutions worked.

I guess I could try with maven 2.

anderjo4 commented 7 years ago

@arran4 Hey!

I went with your solution and it works perfectly. Many thanks! Later on I will try to solve this with annotations as you've said. But this will absolutely do for now!

@cagataygurturk Thanks for trying to help me and for the great lib!

cagataygurturk commented 7 years ago

If this solution can be an addition to the framework please let me know/send a PR.

anderjo4 commented 7 years ago

@cagataygurturk I will.

anderjo4 commented 7 years ago

@arran4 @cagataygurturk Have researched the cause of the problem. My best guess is that the Maven plugin class-path must include the dependencies because the maven plugin class-path is separated from the artifacts dependencies. To solve this the dependencies referenced within the package holding the JAX-RS annotations must be declared twice in the pom.xml. Once in the normal place to put dependencies project.dependencies.dependency and once in the plugin. In this case it's project.build.plugins.plugin.dependencies.dependency

The deployment.package must now be the parent package. Not the package holding the class with JAX-RS annotations

anderjo4 commented 7 years ago

Exampel of working pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>se.some.package</groupId>
    <artifactId>something</artifactId>
    <packaging>jar</packaging>
    <version>0.0.6</version>
    <properties>
        <lambada.version>0.0.6</lambada.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <deployment.bucket>some-bucket</deployment.bucket>
        <deployment.package>se.some.package</deployment.package>
        <!-- Region to deploy-->
        <deployment.region>eu-west-1</deployment.region>
        <!-- Stage to deploy -->
        <deployment.stage>production</deployment.stage>
        <!-- Maximum execution time for lambda -->
        <deployment.maximumExecutionTime>10</deployment.maximumExecutionTime>
        <!-- Maximum allowed memory usage for lambda -->
        <deployment.lambdaMemorySize>1024</deployment.lambdaMemorySize>

    </properties>
    <dependencies>
        <dependency>
            <groupId>org.lambadaframework</groupId>
            <artifactId>runtime</artifactId>
            <version>${lambada.version}</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-log4j</artifactId>
            <version>1.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>1.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20160810</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.lambadaframework</groupId>
                <artifactId>lambada-maven-plugin</artifactId>
                <version>${lambada.version}</version>
                <configuration>
                    <packageName>${deployment.package}</packageName>
                    <regionToDeploy>${deployment.region}</regionToDeploy>
                    <stageToDeploy>${deployment.stage}</stageToDeploy>
                    <lambdaMaximumExecutionTime>${deployment.maximumExecutionTime}</lambdaMaximumExecutionTime>
                    <lambdaMemorySize>${deployment.lambdaMemorySize}</lambdaMemorySize>
                    <lambdaExecutionRolePolicies>
                        <param>arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess</param>
                    </lambdaExecutionRolePolicies>
                </configuration>
            <dependencies>
                <dependency>
                    <groupId>com.amazonaws</groupId>
                    <artifactId>aws-lambda-java-events</artifactId>
                    <version>1.3.0</version>
                </dependency>
                <dependency>
                    <groupId>org.json</groupId>
                    <artifactId>json</artifactId>
                    <version>20160810</version>
                </dependency>
            </dependencies>
                <executions>
                    <execution>
                        <goals>
                            <goal>prepare</goal>
                            <goal>deploy</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <!--
            Maven Shade plugin packages the project to a fat JAR with all required dependencies.
            With minimum dependencies it should create a 5 MB jar file.
            -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.4.3</version>
                <configuration>
                    <!-- It is important to NOT to activate minimizeJar option -->
                    <minimizeJar>false</minimizeJar>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
        <extensions>
            <extension>
                <groupId>org.lambadaframework</groupId>
                <artifactId>wagon</artifactId>
                <version>${lambada.version}</version>
            </extension>
        </extensions>
    </build>
    <distributionManagement>
        <repository>
            <id>aws-releases</id>
            <name>AWS Release Repository</name>
            <url>s3://${deployment.bucket}/releases</url>
        </repository>
        <snapshotRepository>
            <id>aws-snapshots</id>
            <name>AWS Snapshot Repository</name>
            <url>s3://${deployment.bucket}/snapshots</url>
        </snapshotRepository>
    </distributionManagement>
</project>
arranubels commented 7 years ago

I have tested the above, it works. Also I made the lambada maven plugin dependency the project itself, that also worked. Doing that you don't need to duplicate all your dependencies.

            <plugin>
                <groupId>org.lambadaframework</groupId>
                <artifactId>lambada-maven-plugin</artifactId>
                <version>${lambada.version}</version>
                <configuration>
                    <packageName>${deployment.package}</packageName>
                    <regionToDeploy>${deployment.region}</regionToDeploy>
                    <stageToDeploy>${deployment.stage}</stageToDeploy>
                    <lambdaMaximumExecutionTime>${deployment.maximumExecutionTime}</lambdaMaximumExecutionTime>
                    <lambdaMemorySize>${deployment.lambdaMemorySize}</lambdaMemorySize>
                </configuration>
        <dependencies>
        <dependency>
            <groupId>org.lambadaframework</groupId>
            <artifactId>boilerplate</artifactId>
            <version>0.0.6</version>
        </dependency>
        </dependencies>

See: https://github.com/arranubels/lambadaframework-boilerplate/blob/example1.2/pom.xml

We should probably add this to the boiler plate. I'll send a PR.