eclipse / steady

Analyses your Java applications for open-source dependencies with known vulnerabilities, using both static analysis and testing to determine code context and usage for greater accuracy. https://eclipse.github.io/steady/
Apache License 2.0
517 stars 123 forks source link

Getting the following error while trying to run steady on steady-cli #542

Closed ZupeiNie closed 2 years ago

ZupeiNie commented 2 years ago

Describe the bug This problem occurs when I run steady on steady-cli. I didn't find steady-cli-3.2.3, so I used steady-cli-3.2.0 the command ‘java -jar steady-cli-3.2.0-jar-with-dependencies.jar -goal app’ the error [main] [ERROR] e.steady.shared.util.StopWatch - [287797] [00.059 ms] Failed: Application context is required to execute goal [APP] Exception in thread "main" org.eclipse.steady.goals.GoalConfigurationException: Application context is required to execute goal [APP] at org.eclipse.steady.goals.AbstractAppGoal.prepareExecution(AbstractAppGoal.java:107) at org.eclipse.steady.goals.BomGoal.prepareExecution(BomGoal.java:57) at org.eclipse.steady.goals.AbstractGoal.start(AbstractGoal.java:418) at org.eclipse.steady.goals.AbstractGoal.execute(AbstractGoal.java:253) at org.eclipse.steady.goals.AbstractGoal.executeSync(AbstractGoal.java:247) at org.eclipse.steady.cli.VulasCli.main(VulasCli.java:99)

steady-custom.properties file I have edit the , and of the application to be analyzed and my workspace token 6A9A4AB1-CF14-4648-A57B-C669060596A5

steady-cli folder The application code (java, class or JAR files) and all application dependencies (JAR files) In the app folder A2BBC9FF-38D9-4FBD-AD75-6E57253AB71F

I have read the Java source code, but still can't find the problem and how to fix it. Can you please teach me how to solve this problem,thank you!

serenaponta commented 2 years ago

Hello @11111821,

the exception you encountered is due to the values you provided in the configuration file for

vulas.core.appContext.group = 
vulas.core.appContext.artifact = 
vulas.core.appContext.version = 

If you used the values shown in the screenshot <group>, <artifact> and <version>, those are invalid due to the presence of < and >. Those configurations are meant to contain the identifier of the application and resembles the Maven coordinates. An example of valid values are

vulas.core.appContext.group = com.example
vulas.core.appContext.artifact = sampleapp
vulas.core.appContext.version = 0.0.1

Moreover the configuration vulas.core.app.appPrefixes is meant to contain the package prefix of the application (i.e, the Java package under which all application classes are defined). In fact, the app folder contains both application and dependency code and the prefix provided in this configuration is used to discriminate the two. As mentioned in the documentation, the application classes are then used as entrypoints for the Maven goal a2c offered by the steady maven plugin.

Of course vulas.core.space.token must also be set with a token generated from the web frontend-app as described at https://eclipse.github.io/steady/user/tutorials/java_cli/#prerequisites

I now added the latest version of the steady-cli in the Github 3.2.3 release but the issues you encountered are not due to the cli version. I would anyway recommend to use the latest version.

Thank you!

ZupeiNie commented 2 years ago

Hi, thank you so much for your help, I can run steady-cli normally, but I still have some questions about vulas.core.app.appPrefixes, could you please give me an example, such as the project of spring-boot-demo, could you check how the vulas.core.app.appPrefixes of this project should be configured, thank you very much!

serenaponta commented 2 years ago

Hi @11111821,

the configuration vulas.core.app.appPrefixes should contain the common prefix of the Java package used by all modules (quite a number!). I went through the first 4 modules demo-activiti, demo-actuator, demo-admin, demo-async and from those I would use

vulas.core.app.appPrefixes = com.xkcoding

Please check that the same packaging com.xkcoding is used for all demo-* modules in https://github.com/xkcoding/spring-boot-demo. Also, it should not be a package used in any of the application dependencies or they would be treated as application code. If it is used in the dependencies, then it would be necessary to have more specific appPrefixes like

vulas.core.app.appPrefixes = com.xkcoding.activiti, com.xkcoding.actuator,...

picking the Java package for all modules demo-*

ZupeiNie commented 2 years ago

Thank you for your reply! The project of spring-boot-demo was too big, so I chose one of the modules, demo-zookeeper(https://github.com/xkcoding/spring-boot-demo/tree/master/demo-zookeeper). After scanning using Maven, I got the following information, but when I used steady-cli, I still failed and could not determine the value of the parameter vulas.core.app.appPrefixes. I looked at the output of maven and found that it found 77 dependencies through pom.xml, but when I looked at the output of steady-cli, it didn't find any dependencies. Then I read this paper([Detection, assessment and mitigation of vulnerabilities in open source dependencies | SpringerLink](https://link.springer.com/article/10.1007/s10664-020-09830-x)) about how steady works and noticed this sentence: Looping over Java archive analyzers to separate application and dependency code using package prefix (es). So I tried two values based on the contents of pom.xml and the results of the maven run, one of which was: com.xkcoding.zookeeper, com.xkcoding.zookeeper.annotation, com.xkcoding.zookeeper.aspectj, com.xkcoding.zookeeper.config, com.xkcoding.zookeeper.config.props,and the other is org.springframework.boot, org.apache.curator, cn.hutool, org.projectlombok, but neither is successful. Please take a look at why the relevant dependencies cannot be found. Thank you very much.

This is the steady front-end interface.

image-20220414193818694

This is part of the output from maven and steady-cli

#Scanned dependencies
....
[INFO] Dependency [  76]: Dependency [libid=[javax.activation|javax.activation-api|1.2.0], parent=[javax.xml.bind|jaxb-api|2.3.1], path=/Users/XXX/.m2/repository/javax/activation/javax.activation-api/1.2.0/javax.activation-api-1.2.0.jar, direct=false, scope=TEST] created for Maven artifact [g=javax.activation, a=javax.activation-api, base version=1.2.0, version=1.2.0, classifier=null]
[INFO]     com.xkcoding:demo-zookeeper:jar:1.0.0-SNAPSHOT => org.springframework.boot:spring-boot-starter-test:jar:2.1.0.RELEASE => org.xmlunit:xmlunit-core:jar:2.6.2 => javax.xml.bind:jaxb-api:jar:2.3.1 => javax.activation:javax.activation-api:jar:1.2.0
[INFO] Dependency [  77]: Dependency [libid=[org.projectlombok|lombok|1.18.2], parent=null, path=/Users/XXX/.m2/repository/org/projectlombok/lombok/1.18.2/lombok-1.18.2.jar, direct=true, scope=COMPILE] created for Maven artifact [g=org.projectlombok, a=lombok, base version=1.18.2, version=1.18.2, classifier=null]
[INFO]     com.xkcoding:demo-zookeeper:jar:1.0.0-SNAPSHOT => org.projectlombok:lombok:jar:1.18.2

#Analyze the scanned dependencies
ipse.steady.goals.AbstractGoal - Uploaded goal execution info
.steady.java.tasks.JavaBomTask - Searching for Java constructs in search path [/Users/wolfghost/Desktop/steady-use/spring-boot-demo-master/demo-zookeeper/src/main/java] with filter [jar, war, class, java, aar] ...
shared.util.AbstractFileSearch - Found [6] files in absolute path [/Users/wolfghost/Desktop/steady-use/spring-boot-demo-master/demo-zookeeper/src/main/java]
org.eclipse.steady.DirAnalyzer - File analyzers created: [6 total, 0 with error], [6 java]
org.eclipse.steady.DirAnalyzer - Constructs found: [16]
.steady.java.tasks.JavaBomTask - Searching for Java constructs in search path [/Users/wolfghost/Desktop/steady-use/spring-boot-demo-master/demo-zookeeper/target/classes] with filter [jar, war, class, java, aar] ...
shared.util.AbstractFileSearch - Found [6] files in absolute path [/Users/wolfghost/Desktop/steady-use/spring-boot-demo-master/demo-zookeeper/target/classes]
org.eclipse.steady.DirAnalyzer - File analyzers created: [6 total, 0 with error], [6 class]
org.eclipse.steady.DirAnalyzer - Constructs found: [28]
.steady.shared.util.ThreadUtil - Auto-threading enabled: Number of threads is [2 x 8 cores]
dy.java.ArchiveAnalysisManager - [0/77] archives already analyzed, the remaining [77] will be analyzed now ...

#Next, analyze them one by one.
ipse.steady.goals.AbstractGoal - Uploaded goal execution info
.steady.java.tasks.JavaBomTask - Searching for Java constructs in search path [/Users/XXX/XXX/XXX/steady-cli/app] with filter [jar, war, class, java, aar] ...
shared.util.AbstractFileSearch - Found [13] files in absolute path [/Users/XXX/XXX/XXX/steady-cli/app]
org.eclipse.steady.DirAnalyzer - File analyzers created: [13 total, 0 with error], [7 java], [6 class]
.steady.java.tasks.JavaBomTask - Looping over Java archive analyzers to separate application and dependency code using package prefix(es) [com.xkcoding.zookeeper, com.xkcoding.zookeeper.annotation, com.xkcoding.zookeeper.aspectj, com.xkcoding.zookeeper.config, com.xkcoding.zookeeper.config.props] ...
.steady.shared.util.ThreadUtil - Auto-threading enabled: Number of threads is [2 x 8 cores]
dy.java.ArchiveAnalysisManager - All [0/0] archives have been analyzed already
serenaponta commented 2 years ago

Hi @11111821,

when using the steady-cli the app folder must contain both the application code and the dependencies. Then the configuration of vulas.core.app.appPrefixes or vulas.core.app.appJarNames is used to discriminate between the two. Looking at the log of the steady-cli you provided above, I think you have the spring-boot-demo application in the app folder (as the DirAnalyzer found only Java and Class files) but the dependencies are not present (as usually happens in Maven projects). The way to proceed is to add the dependencies, e.g., simply running mvn dependency:copy-dependencies and configuring the package prefix (as you did).

I also tested the demo-zookeeper module: I run the mvn command above to get the dependencies within the app folder and configured vulas.core.app.appPrefixes = com.xkcoding and got the expected 77 dependencies. The first alternative you used for the appPrefixes configuration should also work but if you don't have dependencies under com.xkcoding the one I used makes it simpler :)

ZupeiNie commented 2 years ago

Hi @serenaponta Thank you very much for your help. I succeeded. Thank you!