search

konveyor / tackle2-addon-test-generator

Tackle (2nd generation) addon for TackleTest generator.
Apache License 2.0
0 stars 5 forks source link

Tkltest-unit command Fails #4

Closed JaydipGabani closed 1 year ago

JaydipGabani commented 1 year ago

tkltest-unit command running in task pod right now fails with the below log

------------------------------------------------------------------------","> [INFO] BUILD SUCCESS","> [INFO] ------------------------------------------------------------------------","> [INFO] Total time: 22.422 s","> [INFO] Finished at: 2022-09-29T14:21:30Z","> [INFO] ------------------------------------------------------------------------","> * CTD interaction level: 1","> [tkltest|14:21:07.255] Loading config file /buckets/83144f49-d4ff-4944-9bcc-c03ac3112aba/config.toml","> [tkltest|14:21:07.256] Warning: Unsupported flag in toml file: name","> [tkltest|14:21:07.256] Warning: Unsupported flag in toml file: generate.add_assertions","> [tkltest|14:21:30.721] Generating tests for app onlinebookstore using config file /buckets/83144f49-d4ff-4944-9bcc-c03ac3112aba/config.toml.","> [tkltest|14:21:30.730] Computing coverage goals using CTD","> [tkltest|14:21:31.490] ERROR: Computing CTD coverage goals failed: Command '\"/usr/lib/jvm/java-11-openjdk-11.0.16.1.1-1.el8_6.x86_64/bin/java\" -Xmx2048m -cp /opt/tackle-test-generator-cli/tkltest-lib/tackle-test-generator-unit-main-SNAPSHOT.jar:/opt/tackle-test-generator-cli/tkltest-lib/acts_3.2.jar:/opt/tackle-test-generator-cli/tkltest-lib/commons-cli-1.4.jar:/opt/tackle-test-generator-cli/tkltest-lib/soot-4.1.0.jar:/opt/tackle-test-generator-cli/tkltest-lib/axml-2.0.0.jar:/opt/tackle-test-generator-cli/tkltest-lib/slf4j-api-1.7.5.jar:/opt/tackle-test-generator-cli/tkltest-lib/guava-29.0-jre.jar:/opt/tackle-test-generator-cli/tkltest-lib/failureaccess-1.0.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-analysis-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-commons-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-tree-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-utils-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/heros-1.2.0.jar:/opt/tackle-test-generator-cli/tkltest-lib/httpcore-4.4.6.jar:/opt/tackle-test-generator-cli/tkltest-lib/httpclient-4.5.13.jar:/opt/tackle-test-generator-cli/tkltest-lib/jackson-databind-2.12.6.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/jackson-core-2.12.6.jar:/opt/tackle-test-generator-cli/tkltest-lib/jackson-annotations-2.12.6.jar org.konveyor.tackle.testgen.model.CTDTestPlanGenerator -app onlinebookstore -pt /working/onlinebookstore/OnlineBookStore -clpt /buckets/83144f49-d4ff-4944-9bcc-c03ac3112aba/tkltest-output-unit-onlinebookstore/onlinebookstore_build_classpath.txt -ic 1' returned non-zero exit status 1.","> SLF4J: Failed to load class \"org.slf4j.impl.StaticLoggerBinder\".","> SLF4J: Defaulting to no-operation (NOP) logger implementation","> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.","> Exception in thread \"main\" java.lang.NullPointerException","> \tat soot.JastAddJ.Program.initPaths(Program.java:336)","> \tat soot.SootResolver.initializeProgram(SootResolver.java:91)","> \tat soot.SootResolver.getProgram(SootResolver.java:386)","> \tat soot.JastAddInitialResolver.formAst(JastAddInitialResolver.java:62)","> \tat soot.JavaClassSource.resolve(JavaClassSource.java:66)","> \tat soot.SootResolver.bringToHierarchyUnchecked(SootResolver.java:253)","> \tat soot.SootResolver.bringToHierarchy(SootResolver.java:221)","> \tat soot.SootResolver.bringToSignatures(SootResolver.java:292)","> \tat soot.SootResolver.processResolveWorklist(SootResolver.java:179)","> \tat soot.SootResolver.resolveClass(SootResolver.java:141)","> \tat soot.Scene.loadClass(Scene.java:1067)","> \tat soot.Scene.loadClassAndSupport(Scene.java:1052)","> \tat soot.Scene.loadNecessaryClasses(Scene.java:1859)","> \tat org.konveyor.tackle.testgen.util.Utils.initSootClasses(Utils.java:183)","> \tat org.konveyor.tackle.testgen.model.RapidTypeAnalysis.performAnalysis(RapidTypeAnalysis.java:87)","> \tat org.konveyor.tackle.testgen.model.CTDTestPlanGenerator.<init>(CTDTestPlanGenerator.java:147)","> \tat org.konveyor.tackle.testgen.model.CTDTestPlanGenerator.main(CTDTestPlanGenerator.java:1154)","> ","> "]}

The application I am using is - https://github.com/shashirajraja/onlinebookstore

toml config I am using is - 

name = 'TKLTEST_CONFIG_FILE'

[general]
app_name = 'onlinebookstore'
monolith_app_path = ['/working/onlinebookstore/OnlineBookStore']
java_jdk_home = '/usr/lib/jvm/java-11-openjdk-11.0.16.1.1-1.el8_6.x86_64'
offline_instrumentation = true
build_type = 'maven'

[generate]
time_limit = 2
add_assertions = false
app_build_files = ['/working/onlinebookstore/pom.xml']
target_class_list = []

[generate.ctd_amplified]
base_test_generator = 'combined'
interaction_level = 1
no_ctd_coverage = false
num_seq_executions = 2

Need help with -

  1. Figuring out correct toml config to execute the command right
  2. Understand significance of each toml field to correctly construct config and accommodate all types of java applications
rachelt44 commented 1 year ago

@JaydipGabani for the example above I would use the following toml file:

name = 'TKLTEST_CONFIG_FILE'

[general]
app_name = 'onlinebookstore'
monolith_app_path = []
java_jdk_home = '/usr/lib/jvm/java-11-openjdk-11.0.16.1.1-1.el8_6.x86_64'
build_type = 'maven'

[generate]
time_limit = 10  (** can be changed by user **)
app_build_files = ['/working/onlinebookstore/pom.xml']
target_class_list = []  (** can be changed by user **)
excluded_class_list = [] (** can be changed by user **)

In addition, test_directory needs to be set to the location where you want the output tests to be located, and and max_memory_for_coverage needs to be set to a reasonable number in MB that TackleTest can consume during generation. Current default is 4096, which as far as I understand is way too high for your use case.

djzager commented 1 year ago

@rachelt44 So I've run into a couple of problems.

First and foremost is that, when I exclude monolith_app_path for example:

[general]
app_name = 'theonlinebookstore'
test_directory = 'tests/'

[generate]
app_build_files = ['/working/onlinebookstore/pom.xml']
time_limit = 30

I get the error, ERROR: Failed to automatically obtain modules from user build files.

Second, If I manually add the monolith_app_path, for example:

[general]
app_name = 'theonlinebookstore'
test_directory = 'tests/'
monolith_app_path = ['/working/onlinebookstore/OnlineBookStore']

[generate]
app_build_files = ['/working/onlinebookstore/pom.xml']
time_limit = 30

I feel like I'm making progress but it still fails:

bash-4.4# tkltest-unit --verbose --config-file /buckets/bfde5e6a-c263-4d6d-a817-596539bcdfe0/config.toml generate ctd-amplified
[tkltest|14:39:19.139] Loading config file /buckets/bfde5e6a-c263-4d6d-a817-596539bcdfe0/config.toml
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------< theonlinebookstore:theonlinebookstore >----------------
[INFO] Building theonlinebookstore 0.0.1-SNAPSHOT
[INFO] --------------------------------[ war ]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.3:build-classpath (default-cli) @ theonlinebookstore ---
[INFO] Wrote classpath file '/working/onlinebookstore/MavenClassPath.txt'.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.522 s
[INFO] Finished at: 2022-10-07T14:39:22Z
[INFO] ------------------------------------------------------------------------
[tkltest|14:39:22.276] Generating tests for app theonlinebookstore using config file /buckets/bfde5e6a-c263-4d6d-a817-596539bcdfe0/config.toml.
[tkltest|14:39:22.288] Computing coverage goals using CTD
* CTD interaction level: 1
[tkltest|14:39:22.688] ERROR: Computing CTD coverage goals failed: Command '"/usr/lib/jvm/jre-11/bin/java" -Xmx2048m -cp /opt/tackle-test-generator-cli/tkltest-lib/tackle-test-generator-unit-main-SNAPSHOT.jar:/opt/tackle-test-generator-cli/tkltest-lib/acts_3.2.jar:/opt/tackle-test-generator-cli/tkltest-lib/commons-cli-1.4.jar:/opt/tackle-test-generator-cli/tkltest-lib/soot-4.1.0.jar:/opt/tackle-test-generator-cli/tkltest-lib/axml-2.0.0.jar:/opt/tackle-test-generator-cli/tkltest-lib/slf4j-api-1.7.5.jar:/opt/tackle-test-generator-cli/tkltest-lib/guava-29.0-jre.jar:/opt/tackle-test-generator-cli/tkltest-lib/failureaccess-1.0.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-analysis-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-commons-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-tree-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/asm-utils-7.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/heros-1.2.0.jar:/opt/tackle-test-generator-cli/tkltest-lib/httpcore-4.4.6.jar:/opt/tackle-test-generator-cli/tkltest-lib/httpclient-4.5.13.jar:/opt/tackle-test-generator-cli/tkltest-lib/jackson-databind-2.12.6.1.jar:/opt/tackle-test-generator-cli/tkltest-lib/jackson-core-2.12.6.jar:/opt/tackle-test-generator-cli/tkltest-lib/jackson-annotations-2.12.6.jar org.konveyor.tackle.testgen.model.CTDTestPlanGenerator  -app theonlinebookstore -pt /working/onlinebookstore/OnlineBookStore -clpt /working/onlinebookstore/tkltest-output-unit-theonlinebookstore/theonlinebookstore_build_classpath.txt -ic 1' returned non-zero exit status 1.
Exception in thread "main" java.lang.IllegalArgumentException: Unrecognized app entry type: /working/onlinebookstore/OnlineBookStore
        at org.konveyor.tackle.testgen.model.CTDTestPlanGenerator$ClassListTargets.<init>(CTDTestPlanGenerator.java:805)
        at org.konveyor.tackle.testgen.model.CTDTestPlanGenerator$ClassListTargets.<init>(CTDTestPlanGenerator.java:752)
        at org.konveyor.tackle.testgen.model.CTDTestPlanGenerator.<init>(CTDTestPlanGenerator.java:142)
        at org.konveyor.tackle.testgen.model.CTDTestPlanGenerator.main(CTDTestPlanGenerator.java:1154)

NOTE I'm simplifying things for myself, for now, by excluding max_memory_for_coverage. Will come back to that when it's working.

rachelt44 commented 1 year ago

@djzager can you send a link to the application version you are using? we will look into this

rachelt44 commented 1 year ago

@djzager also can you share the content of /working/onlinebookstore/OnlineBookStore? For some reason it is being treated as an entry of the app. The monolith app path should contain the top folders containing the class files. For example, if the application has a single package named org.myapp and it is located in a folder named app, then app should be given as the monolith app path, and it should contain the folder org, then folder myapp, and then the class files inside myapp.

djzager commented 1 year ago

can you send a link to the application version you are using? we will look into this

https://github.com/konveyor/tackle-test-generator-cli/releases/download/v2.4.0/tackle-test-generator-cli-v2.4.0-all-deps.zip

also can you share the content of /working/onlinebookstore/OnlineBookStore?

There is no content there. Thought it was strange seeing it from the original post but ran with it. Running it manually via latest tkltest container and podman -- updating monolith_app_path = ["src/main/java/"] -- I can get some different errors in this Gist :sunglasses:.

Both these projects look like ones I should be able to run tkltest-unit against, is that right?:

Apologies for the struggle, it has been awhile since I've interacted with Java projects and I'm sure that's playing a role.

rachelt44 commented 1 year ago

@djzager definitely we should be able to run against these apps. So just to clarify, when you give the correct monolith app path, you get the Java error in the soot static analysis phase, and when you don't give a monolith app path, maven fails to obtain the Java modules (as depicted in this issue)?

rachelt44 commented 1 year ago

@djzager I somehow missed this, but monolith app path should not point to the Java source files (as in monolith_app_path = ["src/main/java/"]), but rather to the class files (target/classes).

djzager commented 1 year ago

Closing this as we've addressed these problems.