binaryeq / jcompile

scripts to compile Java projects with different compilers to create a data set of comparable binaries
Apache License 2.0
0 stars 0 forks source link

`revapi` runner script #69

Closed wtwhite closed 11 months ago

wtwhite commented 11 months ago

A quick hacky script to generate a script that will run revapi on all jar pairs reported by the prototype NEQ1 oracle (AdjacentVersionSameArtifactAndCompilerClassOracle).

This is a stopgap to get some results before spending the time to integrate this properly into NEQ1.

wtwhite commented 11 months ago

At https://github.com/binaryeq/jcompile/pull/69/commits/c83914efbad84a9ed832f5f17cb58c67a4327315, running this on AdjacentVersionSameArtifactAndCompilerClassOracle output for run 31 on wtwhite-vuw-vm produces results of BREAKING changes for all 1414 input jar pairs in 90 minutes:

wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ wc -l ../../all_jar_pairs_for_NEQ1.txt 
1414 ../../all_jar_pairs_for_NEQ1.txt
wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ find . -name '*.revapi.json' -ls|wc -l
1414
wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ tail -2 nohup_first_revapi_run_BREAKING.out 
20521.55user 854.52system 1:31:28elapsed 389%CPU (0avgtext+0avgdata 2257072maxresident)k
15480inputs+3545656outputs (11442major+284419835minor)pagefaults 0swaps

But 289 of these jar pairs have empty results (just []):

wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ find . -name '*.revapi.json' -size 2c|wc -l
289
wtwhite commented 11 months ago

Plan: Merge this, but also rerun with minimum severity POTENTIALLY_BREAKING and see if this gets more. It does for at least one example that previously returned empty results:

wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ revapi.sh --extensions=org.revapi:revapi-java:0.28.1,org.revapi:revapi-reporter-text:0.15.0 --old=jars/openjdk-9.0.1/json-20230227.jar --new=jars/openjdk-9.0.1/json-20230618.jar
08:54:43.208 INFO  Downloading checked archives
08:54:43.218 INFO  Downloading extensions
08:54:44.015 INFO  Starting analysis
08:54:44.030 WARN  At least one of `minSeverity` and `minCriticality` should to be defined. Defaulting to the obsolete behavior of reporting all potentially breaking elements.
Analysis results
----------------

Old API: json-20230227.jar
New API: json-20230618.jar
old: class org.json.JSONMLParserConfiguration
new: class org.json.JSONMLParserConfiguration
java.class.nonFinalClassInheritsFromNewClass: Non-final class now inherits from 'org.json.ParserConfiguration'.
BINARY: POTENTIALLY_BREAKING, SOURCE: POTENTIALLY_BREAKING

old: method void org.json.JSONMLParserConfiguration::<init>(boolean, int)
new: method void org.json.JSONMLParserConfiguration::<init>(boolean, int)
java.method.visibilityIncreased: visibility increased
BINARY: EQUIVALENT, SOURCE: POTENTIALLY_BREAKING

old: class org.json.XMLParserConfiguration
new: class org.json.XMLParserConfiguration
java.class.nonFinalClassInheritsFromNewClass: Non-final class now inherits from 'org.json.ParserConfiguration'.
BINARY: POTENTIALLY_BREAKING, SOURCE: POTENTIALLY_BREAKING

08:54:45.338 INFO  Analysis took 1323ms.
wtwhite commented 11 months ago

Reporting everything POTENTIALLY_BREAKING drops the number of jar pairs with no results from 289 to 127. Also running with make -j 3 only saved 6 minutes 😢

wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ nohup time make MINSEVERITY=POTENTIALLY_BREAKING -f ../../Makefile.revapi --output-sync -j 3 &
wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ tail -2 nohup.out 
35341.31user 1694.68system 1:24:39elapsed 729%CPU (0avgtext+0avgdata 2194556maxresident)k
0inputs+4893144outputs (17708major+276412498minor)pagefaults 0swaps
wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ find . -name '*.revapi.POTENTIALLY_BREAKING.json'|wc -l
1414
wtwhite@wtwhite-vuw-vm:~/code/jcompile/runs/31_run_30_with_test_jars_stripped$ find . -name '*.revapi.POTENTIALLY_BREAKING.json' -size 2c|wc -l
127