I finally implemented dynamic arguments, so instead of --nvr being hardcoded, now you can only use it if you declare it first with --build-config-find "changelog.xml:nvr:/build/nvr" for example (but you can define as many arguments as you want).
All the build filtering is now done using the ConfigFinder class, as I intended earlier. The finding of the job result (SUCCESS, UNSTABLE, ABORTED etc.) is now also done using the ConfigFinderClass. The default place the tool is looking in is the build.xml file and /build/result xpath, however the user can set it to whatever they want, using the --build-config-find argument, like so: --build-config-find "myConfig.properties:result:buildResult".
I also implemented a parsing of JSON config files into the ConfigFinder class.
Last thing, I updated the help message:
Test Variant Comparator Usage: java -cp <classpaths> <mainclass> [arguments...]
Where mainclass is:
io.jenkins.plugins.report.jtreg.main.comparator.VariantComparator
And arguments include:
--path <path/to/jenkins/jobs>
A system path to a directory with your jenkins jobs. This argument is mandatory
You can choose one of these to filter the jobs by their name:
--query <querystring>
Filtering of the jobs by a query string (the syntax is described below).
Query string syntax:
The tool splits every job name by . or - characters and compares each
split part with the query string. The query string consists of N parts
separated by spaces (or other whitespace) and each of these parts
corresponds with 1st..Nth part of the split job name.
Example with explanation:
"jtreg~full jp17 * {f37,el8} !aarch64 !{fastdebug,slowdebug} * * *"
jtreg~full - specifies that the job's first part should be exactly
jtreg~full.
jp17 - specifies that the job's second part should be exactly jp17.
* - asterisk is a powerful wildcard that matches everything, so in
this example, the job's parts on the 3rd, 7th, 8th and 9th don't
matter = the tool takes everything on these positions.
To stop the tool from draining system resources by looking at
all jobs, if more than half of the query elements are asterisks,
or the query has less than 4 parts, you must combine it with the
"--force" switch.
{f37,el8} - this is a set of possible matches, so the jobs's part on
4th position can be either f37 or el8. There can me as
many elements as you want, but they must be split by
commas with no spaces between them.
!aarch64 - matches everything, BUT aarch64.
!{fastdebug,slowdebug} - matches everything, but the elements in
the set.
--exact-length <number>
Meant to be used in combination with --query. It filters only the jobs that have this specified length (number of elements in its name).
--regex <regex>
A regular expression to filter the jobs by their names.
You need to choose ONE operation from these:
--print
Print all jobs and their builds that match the rest of arguments, without actually doing any operation on the builds or tests.
--enumerate
Print lists of all variants of jobs (that match the rest of arguments).
--list
Print a table of matched jobs with their builds and the tests that failed in the build.
--compare
Print a table of all failed tests (of matched job builds) and the builds where they failed.
--virtual
Print a table of all matched jobs' builds and their result (e.g. SUCCESS, UNSTABLE, etc.). Can be used as standalone operation or combined with any other operation. Probably should be run with --skip-failed=false switch.
Other arguments:
--help
Print this help message.
--skip-failed <true/false>
Specify whether the comparator should skip failed tests (only take successful and unstable) or take all. The default value is true.
--history <number>
To specify the maximum number of builds to look in.
--only-volatile <true/false>
Specify true to show only non stable tests with the arguments list and compare (shows only tests, that are NOT failed everywhere). The default value is false.
--force
Used for forcing vague requests, that could potentially take a long time.
--formatting <plain/color/html>
Specify the output formatting (plain, color or html). The default is plain.
--exact-tests <regex>
Specify (with regex) the exact tests to show only. The rest of tests will be ignored.
--use-default-build <true/false>
If set to true and no matching build with given criteria was found, the tool will use the latest (default) build instead. Default value is false.
Dynamic arguments:
Another type of arguments you can use are dynamic arguments. They are used
for further filtering the jobs/builds to compare by any value in their config
files.
First, you must define the value to look for in a config file. The general syntax is:
--X "configFileName:whatAreYouLookingFor:queryToFindIt"
Instead of --X, you use one of these arguments:
--build-config-find configFileName:whatAreYouLookingFor:queryToFindIt
Argument used for declaring dynamic arguments. Looks for the specified config file inside the BUILD directory.
--job-config-find configFileName:whatAreYouLookingFor:queryToFindIt
Argument used for declaring dynamic arguments. Looks for the specified config file inside the JOB directory.
configFileName is the name of the config file in the chosen working directory.
whatAreYouLookingFor is the name of the config value you are looking for.
You later use this name as an argument for the filtering itself.
queryToFindIt is the query, to find the value in the config file. Currently,
XPath is supported for XML files, Json Query for JSON files and plain value
for properties files.
After defining the value, you can proceed to the filtering itself. For that, you
use the whatAreYouLooking part from the definition as an argument and this syntax:
It either takes RegEx to match the value with or multiple RegExes
in curly brackets, separated by commas. (e.g. {nvr1.*,nvr2.*})
Example:
--build-config-find "changelog.xml:nvr:/build/nvr"
Then you can use this argument: --nvr "java-17-openjdk.*" to filter only builds,
that have changelog.xml file in the build directory, and the value on XPath /build/nvr
matches java-17-openjdk.*.
Additional info:
When taking the build results, the tool defaults to using the results from build.xml
file with /build/result XPath, but you can set it to different value with
--build-config-findargument. (The whatAreYouLookingFor part of the syntax
should be set to "result".)
Now it is without conflicts
I finally implemented dynamic arguments, so instead of --nvr being hardcoded, now you can only use it if you declare it first with
--build-config-find "changelog.xml:nvr:/build/nvr"
for example (but you can define as many arguments as you want).All the build filtering is now done using the ConfigFinder class, as I intended earlier. The finding of the job result (SUCCESS, UNSTABLE, ABORTED etc.) is now also done using the ConfigFinderClass. The default place the tool is looking in is the build.xml file and /build/result xpath, however the user can set it to whatever they want, using the --build-config-find argument, like so:
--build-config-find "myConfig.properties:result:buildResult"
.I also implemented a parsing of JSON config files into the ConfigFinder class.
Last thing, I updated the help message:
What do you think?