Implemented dynamic arguments and JSON support

[patrikcerbak]

I finally implemented dynamic arguments, so instead of --nvr being hardcoded, now you can only use it if you declare it first with --build-config-find "changelog.xml:nvr:/build/nvr" for example (but you can define as many arguments as you want).

All the build filtering is now done using the ConfigFinder class, as I intended earlier. The finding of the job result (SUCCESS, UNSTABLE, ABORTED etc.) is now also done using the ConfigFinderClass. The default place the tool is looking in is the build.xml file and /build/result xpath, however the user can set it to whatever they want, using the --build-config-find argument, like so: --build-config-find "myConfig.properties:result:buildResult".

I also implemented a parsing of JSON config files into the ConfigFinder class.

Last thing, I updated the help message:

Test Variant Comparator Usage: java -cp <classpaths> <mainclass> [arguments...]

Where mainclass is:
    io.jenkins.plugins.report.jtreg.main.comparator.VariantComparator

And arguments include:

    --path <path/to/jenkins/jobs>
                  A system path to a directory with your jenkins jobs. This argument is mandatory

    You can choose one of these to filter the jobs by their name:
    --query <querystring>
                  Filtering of the jobs by a query string (the syntax is described below).
    Query string syntax:
    The tool splits every job name by . or - characters and compares each
    split part with the query string. The query string consists of N parts
    separated by spaces (or other whitespace) and each of these parts
    corresponds with 1st..Nth part of the split job name.
    Example with explanation:
     "jtreg~full jp17 * {f37,el8} !aarch64 !{fastdebug,slowdebug} * * *"
        jtreg~full - specifies that the job's first part should be exactly
                     jtreg~full.
        jp17 - specifies that the job's second part should be exactly jp17.
        * - asterisk is a powerful wildcard that matches everything, so in
            this example, the job's parts on the 3rd, 7th, 8th and 9th don't
            matter = the tool takes everything on these positions.
            To stop the tool from draining system resources by looking at
            all jobs, if more than half of the query elements are asterisks,
            or the query has less than 4 parts, you must combine it with the
            "--force" switch.
        {f37,el8} - this is a set of possible matches, so the jobs's part on
                    4th position can be either f37 or el8. There can me as
                    many elements as you want, but they must be split by
                    commas with no spaces between them.
        !aarch64 - matches everything, BUT aarch64.
        !{fastdebug,slowdebug} - matches everything, but the elements in
                                 the set.

            --exact-length <number>
                          Meant to be used in combination with --query. It filters only the jobs that have this specified length (number of elements in its name).
    --regex <regex>
                  A regular expression to filter the jobs by their names.

    You need to choose ONE operation from these:
    --print
                  Print all jobs and their builds that match the rest of arguments, without actually doing any operation on the builds or tests.
    --enumerate
                  Print lists of all variants of jobs (that match the rest of arguments).
    --list
                  Print a table of matched jobs with their builds and the tests that failed in the build.
    --compare
                  Print a table of all failed tests (of matched job builds) and the builds where they failed.
    --virtual
                  Print a table of all matched jobs' builds and their result (e.g. SUCCESS, UNSTABLE, etc.). Can be used as standalone operation or combined with any other operation. Probably should be run with --skip-failed=false switch.

    Other arguments:
    --help
                  Print this help message.
    --skip-failed <true/false>
                  Specify whether the comparator should skip failed tests (only take successful and unstable) or take all. The default value is true.
    --history <number>
                  To specify the maximum number of builds to look in.
    --only-volatile <true/false>
                  Specify true to show only non stable tests with the arguments list and compare (shows only tests, that are NOT failed everywhere). The default value is false.
    --force
                  Used for forcing vague requests, that could potentially take a long time.
    --formatting <plain/color/html>
                  Specify the output formatting (plain, color or html). The default is plain.
    --exact-tests <regex>
                  Specify (with regex) the exact tests to show only. The rest of tests will be ignored.
    --use-default-build <true/false>
                  If set to true and no matching build with given criteria was found, the tool will use the latest (default) build instead. Default value is false.

    Dynamic arguments:
        Another type of arguments you can use are dynamic arguments. They are used
        for further filtering the jobs/builds to compare by any value in their config
        files.
        First, you must define the value to look for in a config file. The general syntax is:
          --X "configFileName:whatAreYouLookingFor:queryToFindIt"
            Instead of --X, you use one of these arguments:
    --build-config-find configFileName:whatAreYouLookingFor:queryToFindIt
                  Argument used for declaring dynamic arguments. Looks for the specified config file inside the BUILD directory.
    --job-config-find configFileName:whatAreYouLookingFor:queryToFindIt
                  Argument used for declaring dynamic arguments. Looks for the specified config file inside the JOB directory.
            configFileName is the name of the config file in the chosen working directory.
            whatAreYouLookingFor is the name of the config value you are looking for.
                You later use this name as an argument for the filtering itself.
            queryToFindIt is the query, to find the value in the config file. Currently,
                XPath is supported for XML files, Json Query for JSON files and plain value
                for properties files.
        After defining the value, you can proceed to the filtering itself. For that, you
        use the whatAreYouLooking part from the definition as an argument and this syntax:
            It either takes RegEx to match the value with or multiple RegExes
            in curly brackets, separated by commas. (e.g. {nvr1.*,nvr2.*})
        Example:
        --build-config-find "changelog.xml:nvr:/build/nvr"
        Then you can use this argument: --nvr "java-17-openjdk.*" to filter only builds,
        that have changelog.xml file in the build directory, and the value on XPath /build/nvr
        matches java-17-openjdk.*.
    Additional info:
        When taking the build results, the tool defaults to using the results from build.xml
        file with /build/result XPath, but you can set it to different value with
        --build-config-findargument. (The whatAreYouLookingFor part of the syntax
        should be set to "result".)

What do you think?

[judovana]

the help looks great. The only missing part is that queries are XOR -or not? That you can youse only one, but you must use at least one. Or you can really use both?

[judovana]

break for now. Will continue on 6f44fa53c97775ad627c5281e2bbb6d1330b1060 after it

[judovana]

it loks great. It is necessary to fix the consumed exceptiosn (not easy here!!). Good luck!

[patrikcerbak]

The consumed exceptions are now logged (printing the stack trace) and it gives the user a warning that an exception occurred.

[judovana]

LGT

[judovana]

surpassed by https://github.com/jenkinsci/report-jtreg-plugin/pull/19