adoptium / aqa-tests

Home of test infrastructure for Adoptium builds
https://adoptium.net/aqavit
Apache License 2.0
128 stars 308 forks source link

Conflicting third-party dependency locations preventing download #5427

Open adamfarley opened 2 months ago

adamfarley commented 2 months ago

Summary When downloading third party dependant jars, we get the jars and their shas from two different places, causing conflict

Details In aqa-tests file JenkinsfileBase, we set a custom URL that jars can be downloaded from.

    def customUrl = "https://${jenkinsDomain}/job/test.getDependency/lastSuccessfulBuild/artifact/"

In TKG file getDependencies.pl, we previously used that custom URL (if set) to tell us where to download the jars from.

    # if customUrl is provided, use customUrl and reset $url and $shaurl
        if ($customUrl ne "") {
            $url = "$customUrl/$fn";
            if (defined $shaurl && $shaurl ne '') {
                $shaurl = "$customUrl/$shafn";
            }
        }

https://github.com/adoptium/TKG/commit/c7cf1991fe1534bfc64f62aedddd0386dcfe764f

However, in a later commit, we removed the line that sets $url to $customUrl, but kept the sha url code. https://github.com/adoptium/TKG/commit/87c75aded4797e78c7c2cce3d502a043a6d465f7

As a result, we now get a jar's sha from the jenkins job "test.getDependency", but we get the actual jar from "dependency_pipeline".

Looks to me like we should re-add the line that adds the custom URL to the $url variable in getDependencies.pl, with an extra tweak to ensure that the artifact folder location is added as well (since test.getDependencies stores e.g. jtreg.jar in /jtreg instead of the artifact root).

Example

16:32:31  downloading https://ci.adoptium.net/job/dependency_pipeline/lastSuccessfulBuild/artifact/jtreg/jtreg5.1-b01.tar.gz
16:32:33  --> file downloaded to /home/jenkins/workspace/Test_openjdk22_hs_extended.perf_riscv64_linux_testList_1/../../testDependency/lib/jtreg_5_1_b01.tar.gz
16:32:33  downloading https://ci.adoptium.net//job/test.getDependency/lastSuccessfulBuild/artifact//jtreg_5_1_b01.tar.gz.sha256sum.txt
16:32:33  ERROR: sha checksum error.
16:32:33  --> file downloaded to /home/jenkins/workspace/Test_openjdk22_hs_extended.perf_riscv64_linux_testList_1/../../testDependency/lib/jtreg_5_1_b01.tar.gz.sha256sum.txt
16:32:33  Expected sha is: fbaf7a59b57bd3df33b6702eae52c8e0df3e315a0173115a5329ee58a9e70be2,
16:32:33  Actual sha is  : c0ad6c8868401176b764ea1d33b9cd1a67b0ea817ef1bc4f6122c74411e35895.
16:32:33  Please delete /home/jenkins/workspace/Test_openjdk22_hs_extended.perf_riscv64_linux_testList_1/../../testDependency/lib/jtreg_5_1_b01.tar.gz and rerun the program!

https://ci.adoptium.net/job/Test_openjdk22_hs_extended.perf_riscv64_linux_testList_1/6/consoleFull

adamfarley commented 2 months ago

Here's the sort of fix I propose. Thoughts?

https://github.com/adoptium/TKG/pull/581