search

adoptium / STF

The System Test Framework for executing https://github.com/adoptium/aqa-systemtest
Apache License 2.0
8 stars 35 forks source link

Optimize STF generated paths to keep test paths as short as possible #95

Open Mesbah-Alam opened 3 years ago

Mesbah-Alam commented 3 years ago

Background

Windows platform's MAX_PATH restrictions have sporadically been causing STF based system tests to fail due to generated path lengths > MAX_PATH.

We have already been shortening some system test names as a fix.

However, to avoid such issues in future, we should investigate ways to optimize STF generated paths to keep them as short as possible.

Possible options to consider:

1) Timestamp: When STF based system tests are run coupled with TKG, there are two timestamp values added in the test paths-- one from TKG and one from STF (e.g. F:/Users/jenkins/workspace/Test_openjdk11_j9_sanity.system_x86-64_windows_mixed_cm_Nightly_testList_1/openjdk-tests/\TKG\test_output_16076325439603\MauveSingleInvocationLoadTest_OpenJ9_0/20201210-145632-MauveSingleInvocationLoadTest/generation/generation.st). Could we remove the STF generated timestamp? 1.1) Pro: It gives us test paths with only one timestamp in TKG runs. 1.2) Con: It violates the STF use-case of running tests locally using system test make layer - when the STF generated timestamp is essential to distinguish between multiple runs of a particular test.

2) STF appends the current tests's name after timestamp. This is useful when STF tests are run locally using the make layer, as we then get generated paths such as /tmp/stf/20201215-130158-MixedLoadTest/ - which clearly signify which test they belong to. However, when STF is used coupled with TKG (e.g. in Jenkins builds), we are already in a working directory that contains the test name, so the test folder name under that may not need the test name signifier again (e.g., ../MauveSingleInvocationLoadTest_OpenJ9_0/20201210-145632-MauveSingleInvocationLoadTest - which is unnecessary). Can we simply not append the test name in stf.pl ? 2.1) Pro: This gives us a shorter path in Jenkins runs. 2.2) Con: This means the local run using system test make layer would end up with paths without test name in them (e.g. /tmp/stf/20201215-130158).

@llxia @lumpfish

lumpfish commented 3 years ago

I run locally a lot, and find the current STF output naming system works very well. Isn't the TKG name actually the playlist test target name (as opposed to the STF plugin test name)?

The way this has been fixed before (and the preferable way imo) is to use the Windows subst command to assign a drive letter to the root of the test output directory - e.g.

subst /d "T:" "F:/Users/jenkins/workspace/Test_openjdk11_j9_sanity.system_x86-64_windows_mixed_cm_Nightly_testList_1/openjdk-tests/\TKG\test_output_16076325439603\MauveSingleInvocationLoadTest_OpenJ9_0"

and from that point on use T: to refer to that location.

See https://github.com/AdoptOpenJDK/stf/pull/43 for more history. The subst option appears to have been disabled because of cygwin. Perhaps fixing that issue might be the best solution.

llxia commented 3 years ago

Have we tried mklink? Something like mklink /J myShortDir F:/Users/jenkins/workspace/Test_openjdk11_j9_sanity.system_x86-64_windows_mixed_cm_Nightly_testList_1/