There are several different repositories where we pull test material whose SHAs are not tracked in the TAP file, these include:
[ ] Track perf benchmarks
[ ] Track external test applications
[ ] Track functional test material that is not from the openj9 project (i.e. rh-openjdk repos)
This refers mainly to material pulled in via ant scripts (build.xml files). Any of the test material included in the testenv.properties file gets written to the TAP files.
In addition to tracking test material, we should also check that any dependencies introduced are also tracked / verified against a checksum to ensure it is transparent and clear what is being put onto test machines.
Dependencies (any software that is pulled onto the machine during the test run) include:
prereqs that the Ansible playbooks install / or are defined in Dockerfiles
[ ] Check for any scripts in test material used as part of the AQAvit targets that pull in dependencies opaquely.
These are scripts outside of the mechanisms listed above (i.e. Ansible playbooks, getDependency list), without verification for a checksum, and especially if they are from unofficial or personal branches. We should discourage the use of such scripts as it introduces a level of insecurity that we want to move away from.
smlambert
commented
2 hours ago
There are several different repositories where we pull test material whose SHAs are not tracked in the TAP file, these include:
In addition to tracking test material, we should also check that any dependencies introduced are also tracked / verified against a checksum to ensure it is transparent and clear what is being put onto test machines.
Dependencies (any software that is pulled onto the machine during the test run) include:
prereqs that the Ansible playbooks install / or are defined in Dockerfiles
dependencies pulled in via getDependency jobs
[ ] Check for any scripts in test material used as part of the AQAvit targets that pull in dependencies opaquely.
These are scripts outside of the mechanisms listed above (i.e. Ansible playbooks, getDependency list), without verification for a checksum, and especially if they are from unofficial or personal branches. We should discourage the use of such scripts as it introduces a level of insecurity that we want to move away from.