adoptium / infrastructure

This repo contains all information about machine maintenance.
Apache License 2.0
84 stars 100 forks source link

Add template for tests on new platform/distribution environments #3600

Open sxa opened 2 weeks ago

sxa commented 2 weeks ago

As per https://github.com/adoptium/infrastructure/blob/master/FAQ.md#adding-new-systems we have a processs whereby new types of machines (New OS versions, new distributions etc.) Are run through an AQA_Test_Pipeline run.

It would be good if we had a standard template for logging the results of this e.g.

Key:

Ubuntu 24.04 / JDK17 / aarch64

Test suite Result ✅⚠️❌
sanity.functional
extended.functional
special.functional ❌ (List of failing cases + link to new issue)
sanity.openjdk ⚠️ (IssueLink1, Issuelink2)
extended.openjdk ❌ (List of:failing cases + link to new issue/s)
sanity.system
extended.system
sanity.perf
extended.perf

The table could be extended with other columns if desired / appropriate e.g.

Ubuntu 24.04

Test suite JDK Arch Result ✅⚠️❌
sanity.openjdk 17 riscv64
extended.openjdk 17 riscv64 ❌ (New failing issue links)
sanity.openjdk 21 riscv64
extended.openjdk 21 riscv64 ❌ (New failing issue links)
sanity.openjdk 8 riscv64 ⚠️ (IssueLink1, Issuelink2)
extended.openjdk 8 riscv64 ❌ (New failing issue links)
sxa commented 2 weeks ago

Another option could be to have the AQA_Test_Pipeline job disaplay a suitable markdown table at the end that can be cut and pasted with the links and job status into the relevant issue.

Noting that if we do this it should be in a defined order so the results across tables are easier to compare. The order in which they are initiated in the AQA_Test_Pipeline log does not always map to the order that jenkins schedules them on the machines.