This commit extends the end-to-end tests to cover all currently-supported combinations of Swift version, Linux distribution and CPU architecture. For Ubuntu, there are also tests of SDKs built directly from packages and from container images.
These tests run slowly (about 30 minutes on my machine), so they are disabled unless the SWIFT_SDK_GENERATOR_RUN_SLOW_TESTS environment variable is set.
There is further scope for speeding up the generation of SDKs from container images, which might allow more tests to be included by default. Quite a lot of time, on macOS, is also spent waiting for the toolchain embedded in the SDK to be verified - this is more difficult to mitigate.
Splitting the tests up makes it easier to see problems which affect particular configurations. Each integration test case tries to build two different programs; ideally these would be reported as separate test cases, but for now some manual re-testing is needed to separate the causes of test failures.
Currently there are two pre-existing problems on main which cause tests to fail:
Issue #147 causes ld-linux-aarch64.so.1 not to be copied into SDKs built from containers. Attempted builds using these SDKs fails at the final link stage.
Issue #152 causes new shims not to be copied into all Swift 6.0 SDKs. Attempted builds with these SDKs fail during compilation stage, if Foundation is used.
In some cases issue #152 masks issue #147, because it occurs earlier in the build process. For instance, using ubuntu_aarch64_6.0.2-RELEASE_with-docker to build the 'Foundation' test case currently fails because of #152, but if that were to be fixed it would then still fail because of #147.
euanh
commented
1 day ago
This commit extends the end-to-end tests to cover all currently-supported combinations of Swift version, Linux distribution and CPU architecture. For Ubuntu, there are also tests of SDKs built directly from packages and from container images.
These tests run slowly (about 30 minutes on my machine), so they are disabled unless the SWIFT_SDK_GENERATOR_RUN_SLOW_TESTS environment variable is set.
There is further scope for speeding up the generation of SDKs from container images, which might allow more tests to be included by default. Quite a lot of time, on macOS, is also spent waiting for the toolchain embedded in the SDK to be verified - this is more difficult to mitigate.
Splitting the tests up makes it easier to see problems which affect particular configurations. Each integration test case tries to build two different programs; ideally these would be reported as separate test cases, but for now some manual re-testing is needed to separate the causes of test failures.
Currently there are two pre-existing problems on main which cause tests to fail:
In some cases issue #152 masks issue #147, because it occurs earlier in the build process. For instance, using ubuntu_aarch64_6.0.2-RELEASE_with-docker to build the 'Foundation' test case currently fails because of #152, but if that were to be fixed it would then still fail because of #147.
FAIL1: cannot find /lib/ld-linux-aarch64.so.1 (Issue #147) FAIL2: missing required module '_FoundationCShims' (Issue #152)