This guide assumes that an older Swift version is removed when adding a new one. This makes the process less cumbersome, because updating the test expectations in the Server package (of which there are a number) is easier if one just shuffles Swift version numbers around without having to add and remove versions separately.
Steps marked with ⚠️ are the ones that are relevant for deployment and need to be held back until everything is in place.
[x] Open up a tracking issue with the contents of this guide
Note: There's little point in merging this in order to be able to add new packages that have their tools-version set to this new version unless the server changes have been deployed. If the server can't "package dump" this tools version the analysis step will fail and the package will not show up in the index. However, it is safe to merge this ahead of time. Packages that have been added will start appearing once the server has been updated.
new Swift docker image available (automatically ensured by package CI)
This ensures via the CI build that the validator binary which is built in the PackageList repository can be built successfully with the latest Swift version.
[x] Update SwiftVersion+Build.swift and adjust to changes
[x] Update PackageShow.Model.SwiftVersionResults and adjust to changes (outdated)
[x] Udate SwiftVersion.note in SwiftVersion+BuildResultParameter if needed
[x] Ensure any new SupportedPlatform keys are added to our matching Manifest.Platform.Name enum so we don’t fail to decode package manifests that use them. (See issue 1441 for details.)
Preconditions: New server (one that doesn't trigger the Swift version to be deleted) has been deployed
⚠️ Delete outdated builds when ready
[x] dev
[x] prod
Example SQL:
delete
from builds
where
swift_version->'major' = '5'
and swift_version->'minor' = '6'
This should be run once the new Server has been deployed. Running it too early will lead to the deleted versions being triggered again. It's harmless but wastes build resources.
Deleting the records too long after deployment will inhibit triggering of the new Swift versions, because the presence of the old records will make the build result set appear to be complete. This is also harmless, as these will then be triggered once the deletion has happened.
Deployment notes
The final tasks before actual rollout should be:
set BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING
deploy server (by tagging)
merge builder
delete old Swift version builds
The steps should be followed in that order, although deviations are not critical.
Setting the BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING does not take effect until the server is redeployed.
Deploying the new server version will start triggering new Swift version builds. If the builder cannot handle them yet (because it's not been updated), they will remain in the queue and will eventually be discarded.
Updating the builder means it will stop processing the deleted Swift version and add support for a new one. So if the builder were updated first, old Swift versions would still be triggered and remain in the queue until discarded.
Neither is problematic but it's best to simply immediately merge (and therefore deploy) the new builder when the new server comes online.
Finally, deleting the old Swift versions is what really starts the reprocessing. Without this step, only new packages or versions will trigger any builds for the new Swift version.
Roll-back scenario
In case anything goes wrong, rolling back the builder and the server will bring back the previous configuration.
This will however then start re-processing of the just deleted Swift version (if that has already happened). Given that, it's probably not advisable to roll back unless there's a problem that can't be fixed in any other way, or it should be done in conjunction with a db restore to bring back the deleted Swift versions.
Post deployment
Preconditions: all the above steps have been completed, not time critical
Remove obsolete runner tag swift-x.y in builder CI/DC
Reset BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING to 1
Add a new Swift version (major or minor)
This guide assumes that an older Swift version is removed when adding a new one. This makes the process less cumbersome, because updating the test expectations in the Server package (of which there are a number) is easier if one just shuffles Swift version numbers around without having to add and remove versions separately.
Steps marked with ⚠️ are the ones that are relevant for deployment and need to be held back until everything is in place.
Example issue: https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/issues/2890
Update SPIManifest
Preconditions: None.
This can be done at any time. It only takes effect once the adopting packages (Server and Builder) update to this version.
Example change: https://github.com/SwiftPackageIndex/SPIManifest/pull/39/files
Update PackageList
Preconditions:
new Swift docker image available (alternatively, use a nightly toolchain)
[x] Update PackageList https://github.com/SwiftPackageIndex/PackageList/pull/7208
[x] Merge
Note: There's little point in merging this in order to be able to add new packages that have their tools-version set to this new version unless the server changes have been deployed. If the server can't "package dump" this tools version the analysis step will fail and the package will not show up in the index. However, it is safe to merge this ahead of time. Packages that have been added will start appearing once the server has been updated.
Example change: https://github.com/SwiftPackageIndex/PackageList/pull/6352/files
Update PackageList-Validator
Preconditions:
This ensures via the CI build that the
validator
binary which is built in thePackageList
repository can be built successfully with the latest Swift version.Example change: https://github.com/SwiftPackageIndex/PackageList-Validator/pull/61/files
Update spi-images
Preconditions: new Swift docker image available (automatically ensured by package CI)
The images don't need to be tagged, as they are automatically picked up via their
.../spi-images:basic-5.8-latest
tag.Example change: https://gitlab.com/finestructure/spi-images/-/merge_requests/19/diffs
Update builder machines
Preconditions: new Xcode version available
swift-x.y
in builder CI/DCUpdate Builder
Preconditions:
step
Update builder machines
step
Update spi-images
[x] Update to new SPIManifest version and address all issues arising from non-exhaustive
switch
statements.[x] Update
Makefile
to build and test with new Xcode version (if Swift version is a release version)[x] Update
.gitlab-ci-builder.yml
: add/removebuild <platform> <swift-version>
blocks.[x] Update
.gitlab-ci-testmatrix.yml
: add/remove newtest <platform> <swift-version>
jobs[x] ⚠️ Tag &
make commit
when ready to deploy https://gitlab.com/finestructure/swiftpackageindex-builder/-/merge_requests/287[x] ⚠️ Merge when ready to deploy
Example change: https://gitlab.com/finestructure/swiftpackageindex-builder/-/merge_requests/253/diffs
Update spi-base
Preconditions: new Swift docker image available (automatically ensured by package CI)
Example change: https://gitlab.com/finestructure/spi-base/-/merge_requests/24/diffs
Update Server
Preconditions:
step
Update Builder
step
Update spi-base
[x] Update spi-base to newly tagged version https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/pull/3120
[x] Update
SPIManifest
dependency https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/pull/3121[x] Update
SwiftVersion+Build.swift
and adjust to changes[x] Update
PackageShow.Model.SwiftVersionResults
and adjust to changes (outdated)[x] Udate
SwiftVersion.note
inSwiftVersion+BuildResultParameter
if needed[x] Ensure any new
SupportedPlatform
keys are added to our matchingManifest.Platform.Name
enum so we don’t fail to decode package manifests that use them. (See issue 1441 for details.)[x] Merge https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/pull/3121
[x] set
BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING
to 0.8[x] ⚠️ Deploy when ready
Example change: https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/pull/2899/files
Delete old Swift version builds
Preconditions: New server (one that doesn't trigger the Swift version to be deleted) has been deployed
Example SQL:
This should be run once the new Server has been deployed. Running it too early will lead to the deleted versions being triggered again. It's harmless but wastes build resources.
Deleting the records too long after deployment will inhibit triggering of the new Swift versions, because the presence of the old records will make the build result set appear to be complete. This is also harmless, as these will then be triggered once the deletion has happened.
Deployment notes
The final tasks before actual rollout should be:
BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING
The steps should be followed in that order, although deviations are not critical.
Setting the
BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING
does not take effect until the server is redeployed.Deploying the new server version will start triggering new Swift version builds. If the builder cannot handle them yet (because it's not been updated), they will remain in the queue and will eventually be discarded.
Updating the builder means it will stop processing the deleted Swift version and add support for a new one. So if the builder were updated first, old Swift versions would still be triggered and remain in the queue until discarded.
Neither is problematic but it's best to simply immediately merge (and therefore deploy) the new builder when the new server comes online.
Finally, deleting the old Swift versions is what really starts the reprocessing. Without this step, only new packages or versions will trigger any builds for the new Swift version.
Roll-back scenario
In case anything goes wrong, rolling back the builder and the server will bring back the previous configuration.
This will however then start re-processing of the just deleted Swift version (if that has already happened). Given that, it's probably not advisable to roll back unless there's a problem that can't be fixed in any other way, or it should be done in conjunction with a db restore to bring back the deleted Swift versions.
Post deployment
Preconditions: all the above steps have been completed, not time critical
swift-x.y
in builder CI/DCBUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING
to1