I had some problems with the most recent edgelet packages release, and ended up having to update the GitHub release page manually. These changes fix the problems I encountered, as well as follow-on problems I discovered during testing:
builds/misc/packages-release.yaml:
Shortened the repo aliases in the repository resource section because I found that referring to the resource in a variable is problematic when there are dashes in the alias
Added a repository resource ref for the product repo so I could use it in updateLatestVersion.sh later. I used the long form of the branch ref because I have it working that way in other pipelines, probably not strictly necessary.
Shortened the name of the EsrpCodeSigning task, because shorter is better
Added the UseMinimatch property to the EsrpCodeSigning task to ensure the bits are signed (packages are also signed, so not sure how necessary this is)
Moved snapcraft-store-credentials secret into its own key vault task so we can get it from another vault
Added azure_iotedge_repo_branch variable to the safe_guard job in PublishPackagesGithub in order to reference it in updateLatestVersion.sh
Added azure_iotedge_repo_name variable to the safe_guard and linux jobs in PublishPackagesGithub in order to pass it as the --repo-name argument to publishReleasePackages.sh
In the safe_guard job in PublishPackagesGithub, I moved the invocation of publishReleasePackages.sh after the call to update_product_versions_json because I need to value of the new PRODUCT_VERSION variable that is returned from that function to set up the --is-lts argument to publishReleasePackages.sh
Added the pipeline variable service-account.email so we don't need to hard-code the value in updateLatestVersion.sh
Updated the PowerShell task to create the $env:DOWNLOAD_PATH directory before calling DownloadIdentityService.ps1. Not sure why it wasn't needed before, but something in my changes required that I do it now...
Removed variables that are no longer used, e.g., BASE_VERSION
scripts/linux/github/updateLatestVersion.sh:
Refactored version_ge into two functions, version_gt and version_ge, because the warning variable was confusing
Simplified is_major_minor_bump function by taking advantage of the version_ge refactor
Fixed a bug in get_version_from_json where it returned two version values because we currently support 2 LTS versions of IoT Edge
Added a version_is_lts function, which ultimately gets used to append "LTS" to the name of a product in product-versions.json (e.g., "Azure IoT Edge 1.5 LTS")
Combined the update_product_versions_json and github_update_and_push functions since they are always used together anyways, and because github_update_and_push was doing a checkout after we just finished updated/committing product-versions.json, which seems dangerous
Made variables local inside functions (I'd like to start using set -e in these scripts, but that would require lots more testing)
scripts/linux/publishReleasePackages.sh:
Removed some unused parameters, added a few new ones that were needed
Replaced some functionality related to creating the GitHub relase page with calls to existing functions defined in scripts/linux/release-tools.sh (get_project_release_info, create_github_release_page_in_product_repo)
Added *.ddeb files (Ubuntu debug symbol packages) to the list of assets that get uploaded to GitHub release page
Replaced hard-coded references to "Azure/azure-iotedge" with the GH_REPO_NAME variable, to make testing easier
scripts/linux/release-tools.sh:
Updated logic to use double quotes when extracting the CHANGELOG. This is actually related to a bug uncovered during the recent Metrics Collector release, because that CHANGELOG had an apostrophe in it ("What's new in this release?")
In create_github_release_page_in_product_repo, got rid of the unused variable DIAG_VERSION and simplified the remaining CORE_VERSION to just VERSION. Also added RELEASE_ID, which the function sets prior to returning in the success case. scripts/linux/publishReleasePackages.sh makes use of it
Added to create_github_release_page_in_product_repo the ability to create a draft release
To test, I set up a separate pipeline in a test environment (forks of Azure/iotedge, Azure/azure-iotedge, and Azure/iot-identity-service, separate key vault, service connections, etc. I commented out the PublishPackagesMicrosoft stage since I don't know how to easily run it in a test environment, but that stage worked fine in the last release anyway, and this change doesn't really mess with it. I confirmed that the release pipeline:
Builds edgelet for all architectures
Updates product-versions.json in the product repo Azure/azure-iotedge), commits the result, and tags it with the version number
Creates a GitHub Release page in the product repo and uploads all the right assets
Azure IoT Edge PR checklist:
This checklist is used to make sure that common guidelines for a pull request are followed.
[ ] Title of the pull request is clear and informative.
[ ] Description of the pull request includes a concise summary of the enhancement or bug fix.
Testing Guidelines
[ ] Pull request includes test coverage for the included changes.
Description of the pull request includes
[ ] concise summary of tests added/modified
[ ] local testing done.
Draft PRs
Open the PR in Draft mode if it is:
Work in progress or not intended to be merged.
Encountering multiple pipeline failures and working on fixes.
Note: We use the kodiakhq bot to merge PRs once the necessary checks and approvals are in place. When it merges a PR, kodiakhq converts the PR title to the commit title, PR description to the commit description, and squashes all the commits in the PR to a single commit. The net effect is that entire PR becomes a single commit. Please follow the best practices mentioned here for the PR title and description
I had some problems with the most recent edgelet packages release, and ended up having to update the GitHub release page manually. These changes fix the problems I encountered, as well as follow-on problems I discovered during testing:
builds/misc/packages-release.yaml:
snapcraft-store-credentials
secret into its own key vault task so we can get it from another vaultazure_iotedge_repo_branch
variable to thesafe_guard
job inPublishPackagesGithub
in order to reference it in updateLatestVersion.shazure_iotedge_repo_name
variable to thesafe_guard
andlinux
jobs inPublishPackagesGithub
in order to pass it as the--repo-name
argument to publishReleasePackages.shsafe_guard
job inPublishPackagesGithub
, I moved the invocation of publishReleasePackages.sh after the call toupdate_product_versions_json
because I need to value of the new PRODUCT_VERSION variable that is returned from that function to set up the--is-lts
argument to publishReleasePackages.shservice-account.email
so we don't need to hard-code the value in updateLatestVersion.sh$env:DOWNLOAD_PATH
directory before calling DownloadIdentityService.ps1. Not sure why it wasn't needed before, but something in my changes required that I do it now...BASE_VERSION
scripts/linux/github/updateLatestVersion.sh:
version_ge
into two functions,version_gt
andversion_ge
, because the warning variable was confusingis_major_minor_bump
function by taking advantage of theversion_ge
refactorget_version_from_json
where it returned two version values because we currently support 2 LTS versions of IoT Edgeversion_is_lts
function, which ultimately gets used to append "LTS" to the name of a product in product-versions.json (e.g., "Azure IoT Edge 1.5 LTS")update_product_versions_json
andgithub_update_and_push
functions since they are always used together anyways, and becausegithub_update_and_push
was doing a checkout after we just finished updated/committing product-versions.json, which seems dangerousscripts/linux/publishReleasePackages.sh:
get_project_release_info
,create_github_release_page_in_product_repo
)GH_REPO_NAME
variable, to make testing easierscripts/linux/release-tools.sh:
create_github_release_page_in_product_repo
, got rid of the unused variableDIAG_VERSION
and simplified the remainingCORE_VERSION
to justVERSION
. Also addedRELEASE_ID
, which the function sets prior to returning in the success case. scripts/linux/publishReleasePackages.sh makes use of itcreate_github_release_page_in_product_repo
the ability to create a draft releaseTo test, I set up a separate pipeline in a test environment (forks of Azure/iotedge, Azure/azure-iotedge, and Azure/iot-identity-service, separate key vault, service connections, etc. I commented out the
PublishPackagesMicrosoft
stage since I don't know how to easily run it in a test environment, but that stage worked fine in the last release anyway, and this change doesn't really mess with it. I confirmed that the release pipeline:Azure IoT Edge PR checklist:
This checklist is used to make sure that common guidelines for a pull request are followed.
General Guidelines and Best Practices
Testing Guidelines
Draft PRs
Draft
mode if it is:Note: We use the kodiakhq bot to merge PRs once the necessary checks and approvals are in place. When it merges a PR, kodiakhq converts the PR title to the commit title, PR description to the commit description, and squashes all the commits in the PR to a single commit. The net effect is that entire PR becomes a single commit. Please follow the best practices mentioned here for the PR title and description