freddydk
commented
8 months ago Sorry for the delay here. So, you have setup a StorageContext secret for delivering to storage. Does this work during CI/CD? or does it also fail there?
Open pieter-kok opened 8 months ago
freddydk
commented
8 months ago Sorry for the delay here. So, you have setup a StorageContext secret for delivering to storage. Does this work during CI/CD? or does it also fail there?
pieter-kok
commented
8 months ago Thanks for coming back on this topic. It's correct that I have setup a StorageContext secret and this works during CI/CD. Only the release step fails.
freddydk
commented
8 months ago Could you copy/paste the delivery part of CI/CD here? (so I can see the difference) Please format as Code.
pieter-kok
commented
8 months ago Of course. This is the output of the Deliver step of the job Deliver to Storage part of the pull request that was executed just before the failing releases.
[1](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:1)m [2](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:2)0s
Run microsoft/AL-Go-Actions/Deliver@v4.0
with:
shell: powershell
type: CD
projects: ["."]
deliveryTarget: Storage
artifacts: .artifacts
actor: pieter-kok
token: ***
parentTelemetryScopeJson: 7b7d
atypes: Apps,Dependencies,TestApps
goLive: N
env:
workflowDepth: 1
ALGoOrgSettings: {
"CICDPushBranches": [
"main",
"development",
"release/*",
"feature/*"
],
"CICDPullRequestBranches": [
"main",
"development"
],
"ConditionalSettings": [
{
"branches": [
"development"
],
"settings": {
"doNotPublishApps": false,
"doNotSignApps": false
}
}
]
}
ALGoRepoSettings:
Settings: {"keyVaultCertificateUrlSecretName":"","microsoftTelemetryConnectionString":"InstrumentationKey=84bd922[3](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:3)-67d[4](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:4)-4378-8[5](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:5)90-9e4a4[6](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:6)023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/","NextMajorSchedule":"0 1 20 * *","type":"AppSource App","sendExtendedTelemetryToMicrosoft":false,"appBuild":81,"installTestApps":[],"versioningStrategy":0,"installApps":[],"installTestLibraries":false,"runNumberOffset":0,"codeSignCertificatePasswordSecretName":"codeSignCertificatePassword","vsixFile":"","country":"nl","partnerTelemetryConnectionString":"","doNotRunTests":false,"githubRunner":"windows-latest","generateDependencyArtifact":true,"excludeEnvironments":[],"keyVaultCertificatePasswordSecretName":"","rulesetFile":"","licenseFileUrlSecretName":"licenseFileUrl","enableExternalRulesets":false,"memoryLimit":"","keyVaultClientIdSecretName":"","testFolders":[],"doNotBuildTests":false,"environments":[],"companyName":"","assignPremiumPlan":false,"artifact":"","ghTokenWorkflowSecretName":"ghTokenWorkflow","enableTaskScheduler":false,"enableCodeCop":false,"repoVersion":"1.6","doNotPublishApps":false,"unusedALGoSystemFiles":[],"githubRunnerShell":"powershell","appFolders":[],"runs-on":"windows-latest","cacheKeepDays":3,"bcptTestFolders":[],"updateDependencies":false,"cacheImageName":"my","treatTestFailuresAsWarnings":false,"additionalCountries":["be","fi","ro"],"enableUICop":false,"customCodeCops":[],"appDependencies":[],"doNotRunBcptTests":false,"UpdateGitHubGoSystemFilesSchedule":"0 0 1[7](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:7) * *","appDependencyProbingPaths":[{"AuthTokenSecret":"GhTokenWorkflow","release_status":"release","repo":"https://github.com/BluaceBV/appsourcecommon","version":"latest"}],"useCompilerFolder":false,"installTestFramework":false,"keyVaultCodesignCertificateName":"","keyVaultName":"","installOnlyReferencedApps":true,"CICDPushBranches":["main","development","release/*","feature/*"],"obsoleteTagMinAllowedMajorMinor":"","applicationDependency":"1[8](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:8).0.0.0","applicationInsightsConnectionStringSecretName":"applicationInsightsConnectionString","shell":"powershell","failOn":"error","templateUrl":"https://github.com/microsoft/AL-Go-AppSource@main","projects":[],"projectName":".","testDependencies":[],"appRevision":0,"ConditionalSettings":[{"branches":["development"],"settings":{"doNotPublishApps":false,"doNotSignApps":false}}],"codeSignCertificateUrlSecretName":"codeSignCertificateUrl","configPackages":[],"useProjectDependencies":false,"appSourceCopMandatoryAffixes":["IBLC"],"buildModes":[],"templateBranch":"","NextMinorSchedule":"0 1 1[9](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:9) * *","installPerformanceToolkit":false,"PullRequestTrigger":"pull_request_target","CICDPullRequestBranches":["main","development"],"doNotSignApps":false,"repoName":"appsourceinventorytemplates","fullBuildPatterns":[],"alwaysBuildAllProjects":false,"skipUpgrade":false,"CurrentSchedule":"0 1 18 * *","installTestRunner":false,"adminCenterApiCredentialsSecretName":"adminCenterApiCredentials"}
Secrets: {"StorageContext":"***"}
Run $errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0
$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0
try {
D:\a\_actions\microsoft\AL-Go-Actions\v4.0\Deliver/Deliver.ps1 -actor $ENV:_actor -token $ENV:_token -parentTelemetryScopeJson $ENV:_parentTelemetryScopeJson -projects $ENV:_projects -deliveryTarget $ENV:_deliveryTarget -artifacts $ENV:_artifacts -type $ENV:_type -atypes $ENV:_atypes -goLive ($ENV:_goLive -eq 'Y')
}
catch {
Write-Host "::ERROR::Unexpected error when running action. Error Message: $($_.Exception.Message.Replace("`r",'').Replace("`n",' ')), StackTrace: $($_.ScriptStackTrace.Replace("`r",'').Replace("`n",' <- '))";
exit 1
}
shell: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.EXE -command ". '{0}'"
env:
workflowDepth: 1
ALGoOrgSettings: {
"CICDPushBranches": [
"main",
"development",
"release/*",
"feature/*"
],
"CICDPullRequestBranches": [
"main",
"development"
],
"ConditionalSettings": [
{
"branches": [
"development"
],
"settings": {
"doNotPublishApps": false,
"doNotSignApps": false
}
}
]
}
ALGoRepoSettings:
Settings: {"keyVaultCertificateUrlSecretName":"","microsoftTelemetryConnectionString":"InstrumentationKey=84bd9223-67d4-4378-8590-9e4a46023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/","NextMajorSchedule":"0 1 20 * *","type":"AppSource App","sendExtendedTelemetryToMicrosoft":false,"appBuild":81,"installTestApps":[],"versioningStrategy":0,"installApps":[],"installTestLibraries":false,"runNumberOffset":0,"codeSignCertificatePasswordSecretName":"codeSignCertificatePassword","vsixFile":"","country":"nl","partnerTelemetryConnectionString":"","doNotRunTests":false,"githubRunner":"windows-latest","generateDependencyArtifact":true,"excludeEnvironments":[],"keyVaultCertificatePasswordSecretName":"","rulesetFile":"","licenseFileUrlSecretName":"licenseFileUrl","enableExternalRulesets":false,"memoryLimit":"","keyVaultClientIdSecretName":"","testFolders":[],"doNotBuildTests":false,"environments":[],"companyName":"","assignPremiumPlan":false,"artifact":"","ghTokenWorkflowSecretName":"ghTokenWorkflow","enableTaskScheduler":false,"enableCodeCop":false,"repoVersion":"1.6","doNotPublishApps":false,"unusedALGoSystemFiles":[],"githubRunnerShell":"powershell","appFolders":[],"runs-on":"windows-latest","cacheKeepDays":3,"bcptTestFolders":[],"updateDependencies":false,"cacheImageName":"my","treatTestFailuresAsWarnings":false,"additionalCountries":["be","fi","ro"],"enableUICop":false,"customCodeCops":[],"appDependencies":[],"doNotRunBcptTests":false,"UpdateGitHubGoSystemFilesSchedule":"0 0 [17](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:17) * *","appDependencyProbingPaths":[{"AuthTokenSecret":"GhTokenWorkflow","release_status":"release","repo":"https://github.com/BluaceBV/appsourcecommon","version":"latest"}],"useCompilerFolder":false,"installTestFramework":false,"keyVaultCodesignCertificateName":"","keyVaultName":"","installOnlyReferencedApps":true,"CICDPushBranches":["main","development","release/*","feature/*"],"obsoleteTagMinAllowedMajorMinor":"","applicationDependency":"[18](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:18).0.0.0","applicationInsightsConnectionStringSecretName":"applicationInsightsConnectionString","shell":"powershell","failOn":"error","templateUrl":"https://github.com/microsoft/AL-Go-AppSource@main","projects":[],"projectName":".","testDependencies":[],"appRevision":0,"ConditionalSettings":[{"branches":["development"],"settings":{"doNotPublishApps":false,"doNotSignApps":false}}],"codeSignCertificateUrlSecretName":"codeSignCertificateUrl","configPackages":[],"useProjectDependencies":false,"appSourceCopMandatoryAffixes":["IBLC"],"buildModes":[],"templateBranch":"","NextMinorSchedule":"0 1 [19](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:19) * *","installPerformanceToolkit":false,"PullRequestTrigger":"pull_request_target","CICDPullRequestBranches":["main","development"],"doNotSignApps":false,"repoName":"appsourceinventorytemplates","fullBuildPatterns":[],"alwaysBuildAllProjects":false,"skipUpgrade":false,"CurrentSchedule":"0 1 18 * *","installTestRunner":false,"adminCenterApiCredentialsSecretName":"adminCenterApiCredentials"}
Secrets: {"StorageContext":"***"}
_actor: pieter-kok
_token: ***
_parentTelemetryScopeJson: 7b7d
_projects: ["."]
_deliveryTarget: Storage
_artifacts: .artifacts
_type: CD
_atypes: Apps,Dependencies,TestApps
_goLive: N
Downloading BcContainerHelper latest version from CDN
Import from C:\ProgramData\BcContainerHelper\6.0.1\BcContainerHelper\BcContainerHelper.ps1
BcContainerHelper version 6.0.1
BC.HelperFunctions emits usage statistics telemetry to Microsoft
Applying settings from D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.github\AL-Go-Settings.json
Applying settings from D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.AL-Go\settings.json
No settings found in D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.github\CICD.settings.json
No settings found in D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.AL-Go\CICD.settings.json
No settings found in D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.AL-Go\pieter-kok.settings.json
Artifacts .artifacts
Projects:
.
Project 'appsourceinventorytemplates'
Artifacts:
- appsourceinventorytemplates-main-Apps-1.6.[81](https://github.com/BluaceBV/appsourceinventorytemplates/actions/runs/7132737691/job/19425575660#step:6:82).0
- appsourceinventorytemplates-main-BuildOutput-1.6.81.0
- appsourceinventorytemplates-main-Dependencies-1.6.81.0
- appsourceinventorytemplates-main-TestApps-1.6.81.0
- appsourceinventorytemplates-main-TestResults-1.6.81.0
Azure.Storage Module is available in version 2.1.0
Using Azure.Storage version 2.1.0
WARNING: Microsoft Azure PowerShell collects data about how users use PowerShell cmdlets and some problems they
encounter. Microsoft uses this information to improve our PowerShell cmdlets. Participation is voluntary and when you
choose to participate your device automatically sends information to Microsoft about how you use Azure PowerShell.
If you choose to participate, you can stop at any time by using Azure PowerShell as follows:
1. Use the Disable-AzureDataCollection cmdlet to turn the feature Off. The cmdlet can be found in the
AzureResourceManager module
To disable data collection: PS > Disable-AzureDataCollection
If you choose to not participate, you can enable at any time by using Azure PowerShell as follows:
1. Use the Enable-AzureDataCollection cmdlet to turn the feature On. The cmdlet can be found in the
AzureResourceManager module
To enable data collection: PS > Enable-AzureDataCollection
Select Y to enable data collection [Y/N]:
WARNING: You choose not to participate in Microsoft Azure PowerShell data collection.
WARNING: The setting profile has been saved to the following path 'C:\Users\runneradmin\AppData\Roaming\Windows Azure
Powershell\AzureDataCollectionProfile.json'.
Storage Container Name is appsourceinventorytemplates
Storage Blob Name is {version}/{project}-{type}.zip
Delivering to appsourceinventorytemplates in babuildfileshare
Looking for: appsourceinventorytemplates-main-Apps-*.*.*.*
D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.artifacts\appsourceinventorytemplates-main-Apps-1.6.81.0
Compressing
Delivering 1.6.81.0-preview/appsourceinventorytemplates-apps.zip
Delivering preview/appsourceinventorytemplates-apps.zip
Looking for: appsourceinventorytemplates-main-Dependencies-*.*.*.*
D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.artifacts\appsourceinventorytemplates-main-Dependencies-1.6.81.0
Compressing
Delivering 1.6.81.0-preview/appsourceinventorytemplates-dependencies.zip
Delivering preview/appsourceinventorytemplates-dependencies.zip
Looking for: appsourceinventorytemplates-main-TestApps-*.*.*.*
D:\a\appsourceinventorytemplates\appsourceinventorytemplates\.artifacts\appsourceinventorytemplates-main-TestApps-1.6.81.0
Compressing
Delivering 1.6.81.0-preview/appsourceinventorytemplates-testapps.zip
Delivering preview/appsourceinventorytemplates-testapps.zipAre you using this blob storage account for other things? Did you try again after waiting 7 days (see this https://techcommunity.microsoft.com/t5/azure-paas-blog/troubleshooting-invalidblock-the-specified-block-list-is-invalid/ba-p/1870350)
pieter-kok
commented
7 months ago I'm using this blob storage account also for supplying the development license file and certificate as input for the github actions. So, that's really kind of static data, but it is other use. That is however in a different blob container. Those files are uploaded manually from the Azure portal, so not through some API. Furthermore the storage account is only used for storing the build outputs from AL-Go for GitHub.
When it would be interfering other activities in the storage account that causes this, it makes me wonder why the the storage part of the CI/CD (just before the release action in this case) was working just fine. To my knowledge, the only failed/cancelled uploads are the ones from the failed release actions. Three uploads are done (dependency, app and test app). Once one upload fails, the other steps are cancelled. Is it possible that that is creating uncommitted blocks?
I just created a new release of the product to check the waiting period of 7 days as you requested. It now worked, but I don't really understand what that means now based on the supplied web page and how I can solve the issue within the 7 days when it's happening again.
freddydk
commented
7 months ago I hear about this for the first time - never heard of other people having this problem. Also Business Central artifacts are uploaded to azure blob storage as well, multiple times a day and we have never encountered this. For the limited usage you are describing, you also shouldn't run into the limit described in the blog post - so I really don't know. The CI/CD workflow uploads a preview version - the create release uploads a released version - that is really the main difference. In the future - our recommendation is to use nuget for these things instead of storage accounts.
pieter-kok
commented
6 months ago In our release round this month (last Monday and yesterday) we got this issue on every app that has at least a dependency on an app in another GitHub repository. The only thing we could do was retry the release and eventually the release succeeds, without modifications to the app or changing settings. Some examples of that:
In all situations, one of the upload actions fails and cancelled the other ones. But its not always the same one that fails:
As we've have had many examples of this behaviour in our account, maybe there is some information to collect to solve it.
freddydk
commented
6 months ago Could you try to add a setting to .github/AL-Go-Settings.json like this:
"runs-on": "ubuntu-latest"and then run Update AL-Go System Files and retry?
pieter-kok
commented
6 months ago Thanks for your response. I will do that and let you know the outcome.
pieter-kok
commented
5 months ago During our latest release cycle, after we switched to ubunto-latest, we did not have this issue anymore. Does this mean that the windows runners will not be supported anymore for specific tasks?
philipp-mlr
commented
3 months ago We had the same issue. Setting "runs-on": "ubuntu-latest" fixed it.
freddydk
commented
2 months ago Sorry for the delay on the response here - but this likely means that there is a bug in the AzureRM module, which was installed on the GitHub hosted Windows Runners. We will be (in the next version) switching to use the newer Az PowerShell module always (which is what ubuntu-latest also uses), assuming that this will resolve the issue.
We are working with AL: Go for GitHub in a configuration that we have a separate repository for the dependencies and those are included in the artifacts of the product release. Besides that we publish the result to a storage account.
When running the release action we are blocked by a http 400 error during the 'Deliver to Storage'. This happens on one of the three apps that must be delivered to storage (the dependency, the main app or the test app), the other are skipped due to failure on one of the three.
This happens just after the line: AL-Go action ran: Deliver Telemetry Correlation Id: 69dfd8da-8507-4ff0-96dd-feda57e930e9
Below the complete output of the deliver to storage step. If there is more information required to investigate this issue, please let me know.