microsoft / AL-Go

The plug-and-play DevOps solution for Business Central app development on GitHub
MIT License
293 stars 124 forks source link

AZURE_CREDENTIALS are wrongly formatted. #182

Closed buzzwick closed 2 years ago

buzzwick commented 2 years ago

Working our way through the setup instructions found here and here

We set up an Azure Key Vault with a secret called "LicenseFileUrl." The key value is the URL to a license file stored in Sharepoint on Teams. I set up a GitHub Secret called AZURE_CREDENTIALS, with a key value that is the URL link to the license file stored on teams. When the Test Current workflow is run, we get

Error: AZURE_CREDENTIALS are wrongly formatted.

in the Read Secrets block.

Can you tell me a) if using a Teams doc storage URL will even work (if not, where are people typically storing a license file?), and b) what would be the correct format for the key value of AZURE_CREDENTIALS?

Thanks.

freddydk commented 2 years ago

It sounds like you are putting the license file url in the AZURE_CREDENTIALS secret? AZURE_CREDENTIALS should be Azure Credentials for accessing the KeyVault. Else you can create a GitHub secret called LicenseFileUrl with your license file url - then it will use that.

buzzwick commented 2 years ago

Working on the AZURE_CREDENTIALS angle; created the LicenseFileUrl secret in GitHub. See next error output below--would this be a permissions/access error in trying to actually fetch the license file? Where do people typically store their license files, such that they will be both secure and available to the GitHub workflow?

Run microsoft/AL-Go-Actions/RunPipeline@v1.5 Run try { D:\a_actions\microsoft\AL-Go-Actions\v1.5\RunPipeline/runpipeline.ps1 -actor 'buzzwick' -token '' -parentTelemetryScopeJson '{"Parameters":{},"Emitted":false,"CorrelationId":"ec61b73a-daff-49ae-bb10-14897f787dfd","ParentId":"","AllParameters":{},"TopId":"ec61b73a-daff-49ae-bb10-14897f787dfd","Name":"AL-Go workflow ran: Current","StartTime":"\/Date(1660740783361)\/","Properties":{"eventId":"DO0101","bcContainerHelperVersion":"3.0.12","isAdministrator":"True","stackTrace":"[\n InitTelemetryScope at TelemetryHelper.ps1: line 212\n CreateScope at TelemetryHelper.psm1: line 44\n WorkflowInitialize.ps1 at WorkflowInitialize.ps1: line 37\n 08322d09-6c85-419d-8cd9-a4324f52618f.ps1 at 08322d09-6c85-419d-8cd9-a4324f52618f.ps1: line 1\n \u003cScriptBlock\u003e at \u003cNo file\u003e\n]","repository":"8A353581BFCE717601C7228C07087827B13AABA901DE75E731B02CCE1F672722","runAttempt":"1","runNumber":"4","runId":"2875450592"},"EventId":"DO0101"}' -project '.' -settingsJson '{"testDependencies":[],"ghTokenWorkflowSecretName":"GhTokenWorkflow","cacheKeepDays":3,"codeSignCertificateUrlSecretName":"CodeSignCertificateUrl","templateUrl":"https://github.com/microsoft/AL-Go-AppSource@main","SendExtendedTelemetryToMicrosoft":false,"appFolders":[],"installTestApps":[],"failOn":"error","templateBranch":"","keyVaultCertificateUrlSecretName":"","repoVersion":"1.0","githubRunner":"windows-latest","keyVaultClientIdSecretName":"","artifact":"////latest","licenseFileUrlSecretName":"LicenseFileUrl","obsoleteTagMinAllowedMajorMinor":"","customCodeCops":[],"storageContextSecretName":"StorageContext","generateDependencyArtifact":false,"runNumberOffset":0,"country":"us","versioningStrategy":15,"installPerformanceToolkit":false,"appSourceCopMandatoryAffixes":["\u003caffix\u003e"],"keyVaultCertificatePasswordSecretName":"","updateDependencies":false,"doNotSignApps":false,"doNotBuildTests":false,"applicationDependency":"18.0.0.0","appRevision":0,"alwaysBuildAllProjects":false,"installOnlyReferencedApps":true,"doNotRunBcptTests":false,"Environments":[],"installTestFramework":false,"applicationInsightsConnectionStringSecretName":"ApplicationInsightsConnectionString","enableCodeCop":false,"appDependencies":[],"MicrosoftTelemetryConnectionString":"InstrumentationKey=84bd9223-67d4-4378-8590-9e4a46023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/","installTestLibraries":false,"memoryLimit":"","bcptTestFolders":[],"repoName":"CAI365BC","additionalCountries":[],"installTestRunner":false,"rulesetFile":"","cacheImageName":"","type":"AppSource App","doNotPublishApps":false,"PartnerTelemetryConnectionString":"","codeSignCertificatePasswordSecretName":"CodeSignCertificatePassword","testFolders":[],"companyName":"","adminCenterApiCredentialsSecretName":"AdminCenterApiCredentials","doNotRunTests":false,"enableUICop":false,"appDependencyProbingPaths":[],"installApps":[],"insiderSasTokenSecretName":"InsiderSasToken","keyVaultName":"","skipUpgrade":false,"appBuild":2147483647}' -secretsJson '{"licenseFileUrl":"","insiderSasToken":"","CodeSignCertificateUrl":"","CodeSignCertificatePassword":"","KeyVaultCertificateUrl":"","KeyVaultCertificatePassword":"","KeyVaultClientId":""}' } catch { Write-Host "::Error::Unexpected error when running action ($($.Exception.Message))"; exit 1 } try { D:\a_actions\microsoft\AL-Go-Actions\v1.5\RunPipeline/runpipeline.ps1 -actor 'buzzwick' -token '' -parentTelemetryScopeJson '{"Parameters":{},"Emitted":false,"CorrelationId":"ec61b73a-daff-49ae-bb10-14897f787dfd","ParentId":"","AllParameters":{},"TopId":"ec61b73a-daff-49ae-bb10-14897f787dfd","Name":"AL-Go workflow ran: Current","StartTime":"\/Date(1660740783361)\/","Properties":{"eventId":"DO0101","bcContainerHelperVersion":"3.0.12","isAdministrator":"True","stackTrace":"[\n InitTelemetryScope at TelemetryHelper.ps1: line 212\n CreateScope at TelemetryHelper.psm1: line 44\n WorkflowInitialize.ps1 at WorkflowInitialize.ps1: line 37\n 08322d09-6c85-419d-8cd9-a4324f52618f.ps1 at 08322d09-6c85-419d-8cd9-a4324f52618f.ps1: line 1\n \u003cScriptBlock\u003e at \u003cNo file\u003e\n]","repository":"8A353581BFCE717601C7228C07087827B13AABA901DE75E731B02CCE1F672722","runAttempt":"1","runNumber":"4","runId":"2875450592"},"EventId":"DO0101"}' -project '.' -settingsJson '{"testDependencies":[],"ghTokenWorkflowSecretName":"GhTokenWorkflow","cacheKeepDays":3,"codeSignCertificateUrlSecretName":"CodeSignCertificateUrl","templateUrl":"https://github.com/microsoft/AL-Go-AppSource@main","SendExtendedTelemetryToMicrosoft":false,"appFolders":[],"installTestApps":[],"failOn":"error","templateBranch":"","keyVaultCertificateUrlSecretName":"","repoVersion":"1.0","githubRunner":"windows-latest","keyVaultClientIdSecretName":"","artifact":"////latest","licenseFileUrlSecretName":"LicenseFileUrl","obsoleteTagMinAllowedMajorMinor":"","customCodeCops":[],"storageContextSecretName":"StorageContext","generateDependencyArtifact":false,"runNumberOffset":0,"country":"us","versioningStrategy":15,"installPerformanceToolkit":false,"appSourceCopMandatoryAffixes":["\u003caffix\u003e"],"keyVaultCertificatePasswordSecretName":"","updateDependencies":false,"doNotSignApps":false,"doNotBuildTests":false,"applicationDependency":"18.0.0.0","appRevision":0,"alwaysBuildAllProjects":false,"installOnlyReferencedApps":true,"doNotRunBcptTests":false,"Environments":[],"installTestFramework":false,"applicationInsightsConnectionStringSecretName":"ApplicationInsightsConnectionString","enableCodeCop":false,"appDependencies":[],"MicrosoftTelemetryConnectionString":"InstrumentationKey=84bd9223-67d4-4378-8590-9e4a46023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/","installTestLibraries":false,"memoryLimit":"","bcptTestFolders":[],"repoName":"CAI365BC","additionalCountries":[],"installTestRunner":false,"rulesetFile":"","cacheImageName":"","type":"AppSource App","doNotPublishApps":false,"PartnerTelemetryConnectionString":"","codeSignCertificatePasswordSecretName":"CodeSignCertificatePassword","testFolders":[],"companyName":"","adminCenterApiCredentialsSecretName":"AdminCenterApiCredentials","doNotRunTests":false,"enableUICop":false,"appDependencyProbingPaths":[],"installApps":[],"insiderSasTokenSecretName":"InsiderSasToken","keyVaultName":"","skipUpgrade":false,"appBuild":2147483647}' -secretsJson '{"licenseFileUrl":"","insiderSasToken":"","CodeSignCertificateUrl":"","CodeSignCertificatePassword":"","KeyVaultCertificateUrl":"","KeyVaultCertificatePassword":"","KeyVaultClientId":""}' } catch { Write-Host "::Error::Unexpected error when running action ($($.Exception.Message))"; exit 1 } shell: C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.EXE -command ". '{0}'" env: type: AppSource App country: us artifact: ////latest companyName: repoVersion: 1.0 repoName: CAI365BC versioningStrategy: 15 runNumberOffset: 0 appBuild: 2147483647 appRevision: 0 keyVaultName: licenseFileUrlSecretName: LicenseFileUrl insiderSasTokenSecretName: InsiderSasToken ghTokenWorkflowSecretName: GhTokenWorkflow adminCenterApiCredentialsSecretName: AdminCenterApiCredentials applicationInsightsConnectionStringSecretName: ApplicationInsightsConnectionString keyVaultCertificateUrlSecretName: keyVaultCertificatePasswordSecretName: keyVaultClientIdSecretName: codeSignCertificateUrlSecretName: CodeSignCertificateUrl codeSignCertificatePasswordSecretName: CodeSignCertificatePassword storageContextSecretName: StorageContext additionalCountries: appDependencies: appFolders: testDependencies: testFolders: bcptTestFolders: installApps: installTestApps: installOnlyReferencedApps: True generateDependencyArtifact: False skipUpgrade: False applicationDependency: 18.0.0.0 updateDependencies: False installTestRunner: False installTestFramework: False installTestLibraries: False installPerformanceToolkit: False enableCodeCop: False enableUICop: False customCodeCops: failOn: error rulesetFile: doNotBuildTests: False doNotRunTests: False doNotRunBcptTests: False doNotPublishApps: False doNotSignApps: False appSourceCopMandatoryAffixes: obsoleteTagMinAllowedMajorMinor: memoryLimit: templateUrl: https://github.com/microsoft/AL-Go-AppSource@main templateBranch: appDependencyProbingPaths: githubRunner: windows-latest cacheImageName: cacheKeepDays: 3 alwaysBuildAllProjects: False MicrosoftTelemetryConnectionString: InstrumentationKey=84bd9223-67d4-4378-8590-9e4a46023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/ PartnerTelemetryConnectionString: SendExtendedTelemetryToMicrosoft: False Environments: Settings: {"testDependencies":[],"ghTokenWorkflowSecretName":"GhTokenWorkflow","cacheKeepDays":3,"codeSignCertificateUrlSecretName":"CodeSignCertificateUrl","templateUrl":"https://github.com/microsoft/AL-Go-AppSource@main","SendExtendedTelemetryToMicrosoft":false,"appFolders":[],"installTestApps":[],"failOn":"error","templateBranch":"","keyVaultCertificateUrlSecretName":"","repoVersion":"1.0","githubRunner":"windows-latest","keyVaultClientIdSecretName":"","artifact":"////latest","licenseFileUrlSecretName":"LicenseFileUrl","obsoleteTagMinAllowedMajorMinor":"","customCodeCops":[],"storageContextSecretName":"StorageContext","generateDependencyArtifact":false,"runNumberOffset":0,"country":"us","versioningStrategy":15,"installPerformanceToolkit":false,"appSourceCopMandatoryAffixes":["\u003caffix\u003e"],"keyVaultCertificatePasswordSecretName":"","updateDependencies":false,"doNotSignApps":false,"doNotBuildTests":false,"applicationDependency":"18.0.0.0","appRevision":0,"alwaysBuildAllProjects":false,"installOnlyReferencedApps":true,"doNotRunBcptTests":false,"Environments":[],"installTestFramework":false,"applicationInsightsConnectionStringSecretName":"ApplicationInsightsConnectionString","enableCodeCop":false,"appDependencies":[],"MicrosoftTelemetryConnectionString":"InstrumentationKey=84bd9223-67d4-4378-8590-9e4a46023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/","installTestLibraries":false,"memoryLimit":"","bcptTestFolders":[],"repoName":"CAI365BC","additionalCountries":[],"installTestRunner":false,"rulesetFile":"","cacheImageName":"","type":"AppSource App","doNotPublishApps":false,"PartnerTelemetryConnectionString":"","codeSignCertificatePasswordSecretName":"CodeSignCertificatePassword","testFolders":[],"companyName":"","adminCenterApiCredentialsSecretName":"AdminCenterApiCredentials","doNotRunTests":false,"enableUICop":false,"appDependencyProbingPaths":[],"installApps":[],"insiderSasTokenSecretName":"InsiderSasToken","keyVaultName":"","skipUpgrade":false,"appBuild":2147483647} LicenseFileUrl: RepoSecrets: {"licenseFileUrl":"","insiderSasToken":"","CodeSignCertificateUrl":"","CodeSignCertificatePassword":"","KeyVaultCertificateUrl":"","KeyVaultCertificatePassword":"","KeyVaultClientId":""} Downloading BcContainerHelper latest version BcContainerHelper version 3.0.12 BcContainerHelper emits usage statistics telemetry to Microsoft use settings and secrets Analyzing repository Analyzing releases https://api.github.com/repos/CAI-Software-Inc/CAI365BC/releases Warning: No previous release found Invoke Run-AlPipeline Parameters Pulling Generic Image Creating Container Removing Container Run-AlPipeline Telemetry Correlation Id: f8b13e1d-838b-49c5-b213-81301b4a15c8 Error: Exception calling "DownloadFile" with "2" argument(s): "The remote server returned an error: (403) Forbidden." 403 FORBIDDEN AL-Go action ran: RunPipeline Telemetry Correlation Id: 7cb86327-4300-4af4-947a-d5ba80d5398c Removing BcContainerHelper Error: Process completed with exit code 1.

buzzwick commented 2 years ago

Returning to the AZURE_CREDENTIALS secret setup:, it seems we need answers to these questions:

  1. How does GitHub know where to look for our Azure Vault? We have defined three environment variables: image Are these sufficient for GitHub to know where to look?
  2. What is the correct format of Azure credentials to be stored in the AZURE_CREDENTIALS GitHub secret? And whose credentials must they be? Thanks.
freddydk commented 2 years ago

AZURE_CREDENTIALS needs to contain and be formatted as specified here: https://docs.microsoft.com/en-us/azure/developer/github/github-key-vault (which is a pre-requisite for https://github.com/microsoft/AL-Go/blob/main/Scenarios/UseAzureKeyVault.md) It needs to be compressed JSON - it should not contain spaces and line feeds. Beside that, you can add keyVaultName to the AZURE_CREDENTIALS or to the AL-Go settings file.

FrankAuwers commented 12 months ago

The link https://docs.microsoft.com/en-us/azure/developer/github/github-key-vault is redirected to https://learn.microsoft.com/en-us/azure/developer/github, but I can not find the page where it describe how to setup the key vault for access from GitHub. Can you update this to the correct link?

freddydk commented 12 months ago

I think this is the link: https://learn.microsoft.com/en-us/azure/developer/github/connect-from-azure?tabs=azure-portal%2Cwindows#use-the-azure-login-action-with-a-service-principal-secret Remember that the AZURE_CREDENTIALS needs to be compressed JSON (no spaces nor newlines)

freddydk commented 12 months ago

Will update the docs once I have tested this Also want to try with the OpenID approach to see if that works