Open MortenRa opened 8 months ago
Yeah, that's not easy.
what we do: Step 1 - AppSource Validation this is pretty much the "ALOps App Validation" step.
Step 2 - Upload to AppSource This is the "ALOps AppSource" step
Step 3 - Update master This will update the master branch with the new apps, AND update the AppSourceCop.json THis is the script:
$CollectionURI = "$(System.CollectionUri)"
$ProjectID = "$(System.TeamProjectId)"
$RootDir = "$(System.ArtifactsDirectory)"
$Authorization = "Basic $([Convert]::ToBase64String([System.Text.Encoding]::Default.GetBytes(":$(System.AccessToken)")))"
## Do not run when not enabled
if ("$(UpdateMasterBranches)" -ne "true")
{
break
}
#####################
# Find all apps
#####################
$AppDir = Join-Path -Path $RootDir -ChildPath "Distri Apps"
$PreviousDir = Join-Path $RootDir -ChildPath "AppSourcePrevious"
$SourceFiles = Get-ChildItem -Path $AppDir -Filter "*.app" -Recurse
$PreviousFiles = Get-ChildItem -Path $PreviousDir -Filter "*.app" -Recurse
function Get-TFSFile()
{
param(
[string] $Path,
[string] $RepoID
)
try
{
$TargetUri = "$($CollectionURI)/$($ProjectID)/_apis/git/repositories/($($RepoID))/items?path=$($Path)&api-version=5.1"
Write-Debug "Target Url: [$($TargetUri)]"
return [System.Text.Encoding]::UTF8.GetString((Invoke-WebRequest -UseBasicParsing -Method Get -Uri $TargetUri -ContentType "application/json" -Headers @{ "Authorization" = $Authorization }).Content)
}
catch
{}
return $null
}
function Get-LastCommitID()
{
param(
[string] $RepoID
)
$TargetUri = "$($CollectionURI)/$($ProjectID)/_apis/git/repositories/($($RepoID))/stats/branches?api-version=5.1"
Write-Debug "Target Url: [$($TargetUri)]"
$BranchInfo = (Invoke-RestMethod -Uri $TargetUri -Method Get -Headers @{ "Authorization" = $Authorization }).value
$LastCommitID = ($BranchInfo | Where-Object { $_.name -eq 'master' -or $_.name -eq 'main'}).commit.commitID
if ([String]::IsNullOrEmpty($LastCommitID)) {
$LastCommitID = "0000000000000000000000000000000000000000"
}
return $LastCommitID
}
#####################
# Get Repo's
#####################
$TargetUri = "$($CollectionURI)/$($ProjectID)/_apis/git/repositories?api-version=5.1"
Write-Debug "Target Url: [$($TargetUri)]"
$Repos = (Invoke-RestMethod -Method Get -Uri $TargetUri -Headers @{ "Authorization" = $Authorization }).value
#####################
# Fetch Repo App Info
#####################
$RepoInfo = @()
foreach ($Repo in $Repos) {
$RepoInfo += [PSCustomObject]@{
Name = $Repo.name
Found = $false
AppName = ""
TFSObject = $Repo
}
}
foreach ($item in $RepoInfo) {
$AppJson = Get-TFSFile -Path "App/app.json" -RepoID $item.TFSObject.id
if ($null -ne $AppJson)
{
$item.Found = $true
$item.AppName = ($AppJson | ConvertFrom-Json).name
}
}
#####################
# Link all app files to Repo
#####################
$FindObjects = @()
Foreach ($File in $SourceFiles)
{
$FileInfo = $File.Name.Split('_')
$PreviousFile = $PreviousFiles | Where-Object { $_.Name.Contains("_$($FileInfo[1])_") }
$PreviousFileInfo = @()
if ($null -ne $PreviousFile)
{
$PreviousFileInfo = $PreviousFile.Name.Split('_')
}
$FindObject = [PSCustomObject]@{
Name = $FileInfo[1];
Version = $FileInfo[2];
PreviousVersion = $PreviousFileInfo[2];
Path = $File.FullName;
FileName = $File.Name
PreviousPath = $PreviousFile.FullName;
PreviousFileName = $PreviousFile.Name
Repo = ($RepoInfo | Where-Object {$_.AppName -eq $FileInfo[1]}).TFSObject
}
$FindObjects += $FindObject
}
$FindObjects
Foreach ($File in $FindObjects)
{
Write-Host "*** [$($File.Name)]"
$AppsourceJson = Get-TFSFile -Path "App/AppSourceCop.json" -RepoID $File.Repo.id
if ($null -ne $AppsourceJson)
{
$AppsourceJsonObject = ($AppsourceJson | ConvertFrom-Json)
$CurrentVersion = $AppsourceJsonObject.version
if ($CurrentVersion -eq $File.Version) {
Write-Host -ForegroundColor Green " * No update needed"
continue
}
$fileContentBytes = [System.IO.File]::ReadAllBytes($File.Path)
$fileContentEncoded = [System.Convert]::ToBase64String($fileContentBytes)
if ($null -eq ($AppsourceJsonObject | Get-Member -Name Version))
{
$AppsourceJsonObject | Add-Member -MemberType NoteProperty -Name version -Value ""
}
$AppsourceJsonObject.version = $File.Version
$AppsourceJson = ($AppsourceJsonObject | ConvertTo-Json -Depth 100)
$Changes = @()
if (-not [string]::IsNullOrEmpty($File.PreviousFileName))
{
$Changes += @{
changeType = "delete" ;
item = @{ path = "App/.appSourceCopPackages/$($File.PreviousFileName)" };
}
}
$Changes += @{
changeType = "add" ;
item = @{ path = "App/.appSourceCopPackages/$($File.FileName)" };
newContent = @{
content = "$($fileContentEncoded)";
contentType = "base64encoded"
}
},
@{
changeType = "edit" ;
item = @{ path = "App/AppSourceCop.json" };
newContent = @{
content = $AppsourceJson;
contentType = "rawtext"
}
}
$ChangedFile = @{
refUpdates = @(
@{
name = "refs/heads/master";
oldObjectId = "$(Get-LastCommitID -RepoID $File.Repo.id)"
}
);
commits = @(
@{
comment = "AppSource update to version [$($File.Version)]" ;
changes = $Changes
}
)
}
$TargetUri = "$($CollectionURI)/$($ProjectID)/_apis/git/repositories/($($File.Repo.id))/pushes?api-version=5.1"
Write-Debug "Target Url: [$($TargetUri)]"
$result = Invoke-RestMethod -Method Post -Uri $TargetUri -ContentType "application/json" -Body ($ChangedFile | ConvertTo-Json -Depth 100) -Headers @{ "Authorization" = $Authorization }
}
}
Hope it helps ..
@MortenRa
Can ALops offer any Task in Release Pipeline to use the artifact from the Release to be able to create a branch and update appsourceCop and .previous folder with new app file to be ready for Merge.
This was our initial expectation from ALOPs as well, not only to provide us with the building blocks to configure a fully functioning CI/CD setup for AL projects, but also including some out-of-the-box tooling to smoothen the post-release steps and minimize the friction for the developer as well. (PS: seems not supported by AL-Go on github either)
So, we decided to build something custom. High-level, we adopt below process:
which mainly updates the app.json file, appsource.json file and replaces the just released app in a separate folder, on a separate release branch (tagged) + makes the PR.
All the developer has to do when pulling the latest master changes, is to copy-paste the app file to his central alpackages path and done.
As an inspiration, here's part of the script (not created by a PS expert :( )
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)]
[string]$WorkingDirectory,
[Parameter(Mandatory = $true)]
[string]$ArtifactsDirectory,
[boolean]$UpdatePreviousApps = $true,
[string]$AppSourceVersion,
[boolean]$IncreaseAppVersion = $true,
[string]$AppVersion
)
#Settings
$AppsDirectory = Join-Path $ArtifactsDirectory '\Apps'
# Functions
# Formats JSON in a nicer format than the built-in ConvertTo-Json does.
# https://stackoverflow.com/questions/33145377/how-to-change-tab-width-when-converting-to-json-in-powershell
function Format-Json([Parameter(Mandatory, ValueFromPipeline)][String] $json) {
$indent = 0;
# https://stackoverflow.com/questions/69345030/sign-is-converted-into-u0026-through-powershell
$json = $json -replace '\\u0026', '&' -replace '\\u0027', "'" -replace '\\u003c', '<' -replace '\\u003e', '>'
($json -Split "`n" | % {
if ($_ -match '[\}\]]\s*,?\s*$') {
# This line ends with ] or }, decrement the indentation level
$indent--
}
$line = (' ' * $indent) + $($_.TrimStart() -replace '": (["{[])', '": $1' -replace ': ', ': ')
if ($_ -match '[\{\[]\s*$') {
# This line ends with [ or {, increment the indentation level
$indent++
}
$line
}) -Join "`n"
}
function Update-Dependencies {
param (
[Parameter(Mandatory = $true)]
[string]$Publisher,
[Parameter(Mandatory = $true)]
[string]$Version
)
if ($AppContent.dependencies) {
foreach ($Dependency in $AppContent.dependencies) {
if ($null -ne $Dependency.publisher -and $Dependency.publisher -eq $Publisher) {
Write-Host " Update version to $AppVersion (dependency $($Dependency.name))"
$Dependency.version = $Version
}
}
}
}
function Get-Repositories() {
param (
$ParentFolder
)
Get-ChildItem -Path $ParentFolder -Directory -Depth 1 -Filter '.git' -Force `
| Where-Object { $_.Parent.Name -ne 'Scripts' } `
| ForEach-Object { $_.Parent }
}
$Repositories = (Get-Repositories -ParentFolder $WorkingDirectory) # Should be limited to only one, but this work-around points to the correct folder name for the \App
foreach ($Repository in $Repositories) {
Set-Location -Path $Repository.FullName
git config core.eol lf
git config core.autocrlf input
git config core.safecrlf false
$UserName = (git config user.name)
$UserEmail = (git config user.email)
if (-not $UserName) {
$UserName = $env:BUILD_REQUESTEDFOR
git config user.name "$UserName"
}
if (-not $UserEmail) {
$UserEmail = $env:BUILD_REQUESTEDFOREMAIL
git config user.email "$UserEmail"
}
$AppDirectory = Join-Path $Repository.FullName '\App'
$PackagesDirectory = Join-Path $AppDirectory '\.appSourceCopPackages'
Write-Host "Settings ..." -ForegroundColor Yellow
Write-Host " Repository: $($Repository.Name)"
Write-Host " AppDirectory: $AppDirectory"
Write-Host " PackagesDirectory: $PackagesDirectory"
Write-host " Running under $UserName / $UserEmail"
# Update Previous Apps
if ($UpdatePreviousApps) {
Write-Host "Update Previous Apps" -ForegroundColor Green
Write-Host "Settings ..." -ForegroundColor Yellow
Write-Host " AppSourceVersion: $AppSourceVersion"
# Remove old app
Write-Host "Removing previous Dynavision apps ..." -ForegroundColor Yellow
Write-Host " Directory: $PackagesDirectory"
if (Test-Path -Path $PackagesDirectory) {
Get-ChildItem -Path $PackagesDirectory -Filter 'Dynavision*.app' | ForEach-Object {
Write-Host " Removing $($_.Name)"
Remove-Item -Path $_.FullName -Force
git rm -f "App/.appSourceCopPackages/$($_.Name)" # *> $null
}
}
# Move latest app
Write-Host "Replacing previous Dynavision apps..." -ForegroundColor Yellow
Write-Host " Directory: $AppsDirectory"
if (-not (Test-Path $PackagesDirectory)) {
new-item -Path $PackagesDirectory -ItemType Directory
}
Get-ChildItem -Path $AppsDirectory -Filter 'Dynavision*.app' | Foreach-Object {
Write-Host " Adding $($_.Name)"
Move-item -Path $_.FullName -Destination $PackagesDirectory
git add -f "App/.appSourceCopPackages/$($_.Name)" # *> $null
}
# Update AppSourceCop.json
Write-Host "Updating AppSourceCop.json ..." -ForegroundColor Yellow
Write-Host " Directory: $AppDirectory"
$AppSourceCopFile = Join-Path -Path $AppDirectory -ChildPath 'AppSourceCop.json'
$AppSourceCopContent = Get-Content $AppSourceCopFile -raw | ConvertFrom-Json
$VersionParsed = $null
if ([Version]::TryParse($AppSourceVersion, [ref]$VersionParsed)) {
$AppSourceVersion = $VersionParsed.ToString()
Write-Host " Update version to $AppSourceVersion"
$AppSourceCopContent.version = $AppSourceVersion
$Updated = $true
}
else {
Write-Host " Version ignored."
}
$VersionParsed = $null
if ([Version]::TryParse($MajorMinor, [ref]$VersionParsed)) {
$MajorMinor = $VersionParsed.ToString()
Write-Host " Update obsoleteTagVersion to $MajorMinor"
$AppSourceCopContent.obsoleteTagVersion = $MajorMinor
$Updated = $true
}
else {
Write-Host " obsoleteTagVersion ignored."
}
if ($Updated) {
$AppSourceCopContent | ConvertTo-Json -Depth 32 | Format-Json | Set-Content $AppSourceCopFile
git add "App/.appSourceCopPackages/$($AppSourceCopFile.Name)" # *> $null
}
$Message = "Post release $($AppSourceVersion) -> Update previous apps"
git stage .
git commit -q -m $Message # *> $null
git tag -a $AppSourceVersion -m "Release $AppSourceVersion" # *> $null
}
if ($IncreaseAppVersion) {
$Updated = $false
Write-Host "Increase App Version" -ForegroundColor Green
Write-Host "Settings ..." -ForegroundColor Yellow
Write-Host " AppVersion: $AppVersion"
# Write-Host " PlatformVersion: $PlatformVersion"
# Write-Host " ApplicationVersion: $ApplicationVersion"
# Write-Host " RuntimeVersion: $RuntimeVersion"
Write-Host " Reference: https://dev.azure.com/xxxx/Dynavision/_library?itemType=VariableGroups&view=VariableGroupView&variableGroupId=2&path=PipelineVariables"
# Update App.json version
Write-Host "Updating App.json ..." -ForegroundColor Yellow
Write-Host " Directory: $AppDirectory"
$AppFile = Join-Path -Path $AppDirectory -ChildPath 'App.json'
$AppContent = Get-Content $AppFile -raw | ConvertFrom-Json
$VersionParsed = $null
if ([Version]::TryParse($AppVersion, [ref]$VersionParsed)) {
$AppVersion = $VersionParsed.ToString()
if ($VersionParsed -lt [version]$AppContent.version) {
throw " version $AppVersion should be higher than $($AppContent.version)"
}
Write-Host " Update version to $AppVersion ($($AppContent.name))"
$AppContent.version = $AppVersion
Update-Dependencies -Publisher 'Dynavision' -Version $AppVersion
$Updated = $true
}
else {
Write-Host " Version ignored."
}
# runtime, platform, application
# Update-Dependencies -InputObject $Configuration -Publisher 'Microsoft' -Version $ApplicationVersion
if ($Updated) {
$AppContent | ConvertTo-Json -Depth 32 | Format-Json | Set-Content $AppFile
# git add $AppFile *> $null
}
# Update AppSourceCop.json - obsoleteTagVersion
# $MajorMinor = "$($DestinationVersionParsed.Major).$($DestinationVersionParsed.Minor)"
$Message = "Post release $($AppSourceVersion) -> Increase version to $($AppVersion)"
git stage .
git commit -q -m $Message # *> $null
}
}
Thank you for describing your solution, @fvet .
Too bad to hear we disappoint you in this matter - but do know that it's a very conscious choice to never add steps that "changes your code" or "adds commits to your repo" . That's a box of Pandora that we will keep shut.
I would like to join this discussion. We're also struggling with the same challenge: How do I automate the content of the .appSourceCopPackages folder?
While do don't disagree with "box of Pandora" that @waldo1001 wan't to keep closed, I'm thinking can we take an other approach on this? I'm just thinking out loud here: An ALOps building block that populates an .appSourceCopPackages folder in a local directory of the Agent (or even better: in a Azure Storage), where we ourselves create the logic to commit this into our repo?
We've created something in PowerShell that on every PR updates the content of the .appSourceCopPackages folder. You can see below why it's not a good idea for us to write our own PowerShell 🙈 and have the experts at ALOps provide this service to us. (I was doubting if I would share this monster ugly written PowerShell, please don't judge and and don't use this as you'll probably end up in misery, due to the poor quality of the code).
parameters:
country: 'W1'
suppress_pullrequest: false
steps:
- ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
- task: PowerShell@2
displayName: 'Checkout $(System.PullRequest.SourceBranch)'
env:
SYSTEM_PULLREQUEST_SOURCEBRANCH: $(System.PullRequest.SourceBranch)
inputs:
targetType: 'inline'
script: |
Write-Host "##[command]git fetch"
git fetch
Write-Host "Branch name is $($env:SYSTEM_PULLREQUEST_SOURCEBRANCH.Replace('refs/heads/',''))"
git checkout $env:SYSTEM_PULLREQUEST_SOURCEBRANCH.Replace('refs/heads/','') --
- task: PowerShell@2
displayName: 'Verify AppSourceCop.json'
condition: succeeded()
env:
BUILD_SOURCESDIRECTORY: $(Build.SourcesDirectory)
inputs:
targetType: 'inline'
script: |
$appFolder = (Join-Path $env:BUILD_SOURCESDIRECTORY 'App')
$appSourceCopFile = (Join-Path $appFolder 'AppSourceCop.json')
if (!(Test-Path -PathType Leaf $appSourceCopFile)) {
Write-Host "##[command]Creating $appSourceCopFile"
$json = @{
baselinePackageCachePath = "./.appSourceCopPackages"
mandatoryAffixes = @("AFFIX1", "AFFIX2", "AFFIX3")
supportedCountries = @("COUNTRY1", "COUNTRY2", "COUNTRY2")
}
$json | ConvertTo-Json | Set-Content $appSourceCopFile
}
### Grab the version from the AppSourceCop file
Write-Host "##[command]Read $appSourceCopFile"
$appSourceCopObject = Get-Content -Raw -Path $appSourceCopFile | ConvertFrom-Json
$baselinePackageCachePath = $(Join-Path(Join-Path $env:BUILD_SOURCESDIRECTORY 'App') $($appSourceCopObject.baselinePackageCachePath))
if ($appSourceCopObject.psobject.Properties.name -contains 'baselinePackageCachePath') {
Write-Host "##[command]Found baselinePackageCachePath '$($appSourceCopObject.baselinePackageCachePath)'."
Write-Host "##vso[task.setvariable variable=APPSOURCECOP_BASELINEPACKAGECACHEPATH]$baselinePackageCachePath"
Write-Host "##[command]APPSOURCECOP_BASELINEPACKAGECACHEPATH:$baselinePackageCachePath"
}
else {
Throw "Missing property 'baselinePackageCachePath' in $appSourceCopObject"
}
# Create baselinePackageCachePath if not exits
if (!(Test-Path $baselinePackageCachePath)) {
New-Item -Path $baselinePackageCachePath -ItemType Directory -Force | Out-Null
}
- task: PowerShell@2
displayName: 'Populate baselinePackageCachePath'
condition: succeeded()
env:
AGENT_TEMPDIRECTORY: $(Agent.TempDirectory)
BUILD_SOURCESDIRECTORY: $(Build.SourcesDirectory)
APPSOURCECOP_BASELINEPACKAGECACHEPATH: $(APPSOURCECOP_BASELINEPACKAGECACHEPATH)
COUNTRY: ${{ parameters.country }}
inputs:
targetType: 'inline'
script: |
Import-Module BcContainerHelper -DisableNameChecking
Import-Module D365BCAppHelper -DisableNameChecking
function Get-BusinessCentralArtifactIndex {
Param (
[string] $country = 'W1',
[string] $version = '',
[string] $select = 'Daily',
[switch] $silent
)
if ($country.ToUpper() -eq 'BASE') {
$country = 'W1'
}
if ($version -ne '') {
Write-Host "Get-BcArtifactUrl -type Sandbox -country $country -version $version"
$artifactPaths = Download-Artifacts -artifactUrl (Get-BcArtifactUrl -type Sandbox -country $country -version $version) -includePlatform
}
else {
Write-Host "Get-BcArtifactUrl -type Sandbox -country $country -select $select"
$artifactPaths = Download-Artifacts -artifactUrl (Get-BcArtifactUrl -type Sandbox -country $country -select $select) -includePlatform
}
$artifactAppFilesIndex = @{}
$artifactAppFiles = Get-ChildItem -Path (Join-Path $artifactPaths[0] 'Extensions') -Filter *.app -Recurse
$artifactAppFiles += Get-ChildItem -Path $artifactPaths[1] -Filter *.app -Recurse
$artifactAppFiles | % {
$xmlManifest = Get-D365BCManifestFromAppFile -Filename $_.FullName
if (!$silent) {
Write-Host "**************************************"
Write-Host "$($spaces)* App File = [$($_.FullName)]"
Write-Host "$($spaces)"
Write-Host "$($spaces)* App.ID = $($xmlManifest.App.Id)"
Write-Host "$($spaces)* App.Name = $($xmlManifest.App.Name)"
Write-Host "$($spaces)* App.Publisher = $($xmlManifest.App.Publisher)"
Write-Host "$($spaces)* App.Version = $($xmlManifest.App.Version)"
Write-Host "$($spaces)"
Write-Host "$($spaces)"
}
if (!( $artifactAppFilesIndex.ContainsKey($xmlManifest.App.Id))) {
$artifactAppFilesIndex.Add($xmlManifest.App.Id, $_.FullName)
}
}
$artifactAppFilesIndex
}
function Resolve-BusinessCentralDependenciesFromArtifactIndex {
Param(
[Parameter(Mandatory = $true)]
[hashtable] $artifactIndex,
[Parameter(Mandatory = $true)]
[string] $appsFolder,
[string] $outputFolder = (Join-Path $appsFolder '.appSourceCopPackages'),
[int] $lvl = -1,
[string[]] $ignoredDependencies = @(),
[string[]] $ignoredPublishers = @()
)
if ([String]::IsNullOrWhiteSpace($outputFolder)) {
$outputFolder = (Join-Path $appsFolder '.appSourceCopPackages')
}
try {
$spaces = ''
$lvl++
For ($i = 0; $i -le $lvl; $i++) {
$spaces += ' '
}
# Create outputFolder if not exits
if (!(Test-Path $outputFolder)) {
New-Item -Path $outputFolder -ItemType Directory -Force | Out-Null
}
# Search for app.json(s)
$apps = @(Get-ChildItem -Path (Join-Path $appsFolder '/*/app.json'))
if ($apps.Count -eq 0) {
# Look for a single app.json in root
try {
$apps = @(Get-ChildItem -Path (Join-Path $appsFolder '/app.json') -ErrorAction SilentlyContinue)
}
catch {}
}
if ($apps.Count -eq 0) {
# If no app.json(s) found look for .app files.
$apps = @(Get-ChildItem -Path (Join-Path $appsFolder '*.app'))
}
Write-Host "$($spaces)$($apps.Count) apps found";
$apps | % {
### Prevent retrieving itself, so adding the AppId to the $ignoredDependencies list
if ($_.Extension -eq ".app" -and $lvl -eq 0) {
$xmlManifest = Get-D365BCManifestFromAppFile -Filename $_
$ignoredDependencies += $xmlManifest.App.Id
Write-Host "$($spaces)Added itself '$($xmlManifest.App.Name)' with AppId [$($xmlManifest.App.Id)] to ignored apps";
}
if ($_.Extension -eq ".json" -and $lvl -eq 0) {
$AppJson = Get-Content -Raw -Path $_ | ConvertFrom-Json
if ($AppJson.psobject.Properties.name -contains "id") {
$ignoredDependencies += $AppJson.id
Write-Host "$($spaces)Added itself '$($AppJson.name)' with AppId [$($AppJson.id)] to ignored apps";
}
}
if ($_.Name -eq "app.json") {
$appContext = "$($_.DirectoryName | Split-Path -Leaf)\app.json"
Write-Host "$($spaces)Get dependencies from '$appContext'"
$dependencies = Get-D365BCDependenciesFromJson -Filename $_
}
else {
$appContext = "$($_.Name)"
Write-Host "$($spaces)Get dependencies from '$appContext'"
$dependencies = Get-D365BCDependenciesFromFile -Filename $_
}
$dependencies | % {
if (Test-IsGuid $_.Id) {
if ($_.Publisher -in $ignoredPublishers) {
$ignoredDependencies += $($_)
Write-Host "$($spaces)Publisher $($_.publisher) in ignore list. Added $($_.name) with AppId[$($_.Id)] to ignored apps";
}
if (($_.Id -notin $ignoredDependencies) -and ($artifactIndex.ContainsKey($_.Id))) {
# Create temp folders
$tempAppFolder = Join-Path ((Get-Item -Path $env:AGENT_TEMPDIRECTORY).FullName) ([Guid]::NewGuid().ToString())
$tempAppDependenciesFolder = Join-Path $tempAppFolder 'dependencies'
try {
New-Item -path $tempAppFolder -ItemType Directory -Force | Out-Null
New-Item -path $tempAppDependenciesFolder -ItemType Directory -Force | Out-Null
try {
$tempAppDependencyFolder = Join-Path $tempAppDependenciesFolder $_
New-Item -path $tempAppDependencyFolder -ItemType Directory -Force | Out-Null
Copy-Item -Path $artifactIndex[$_.Id] -Destination $tempAppDependencyFolder -Force
$dep = @(Get-ChildItem -Path (Join-Path $tempAppDependencyFolder '*.app'))
$dep | % {
if (!(Test-Path (Join-Path $outputFolder $_.Name))) {
Copy-Item -Path $_ -Destination $outputFolder -Force
Write-Host "$($spaces)The file '$($_.Name)' Copied to $($outputFolder)"
Resolve-BusinessCentralDependenciesFromArtifactIndex -artifactIndex $artifactIndex -appsFolder $(Split-Path $_) -outputFolder $outputFolder -lvl $lvl -ignoredDependencies $ignoredDependencies -ignoredPublishers $ignoredPublishers
}
else {
Write-Host "$($spaces)The App '$($_.Name)' already exists!"
}
}
}
catch {
Write-Host "$($spaces)Copied 0 .app file(s)"
Write-Host $_
}
}
finally {
Remove-Item -Path $tempAppFolder -Recurse -Force -ErrorAction SilentlyContinue
}
}
else {
Write-Host "$($spaces) App '$($_.Name)' with AppId [$($_.Id)] is not a default App from the bcartifacts.cache"
}
}
else {
Write-Host "$($spaces)No dependencies found in '$appContext'"
}
} #end loop $dependencies
} #end loop $apps
}
catch {
Write-Host $_
}
finally { }
}
function Get-D365BCDependenciesFromJson {
[OutputType([array])]
Param(
[Parameter(Mandatory = $true)]
[string] $Filename
)
$appJson = Get-Content -Raw -Path $Filename | ConvertFrom-Json
$result = @()
if ($appJson.psobject.Properties.name -contains 'platform') {
$result += [pscustomobject] @{ Id = '8874ed3a-0643-4247-9ced-7a7002f7135d'
Name = 'System'
Publisher = 'Microsoft'
MinVersion = $appJson.platform
CompatibilityId = '0.0.0.0'
}
}
if ($appJson.psobject.Properties.name -contains 'application') {
$result += [pscustomobject] @{ Id = 'c1335042-3002-4257-bf8a-75c898ccb1b8'
Name = 'Application'
Publisher = 'Microsoft'
MinVersion = $appJson.application
CompatibilityId = '0.0.0.0'
}
}
### Return empty result when no dependencies property exists
if ($appJson.psobject.Properties.name -notcontains "dependencies") {
Write-Host "No addition dependencies, return application and platform dependencies"
return $result
}
$appJson.dependencies | % {
if ($_.psobject.Properties.name -contains "id") {
$id = $_.id
}
elseif ($_.psobject.Properties.name -contains "appId") {
$id = $_.appId
}
$result += [pscustomobject] @{ Id = $id
Name = $_.name
Publisher = $_.publisher
MinVersion = $_.version
CompatibilityId = '0.0.0.0'
}
}
return $result
}
function Test-IsGuid {
[OutputType([bool])]
param
(
[Parameter(Mandatory = $true)]
[AllowEmptyString()]
[string]$StringGuid
)
$ObjectGuid = [System.Guid]::empty
return [System.Guid]::TryParse($StringGuid, [System.Management.Automation.PSReference]$ObjectGuid) # Returns True if successfully parsed
}
function Prep-BusinessCentralDependenciesFromArtifactIndex {
param
(
[Parameter(Mandatory = $true)]
[string] $outputFolder
)
# If the folder is already there, remove all contents
if ((Get-ChildItem $outputFolder -Force | Select-Object -First 1 | Measure-Object).Count -ne 0) {
Remove-Item -Path $outputFolder -Recurse -Force -ErrorAction SilentlyContinue
New-Item -Path $outputFolder -ItemType Directory -Force | Out-Null
}
}
function Post-BusinessCentralDependenciesFromArtifactIndex {
param
(
[Parameter(Mandatory = $true)]
[string] $outputFolder
)
$microsoftSystemAppFile = Join-Path $outputFolder 'System.app'
if (Test-Path $microsoftSystemAppFile) {
$xmlManifest = Get-D365BCManifestFromAppFile -Filename $microsoftSystemAppFile
Rename-Item -Path $microsoftSystemAppFile -NewName "Microsoft_System_$($xmlManifest.App.Version).app"
}
}
function Get-BusinessCentralApplicationVersionFromAppJson {
[OutputType([string])]
Param (
[Parameter(Mandatory = $true)]
[string] $appFolder
)
Write-Host "Read $(Join-Path $appFolder app.json)"
$AppFileObject = Get-Content -Raw -Path (Join-Path $appFolder app.json) | ConvertFrom-Json
$ApplicationVersion = $null
if ([System.Version]::TryParse($AppFileObject.application, [ref]$ApplicationVersion)) {
Write-Host "##[command]Found Application version $ApplicationVersion in App.json [$($AppFileObject.Name)]"
}
else {
Write-Error -Message "Invalid Application version in App.json" -ErrorAction Stop
}
$PlatformVersion = $null
if ([System.Version]::TryParse($AppFileObject.platform, [ref]$PlatformVersion)) {
Write-Host "##[command]Found Platform version $PlatformVersion in App.json [$($AppFileObject.Name)]"
}
else {
Write-Error -Message "Invalid Platform version in App.json [$($AppFileObject.Name)]" -ErrorAction Stop
}
if ([version]$PlatformVersion -gt [version]$ApplicationVersion) {
Write-Error -Message "The platform version '$($PlatformVersion)' is higher then the application version '$($ApplicationVersion)' in App.json [$($AppFileObject.Name)]" -ErrorAction Stop
}
$version = $ApplicationVersion.Major.ToString() + "." + $ApplicationVersion.Minor.ToString()
return $version
}
Prep-BusinessCentralDependenciesFromArtifactIndex `
-outputFolder $env:APPSOURCECOP_BASELINEPACKAGECACHEPATH
$applicationVersion = Get-BusinessCentralApplicationVersionFromAppJson `
-appFolder $(Join-Path $env:BUILD_SOURCESDIRECTORY 'App')
Resolve-BusinessCentralDependenciesFromArtifactIndex `
-artifactIndex (Get-BusinessCentralArtifactIndex -country $env:COUNTRY -version $applicationVersion -silent) `
-appsFolder $(Join-Path $env:BUILD_SOURCESDIRECTORY 'App') `
-outputFolder $env:APPSOURCECOP_BASELINEPACKAGECACHEPATH
Post-BusinessCentralDependenciesFromArtifactIndex `
-outputFolder $env:APPSOURCECOP_BASELINEPACKAGECACHEPATH
- task: PowerShell@2
displayName: 'Download current version'
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
AGENT_TEMPDIRECTORY: $(Agent.TempDirectory)
BUILD_SOURCESDIRECTORY: $(Build.SourcesDirectory)
APPSOURCECOP_BASELINEPACKAGECACHEPATH: $(APPSOURCECOP_BASELINEPACKAGECACHEPATH)
SYSTEM_COLLECTIONURI: $(System.CollectionUri)
inputs:
targetType: 'inline'
script: |
Import-Module D365BCAppHelper -DisableNameChecking
# Get the AppId from the app.json file
$AppJson = Get-Content -Raw -Path (Join-Path (Join-Path $env:BUILD_SOURCESDIRECTORY 'App') 'app.json') | ConvertFrom-Json
if (!($AppJson.psobject.Properties.name -contains "id")) {
Throw "Can't find id in AppJson"
}
try {
# Create temporary folder
$tempAppFolder = Join-Path ((Get-Item -Path $env:AGENT_TEMPDIRECTORY).FullName) ([Guid]::NewGuid().ToString())
New-Item -path $tempAppFolder -ItemType Directory -Force | Out-Null
# Download the current version
az artifacts universal download `
--organization $env:SYSTEM_COLLECTIONURI `
--feed 'BusinessCentral' `
--name $($AppJson.id) `
--version '*' `
--path $tempAppFolder
$appFile = Get-ChildItem -Path $tempAppFolder -Depth 0 -Filter *.app | Select-Object -First 1 | % { $_.FullName }
if (-Not ([String]::IsNullOrWhiteSpace($appFile))) {
# Grab the version (xmlManifest) from the downloaded $appFile
$xmlManifest = Get-D365BCManifestFromAppFile -Filename $appFile
Write-Host "Copy-Item -Path $appFile -Destination (Join-Path $env:APPSOURCECOP_BASELINEPACKAGECACHEPATH (Split-Path $appFile -leaf)) -Force"
Copy-Item -Path $appFile -Destination (Join-Path $env:APPSOURCECOP_BASELINEPACKAGECACHEPATH (Split-Path $appFile -leaf)) -Force
# Get the AppSourceCop.json file (no check if file exists, in previous step already covered)
$appFolder = (Join-Path $env:BUILD_SOURCESDIRECTORY 'App')
$appSourceCopFile = (Join-Path $appFolder 'AppSourceCop.json')
$appSourceCopObject = Get-Content -Raw -Path $appSourceCopFile | ConvertFrom-Json
if ($appSourceCopObject.PSObject.Properties.name -notcontains 'version') {
$appSourceCopObject | Add-Member -MemberType NoteProperty -Name 'version' -Value $($xmlManifest.App.Version)
}
else {
$appSourceCopObject.version = $($xmlManifest.App.Version)
}
Write-Host "Set version $($appSourceCopObject.version) to $appSourceCopFile"
$appSourceCopObject | ConvertTo-Json | Set-Content $appSourceCopFile
}
}
catch {
Write-Host $_
}
finally {
exit 0
}
### Get possible dependency apps
- template: dependency_artifacts.yml
parameters:
target_folder: $(APPSOURCECOP_BASELINEPACKAGECACHEPATH)
### Commit logic
- ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
- task: PowerShell@2
displayName: 'Create Commit'
env:
APPSOURCECOP_BASELINEPACKAGECACHEPATH: $(APPSOURCECOP_BASELINEPACKAGECACHEPATH)
inputs:
targetType: 'inline'
script: |
$env:GIT_REDIRECT_STDERR = '2>&1' # Do not report git output as error... However error is also only text now
$appFolder = (Join-Path $env:BUILD_SOURCESDIRECTORY 'App')
$appSourceCopFile = (Join-Path $appFolder 'AppSourceCop.json')
$AreChangedAppSourceCopPackagesFiles = $(git status $($env:APPSOURCECOP_BASELINEPACKAGECACHEPATH) --porcelain | Measure-Object | Select-Object -expand Count)
$IsChangedAppSourceCopJsonFile = $(git status $appSourceCopFile --porcelain | Measure-Object | Select-Object -expand Count)
if (($AreChangedAppSourceCopPackagesFiles -gt 0) -or ($IsChangedAppSourceCopJsonFile -gt 0)) {
git config user.email "noreply@vanroey.be"
git config user.name "Azure DevOps"
if ($AreChangedAppSourceCopPackagesFiles -gt 0) {
Write-Host "##[command]git add '$($env:APPSOURCECOP_BASELINEPACKAGECACHEPATH)'"
git add "$($env:APPSOURCECOP_BASELINEPACKAGECACHEPATH)"
}
if ($IsChangedAppSourceCopJsonFile -gt 0) {
Write-Host "##[command]git add '$($appSourceCopFile)'"
git add "$appSourceCopFile"
}
Write-Host "##[command]git commit -m 'Updated files in BaselinePackageCachePath'"
git commit -m 'Updated files in BaselinePackageCachePath'
Write-Host "##[command]git push"
git push
}
else {
Write-Host "##[command]GIT: No untracked files found..."
}
Look, we won't do any git-changes. Don't ask that from us.
What we can do (suggestion): in the compile-step, a "previous_app" location setting which will download it from a step. We will extract version info form it, and put in AppSourceCopj.son. You can feed it simply with a file copy, like the default steps in DevOps: Or any kind of copy you use to save your build artifact to a location that you use to save the previous of your apps.
How does that sound?
Is it possible to have this as a separate step and not integrated with the compile-step? Mostly because we want to add these files after the release has gone through on AppSource and not (re)compile the app again.
The steps that I would expect this to do is: A) Search for a baselinePackageCachePath property in the AppSourceCop.json file. B) Retrieve platform, application and dependencies from app.json file. C) Replace artifact App with the "released" version in .appSourceCopPackages folder D) Replace System, System Application, Application, and Base App files in .appSourceCopPackages folder E) Replace dependencies artifacts in .appSourceCopPackages folder F) Update version property in AppSourceCop.json file from the artifact of step C)
There are some challenges I believe A) How do we determine what the latest release is in AppSource? -- Do we need to provide this as a param? -- Can we utilize the API of the AppSource to retrieve current the published version? B) Artifact of the App -- Receiving the compiled artifact of the App itself could be challenging, due to the different ADO configurations out there. Maybe we should make the jump here to specify a NuGet feed, where the artifacts can be resolved? The benefit of an NuGet feed could also applied for the Microsoft Apps ánd we could retrieve symbols of all other AppSource apps.
I'm open for a discussion how other ALOps users are looking at this.
An additional thought: Currently we handle versioning of the application/platform different from dependencies.
The Platform and Application of the app.json, we match the with Major.Minor version of the artifact. So Application set to 22.3 will place the symbols of the artifact of 22.3 in the .appSourceCopPackages folder.
While the dependencies of the other apps, we target the newest release, not matter what the version of that dependency is set. The idea here is that on installing the App, the latest versions of the dependencies are installed (in case not already there).
We'll see what we can do with the AppSource/PartnerCenter API ..
After a release you new app to appsource, how do you handle updating the new app and appsourcecop.json in your CI/CD setup using alops or extending this with maybe some powershell and how do your developer use that app to check for breaking changes.
Looking for some inspiration for a multi developer environment setup and CI/CD to be updated automatically after release to appsource.
Currently we upload this to a previous folder and update the appsourcecop.json after every release by creating a new branch and merge. but looking into maybe put this as a step into the Release Pipeline.
So what do you do:
How do you update appsource.json after a Release to appsource Do you store the new app in the repo or another storage How do your developers use the a app to make sure not to do breaking changes to you handle this in vs code or in CI/CD (could require some iteration).
Can ALops offer any Task in Release Pipeline to use the artifact from the Release to be able to create a branch and update appsourceCop and .previous folder with new app file to be ready for Merge.