BC moving from Docker images to 'BC Artifacts'

[fvet]

What kind of changes should we expect from ALOps when reading through https://freddysblog.com/2020/06/25/changing-the-way-you-run-business-central-in-docker/

• What will be the impact on our current pipelines? (we're especially using fixed build / nextminor / nextmajor images)
• Will ALOps provide some documentation / instructions on how to update pipelines / implement the new artifacts?
• Any ALOps timing / planning (esp. since the insider builds would only ship till end of July?)

Can we expect some ALOps improvements on the 'compile' part (if we would not be interested in publishing / running tests as part of the pipeline)? (https://tobiasfenster.io/making-use-of-the-new-bc-artifacts)

[waldo1001]

Hi Frédéric,

we are working on it.

We will try to make the experience as seemless as at all possible. But it has to make sense. So many people using ALOps in so many different ways .. .

By the way ... you can already use this: just create your images and refer to those images in the pipeline. Only, we want to make an extra step to be able to create images based on artifact urls .. that's more user friendly.

Do know there is not an immediate expected advantage. In a way, we need to do an extra step (building an image) to get exactly the same like we were able to download.. .

More coming very soon!

[kasperdj]

Hi @waldo1001

Microsoft has already stopped releasing new images at mcr.microsoft.com/businesscentral As I understand ALOps can already utilize BC Artifacts based your previous comments. I would like to test this out and get an overview of the pipeline updates needed for this.

In one of my pipelines I use the DK image for BC16.3:

name: 'dockerimage' value: 'mcr.microsoft.com/businesscentral/onprem:1910-rtm-dk-ltsc2019'

This works fine, but if I want to work on e.g. BC16.4 I need to use BC Artificats. Can you provide yaml configuration examples for using BC Artifacts for using:

1. BC16.4 DK
2. BC16 latest DK version (latest minor)
3. BC16 next minor DK version
4. Upcoming next major DK version

Thx. in advance.

[waldo1001]

Hi Kasper,

What I would do, is create a pipeline to create all images on a host, and schedule it overnight. The pipeline could look something like this:

name: Build Docker Images

pool: WaldoHetzner

variables:
  - group: Secrets
  - name: DockerImageName.current
    value: bccurrent
  - name: DockerImageName.insider
    value: bcinsider

steps:
# Update BcContainerHelper
- task: PowerShell@2
  displayName: Install/Update BcContainerHelper
  inputs:
    targetType: 'inline'
    script: |
      [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor [System.Net.SecurityProtocolType]::Tls12
      install-module BcContainerHelper -verbose -force
      Import-Module bccontainerhelper

# W1
- task: PowerShell@2
  displayName: Creating W1 Image
  inputs:
    targetType: 'inline'
    script: |
      $artifactUrl = Get-BCArtifactUrl
      New-BcImage -artifactUrl $artifactUrl -imageName $(DockerImageName.current):w1-latest

# Belgium
- task: PowerShell@2
  displayName: Creating BE Image
  inputs:
    targetType: 'inline'
    script: |
      $artifactUrl = Get-BCArtifactUrl -country be
      New-BcImage -artifactUrl $artifactUrl -imageName $(DockerImageName.current):be-latest

# Belgium - Insider 
- task: PowerShell@2
  displayName: Creating BE Image (insider)
  inputs:
    targetType: 'inline'
    script: |
      $artifactUrl = Get-bcartifacturl -country be -storageAccount bcinsider -sasToken "$(bc.insider.sasToken)"
      New-BcImage -artifactUrl $artifactUrl -imageName $(DockerImageName.insider):be-latest

# Images
- task: PowerShell@2
  displayName: Docker Images Info
  inputs:
    targetType: 'inline'
    script: |
      docker images

To get to the exact artifacts you're after, you can do this:

1. BC16.4 DK: Get-BCArtifactUrl -country dk -version 16.4
2. BC16 latest DK version (latest minor): Get-BCArtifactUrl -country dk -select Latest
3. BC16 next minor DK version: Get-BCArtifactUrl -country dk -select SecondToLastMajor -storageAccount bcinsider -sasToken "<thecurrentsastoken>"
4. Upcoming next major DK version: Get-BCArtifactUrl -country dk -select Latest -storageAccount bcinsider -sasToken "<thecurrentsastoken>"

We have ideas to make it much easier with ALOps to work with Artifacts, but as said, we need this month to finish that.. . IN the meantime, the above yaml does work.

[AdminHodor]

Dear @fvet , @kasperdj ,

The Artifact story covered in the next release (expected 23/8 ~ 24/8). Took some time, but we managed to get a bunch of functionality/logic hidden behind a very small understandable parameter set.

We support creating Docker images from NAV v9.0 to BC v17.0 with the new task "ALOpsDockerCreate". The task includes:

• Creating BC Docker images on the local repository
• Creating Docker Images for NAV: 2016 (v9), 2017 (v10), 2018 (v11)
• Creating Docker Images for BC: v13, v14, v15, v16
• Creating Docker Images for BC-Insider: v17 (SASToken / StorageAccount)
• Pull/Push images with public/private registry.
• Supports a "Hint", where the old name of docker images can be supplied, we'll try and figure out the correct new parameters
• The new task should be inserted in the pipeline above "ALOpsDockerStart"

Some Examples:

Most basic Image creation based possible, only based on the old container naming.

- task: ALOpsDockerCreate@1
  inputs:
    artifacthint: 'mcr.microsoft.com/businesscentral/onprem:16.2.13509.13779-be-ltsc2019'

The proper, new way of defining the version/type/country you want can be done very easy:

- task: ALOpsDockerCreate@1
  inputs:
    artifactversion: '16.2'
    artifactcountry: 'be'
    artifacttype: OnPrem

Also not all parameters are required, if for example "Version" is not specified, we automatically take the latest. Example below will (today) resolve to "onprem-16.4.14693.15445-be"

- task: ALOpsDockerCreate@1
  inputs:
    artifactcountry: 'be'

For BC-Insider you just have to specify the [sastoken] parameter, if [StorageAccount] is blank we assume "bcinsider". Do of course make sure to set the type to "Sandbox" as there are no OnPrem builds in insider.

- task: ALOpsDockerCreate@1
  inputs:
    artifactcountry: 'be'
    artifacttype: Sandbox
    sastoken: $(bcinsider_sas_token)

To work with docker registries, specify the [dockerregistry] parameter, if its secured additionally specify [dockerusername] and [dockerpassword]. Tested on Docker registries [onprem/hosted] and Microsoft Container Registry (Azure)

- task: ALOpsDockerCreate@1
  inputs:
    artifactversion: '16.2'
    artifactcountry: 'be'
    artifacttype: OnPrem
    dockerregistry: 'alops.azurecr.io'
    dockerusername: $(dtr_username)
    dockerpassword: $(dtr_password)

When using the new "ALOpsDockerCreate" in a pipeline, you have to clear the [docker_image] parameter on ALOpsDockerStart. ALOpsDockerStart with automatically pickup the image-name from the the ALOpsDockerCreate task.

- task: ALOpsDockerStart@1
  inputs:
    docker_pull: false
    memory_gb: 8
    docker_parameters: '--isolation=hyperv'

Please monitor this issue, we'll let you know when the release is online.

Kind regards.

[kasperdj]

Hi @AdminHodor

That sounds really good 👍 Can you please confirm the following:

• Has the new version of ALOps been released already and if not, will be please notify us when it is released?
• Is there a dependency on the BCContainerHelper from Freddy that should be installed on the Build Agents?
• How is the build time using BC artifacts vs old style image from MS, e.g. on BC16.2 - I am trying to identify if we can skip setting up our own Docker Host for handling the images and just update our YAML to utilize the BC Artifacts feature in ALOps?

[waldo1001]

Hi @kasperdj,

it's not released yet - we're finetuning and working on the documentation. We will notify you! ALOps will take care of any dependencies, so you don't need to install the BCContainerHelper yourself. It takes about 8 minutes to build an image - but not every pipeline run: it will reuse an image if possible. We built in a caching system that also looks at the windows build: so if you update your windows, it will rebuild the image (basically).

[AdminHodor]

We just release v1.433.1659, can you check if this works for you?

Kind regards,

[pri-kise]

I receiv an error on the Docker Create Task: I guess that this is caused by navcontainerhelper which is already installed on the build agent?

[error]Die folgenden Befehle sind in diesem System bereits verfügbar: Add-FontsToBCContainer,Backup-BCContainerDatabases,Compile-AppInBCContainer,Copy-CompanyInBCContainer,Copy-FileFromBCContainer,Copy-FileToBCContainer,Create-AadAppsForBC,Create-AadUsersInBCContainer,Create-AlProjectFolderFromBcContainer,Enter-BCContainer,Export-BCContainerDatabasesAsBacpac,Extract-FilesFromBCContainerImage,Extract-FilesFromStoppedBCContainer,Get-BCContainerApiCompanyId,Get-BCContainerApp,Get-BCContainerAppInfo,Get-BCContainerAppRuntimePackage,Get-BCContainerBCUser,Get-BCContainerCountry,Get-BCContainerDebugInfo,Get-BCContainerEula,Get-BCContainerEventLog,Get-BCContainerGenericTag,Get-BCContainerId,Get-BCContainerImageLabels,Get-BCContainerImageName,Get-BCContainerImageTags,Get-BCContainerIpAddress,Get-BCContainerLegal,Get-BCContainerName,Get-BCContainerNavVersion,Get-BCContainerOsVersion,Get-BCContainerPath,Get-BCContainerPlatformVersion,Get-BCContainers,Get-BCContainerServerConfiguration,Get-BCContainerSession,Get-BCContainerSharedFolders,Get-BCContainerTenants,Get-BestBCContainerImageName,Get-CompanyInBCContainer,Get-TestsFromBCContainer,Import-BCContainerLicense,Import-ConfigPackageInBCContainer,Import-PfxCertificateToBCContainer,Import-TestToolkitToBCContainer,Install-BCContainerApp,Install-NAVSipCryptoProviderFromBCContainer,Invoke-BCContainerApi,Invoke-ScriptInBCContainer,New-BCContainer,New-BCContainerBCUser,New-BCContainerTenant,New-BCContainerWindowsUser,New-CompanyInBCContainer,Open-BCContainer,Publish-BCContainerApp,Publish-NewApplicationToBcContainer,Remove-BCContainer,Remove-BCContainerSession,Remove-BCContainerTenant,Remove-CompanyInBCContainer,Remove-ConfigPackageInBCContainer,Repair-BCContainerApp,Replace-BCServerContainer,Restart-BCContainer,Restore-DatabasesInBCContainer,Run-TestsInBCContainer,Setup-BCContainerTestUsers,Setup-TraefikContainerForBCContainers,Sign-BCContainerApp,Start-BCContainer,Start-BCContainerAppDataUpgrade,Stop-BCContainer,Sync-BCContainerApp,Test-BCContainer,UnInstall-BCContainerApp,UnPublish-BCContainerApp,Wait-BCContainerReady,Add-FontsToNavContainer,Add-GitToAlProjectFolder,Backup-NavContainerDatabases,Check-NavContainerHelperPermissions,Clean-BcContainerDatabase,Compile-AppInNavContainer,Compile-ObjectsInNavContainer,Convert-AlcOutputToAzureDevOps,Convert-ModifiedObjectsToAl,Convert-Txt2Al,Copy-AlSourceFiles,Copy-CompanyInNavContainer,Copy-FileFromNavContainer,Copy-FileToNavContainer,Create-AadAppsForNav,Create-AadUsersInNavContainer,Create-AlProjectFolderFromNavContainer,Create-MyDeltaFolder,Create-MyOriginalFolder,Download-File,Enter-NavContainer,Export-ModifiedObjectsAsDeltas,Export-NavContainerDatabasesAsBacpac,Export-NavContainerObjects,Extract-AppFileToFolder,Extract-FilesFromNavContainerImage,Extract-FilesFromStoppedNavContainer,Flush-ContainerHelperCache,Generate-SymbolsInNavContainer,Get-BestGenericImageName,Get-BestNavContainerImageName,Get-CompanyInNavContainer,Get-LatestAlLanguageExtensionUrl,Get-LocaleFromCountry,Get-NavContainerApiCompanyId,Get-NavContainerApp,Get-NavContainerAppInfo,Get-NavContainerAppRuntimePackage,Get-NavContainerCountry,Get-NavContainerDebugInfo,Get-NavContainerEula,Get-NavContainerEventLog,Get-NavContainerGenericTag,Get-NavContainerId,Get-NavContainerImageLabels,Get-NavContainerImageName,Get-NavContainerImageTags,Get-NavContainerIpAddress,Get-NavContainerLegal,Get-NavContainerName,Get-NavContainerNavUser,Get-NavContainerNavVersion,Get-NavContainerOsVersion,Get-NavContainerPath,Get-NavContainerPlatformVersion,Get-NavContainers,Get-NavContainerServerConfiguration,Get-NavContainerSession,Get-NavContainerSharedFolders,Get-NavContainerTenants,Get-NavVersionFromVersionInfo,Get-TestsFromNavContainer,Import-ConfigPackageInNavContainer,Import-DeltasToNavContainer,Import-NavContainerLicense,Import-ObjectsToNavContainer,Import-PfxCertificateToNavContainer,Import-TestToolkitToNavContainer,Install-NavContainerApp,Install-NAVSipCryptoProviderFromNavContainer,Invoke-NavContainerApi,Invoke-NavContainerCodeunit,Invoke-ScriptInNavContainer,New-CompanyInNavContainer,New-DesktopShortcut,New-LetsEncryptCertificate,New-NavContainer,New-NavContai

[waldo1001]

You need BcContainerHelper latest version - ALOps will install it - but having both NAVContainerHelper and BcContainerHelper on there, is not a good idea.

[pri-kise]

I expected that ALOps would uninstall NavContainerHelper when you mentioned that it would take care of the dependencies. I guess manual calling Uninstall-Module -Name NavContainerHelper on the BuildAgents should do it?

[waldo1001]

We will not uninstall modules we don't use .. . We never supported or used NAVContainerhelper. However, we do use BcContainerHelper - and we will maintain that.

[pri-kise]

Okay, thanks. Solved it.

Now one other strange behavior: Sometimes I receive an error on the end of the task ALOpsDockerWait:

Error that cannot be handled by Windows PowerShell. A remote session may have been terminated.

*** Initiate Docker Session
*** Exception creating session: Fehler, der nicht von Windows PowerShell behandelt werden kann. Eine Remotesitzung wurde möglicherweise beendet.
*** Reinitializing!
##[error]Fehler, der nicht von Windows PowerShell behandelt werden kann. Eine Remotesitzung wurde möglicherweise beendet.
Finishing: ALOpsDockerWait

[kasperdj]

Hi @waldo1001

I have update my yaml to this:

`#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

• task: ALOpsDockerCreate@1 inputs: artifactversion: '16.2' artifactcountry: 'dk' artifacttype: OnPrem

  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

• task: ALOpsDockerStart@1 inputs: docker_pull: false accept_image_eula: true accept_image_outdated: true docker_parameters: '--isolation=hyperv' memory_gb: 16

  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

• task: ALOpsDockerWait@1 inputs: search_string: 'Ready for connections!'

  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<`

The ALOpsDockerCreate is successfull - only comment is initial run took 23 minutes - hopefully this will improve with the caching mechanism, but the ALOpsDockerStart fails with the following error:

Error response from daemon: hcsshim::CreateComputeSystem 5bd2956d7d5a479fe3cd1713292980f1c420af5d0a12ed9132ebb35d7f1b53d0: The paging file is too small for this operation to complete.

Please advice!

[kasperdj]

I think this might be a disk issue - will investigate and return with an update

[kasperdj]

Weee, after solving the disk issue, my pipeline is all green and the yaml update took 2 minutes to refactor, good job :-)

[AdminHodor]

Weee, after solving the disk issue, my pipeline is all green and the yaml update took 2 minutes to refactor, good job :-) Thanks for the feedback, it took some time to get the Artifacts-Update out, but we had to make sure transition/migration was super-easy. Glad you are up-and-running :-)