Error Compiling Business Central App through Azure Pipelines

[mohamedkathman]

ForEach-Object : Exception calling "DownloadFile" with "2" argument(s): "The operation has timed out" At C:\Program Files\WindowsPowerShell\Modules\navcontainerhelper\0.6.4.9\AppHandling\Compile-AppInNavContainer.ps1:256 char:21

• $dependencies | ForEach-Object {

• 
  + CategoryInfo          : NotSpecified: (:) [ForEach-Object], MethodInvocationException
  + FullyQualifiedErrorId : WebException,Microsoft.PowerShell.Commands.ForEachObjectCommand
  PowerShell exited with code '1'.

[freddydk]

You probably need to supply some additional information.

Hent Outlook til iOShttps://aka.ms/o0ukef

---

Fra: Mohamed Athman notifications@github.com Sendt: Tuesday, October 8, 2019 8:43:27 AM Til: microsoft/navcontainerhelper navcontainerhelper@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Emne: [microsoft/navcontainerhelper] Error Compiling Business Central App through Azure Pipelines (#654)

ForEach-Object : Exception calling "DownloadFile" with "2" argument(s): "The operation has timed out" At C:\Program Files\WindowsPowerShell\Modules\navcontainerhelper\0.6.4.9\AppHandling\Compile-AppInNavContainer.ps1:256 char:21

*

$dependencies | ForEach-Object {

*

            ~~~~~~~~~~~~~~~~

 *   CategoryInfo : NotSpecified: (:) [ForEach-Object], MethodInvocationException
 *   FullyQualifiedErrorId : WebException,Microsoft.PowerShell.Commands.ForEachObjectCommand

PowerShell exited with code '1'.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fnavcontainerhelper%2Fissues%2F654%3Femail_source%3Dnotifications%26email_token%3DACSGUA7RVCEGLT3RX2LQAMTQNQTY7A5CNFSM4I6NUPOKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HQIJFOQ&data=02%7C01%7Cfreddy.kristiansen%40microsoft.com%7Cc18b95fdd4c2489a670108d74bbad3c8%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637061138109862324&sdata=QaKHr5aLDM6aXi%2FAw1hJXDZ%2FB6%2FVrqqrXPc8unbkFoM%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACSGUAYX5G7ODYQEM2MQZBTQNQTY7ANCNFSM4I6NUPOA&data=02%7C01%7Cfreddy.kristiansen%40microsoft.com%7Cc18b95fdd4c2489a670108d74bbad3c8%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637061138109862324&sdata=i6ij06L6uihtvw0xZNeGYPxX0UEvCDo%2BtT%2Fc66pG3UU%3D&reserved=0.

[mohamedkathman]

Hi freddydk

I was actually following your ci/cd workshop document and i've created a self hosted vm. It creates the helloworld-ci container well. I tried to login directly to this container and i could download symbols manually but for some reason i get the said error on compilation.

I've attached the logs.

Quick background

• The build server firewall is off.
• All ports are open

Let me know if you need any further information

Thanks

[freddydk]

If you open the Compile-app - it will probably show some more information?

[mohamedkathman]

freddyk,

This is the extra information on compile app..

[freddydk]

Could you display the base\app.json?

[mohamedkathman]

freddyk,

Here it is...

[freddydk]

Very very strange - cannot see any reason why this times out. On the build machine, can you ping 172.24.86.246? When you download the symbols manually - how did you do that?

[mohamedkathman]

freddydk,

Am able to ping 172.24.86.246. See screenshot. I have vscode installed directly in the build machine and i can download symbols. See screenshot.

[freddydk]

and if you ping helloworld-ci - it resolves to the same IP number???

[mohamedkathman]

it resolves well to the ip....baffling

[Spycoclown]

I had this problem sometimes, after restart the pipeline it worked. Mostly it happen when i just edit the pipeline.

[mohamedkathman]

Hi Spycoclown,

Thanks for this insight...let me test it then revert

[freddydk]

We had 30+ people follow the workshop at Directions, none ran into this issue. Let me know if this issue still persists

[freddydk]

I think I know what is wrong here. Try to run Flush-ContainerHelperCache on the machine and run the pipeline again.

[mohamedkathman]

HI Freddy,

So i flashed the container and tried to run the pipeline but was getting some error as follows

so i opted to create a new container - helloworld but unfortunately still getting the same error even within this new container. Strangely enough i can download symbols directly on vscode from the build machine. See screenshot.

[mohamedkathman]

Update:

Ive tried running the link for downloading symbols manually using container ip and container name. For container ip it just goes round and round but for container name i get an authorisation error. Not sure if this points to anything

[freddydk]

Something is open, which disallows to removal of the container. Restart the computer, run Flush-ContainerHelpercache - run the pipeline. Copy/Paste the text - and not the images of the output (Create Container and the failing step if any)

[mohamedkathman]

Create build container is failing as before...after restarting the computer, flushing container cache then running the pipeline

[section]Starting: Create Build Container

============================================================================== Task : PowerShell Description : Run a PowerShell script on Linux, macOS, or Windows Version : 2.151.2 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell

Generating script. Formatted command: . 'C:\agent_work\2\s\scripts\Create-Container.ps1' ========================== Starting Command Output ===========================

[command]"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'C:\agent_work_temp\fff0a812-8310-40dd-b980-7fcca1873f8a.ps1'"

Create helloworld from mcr.microsoft.com/businesscentral/onprem:1910-rtm-w1 Database backup C:\ProgramData\NavContainerHelper\onprem-15.0.36560.0-W1-bakFolders\helloworld\database.bak not found at Restore, : line 13 at , : line 57 NavContainerHelper is version 0.6.4.10 Host is Microsoft Windows Server 2019 Standard - ltsc2019 Docker Client Version is 19.03.2 Docker Server Version is 19.03.2 Using image mcr.microsoft.com/businesscentral/onprem:1910-rtm-w1-ltsc2019 Disabling Health Check (always report healthy) Removing container helloworld Removing helloworld from hosts Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 1) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 2) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 3) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 4) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 5) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 6) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 7) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 8) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 9) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Error removing C:\ProgramData\NavContainerHelper\Extensions\helloworld (attempts: 10) Please close any apps, prompts or files using this folder Retrying in 10 seconds Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld

[error]Could not remove C:\ProgramData\NavContainerHelper\Extensions\helloworld

At C:\Program Files\WindowsPowerShell\Modules\navcontainerhelper\0.6.4.10\ContainerHandling\Remove-NavContainer.ps1:52 char:21

• throw "Could not remove $containerFolder"

• 
  
  + CategoryInfo          : OperationStopped: (Could not remov...ions\helloworld:String) [], RuntimeException
  
  + FullyQualifiedErrorId : Could not remove C:\ProgramData\NavContainerHelper\Extensions\helloworld
  ##[error]PowerShell exited with code '1'.
  ##[section]Finishing: Create Build Container

[freddydk]

As it states - you have something open, which locks a file or folder in C:\ProgramData\NavContainerHelper\Extensions\helloworld a command prompt, powershell prompt, windows explorer, event viewer or like.

This has nothing to do with the pipeline problem.

[mohamedkathman]

OK, let me check. Ill update shortly

[freddydk]

If you restart - and don't open anything - before running the pipeline - you should be good.

[mohamedkathman]

OK, let me do that again and see

[mohamedkathman]

So the build has run and filed on the compile app.

[section]Starting: Compile App

============================================================================== Task : PowerShell Description : Run a PowerShell script on Linux, macOS, or Windows Version : 2.151.2 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell

Generating script. Formatted command: . 'C:\agent_work\2\s\scripts\Compile-App.ps1' -appFolders "app,base" -appVersion "1.0.17.0" ========================== Starting Command Output ===========================

[command]"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'C:\agent_work_temp\7a45da01-ced1-48e9-9581-517a6d5a3fb3.ps1'"

Using Version 1.0.17.0 Compiling base Using Symbols Folder: C:\agent_work\2\s.alPackages WARNING: This license is not compatible with this version of Business Central. WARNING: This license is not compatible with this version of Business Central. Downloading symbols: Microsoft_System_15.0.36510.0.app Url : http://172.30.238.209:7049/BC/dev/packages?publisher=Microsoft&appName=System&versionText=15.0.0.0&tenant=default

[error]ForEach-Object : Exception calling "DownloadFile" with "2" argument(s): "The operation has timed out"

At C:\Program Files\WindowsPowerShell\Modules\navcontainerhelper\0.6.4.10\AppHandling\Compile-AppInNavContainer.ps1:256 char:21

• $dependencies | ForEach-Object {

• 
  
  + CategoryInfo          : NotSpecified: (:) [ForEach-Object], MethodInvocationException
  
  + FullyQualifiedErrorId : WebException,Microsoft.PowerShell.Commands.ForEachObjectCommand
  ##[error]PowerShell exited with code '1'.
  ##[section]Finishing: Compile App

[freddydk]

and the output from the container generation?

[mohamedkathman]

Here it is...

[section]Starting: Create Build Container

============================================================================== Task : PowerShell Description : Run a PowerShell script on Linux, macOS, or Windows Version : 2.151.2 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell

Generating script. Formatted command: . 'C:\agent_work\2\s\scripts\Create-Container.ps1' ========================== Starting Command Output ===========================

[command]"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'C:\agent_work_temp\421e1de2-0a4f-470a-aa90-a889177f44cb.ps1'"

Create helloworld from mcr.microsoft.com/businesscentral/onprem:1910-rtm-w1 NavContainerHelper is version 0.6.4.10 Host is Microsoft Windows Server 2019 Standard - ltsc2019 Docker Client Version is 19.03.2 Docker Server Version is 19.03.2 Using image mcr.microsoft.com/businesscentral/onprem:1910-rtm-w1-ltsc2019 Disabling Health Check (always report healthy) Removing C:\ProgramData\NavContainerHelper\Extensions\helloworld Creating Container helloworld Version: 15.0.36560.0-W1 Style: onprem Platform: 15.0.36510.0 Generic Tag: 0.0.9.95 Container OS Version: 10.0.17763.737 (ltsc2019) Host OS Version: 10.0.17763.1 (ltsc2019) WARNING: The container operating system matches the host operating system, but the revision is different. If you encounter issues, you might want to specify -isolation hyperv Using locale en-US Using process isolation Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable) Using license file C:\bclicense\license.flf Files in C:\ProgramData\NavContainerHelper\Extensions\helloworld\my:

• AdditionalOutput.ps1
• license.flf
• MainLoop.ps1
• updatehosts.ps1 Creating container helloworld from image mcr.microsoft.com/businesscentral/onprem:1910-rtm-w1-ltsc2019 ba7361c235552a140b9e696d45ed19ca0158894540c70480043a6e137a312979 Waiting for container helloworld to be ready Initializing... Starting Container Hostname is helloworld PublicDnsName is helloworld Using NavUserPassword Authentication Starting Local SQL Server Starting Internet Information Server Creating Self Signed Certificate Self Signed Certificate Thumbprint 677CBA173B2AE0B588CA141E36B5D0B347594295 Modifying Service Tier Config File with Instance Specific Settings Starting Service Tier Registering event sources Creating DotNetCore Web Server Instance Using license file 'c:\run\my\license.flf' Import License Creating http download site Setting SA Password and enabling SA Creating admin as SQL User and add to sysadmin WARNING: This license is not compatible with this version of Business Central. Creating SUPER user WARNING: This license is not compatible with this version of Business Central. WARNING: This license is not compatible with this version of Business Central. Container IP Address: 172.30.238.209 Container Hostname : helloworld Container Dns Name : helloworld Web Client : http://helloworld/BC/ Dev. Server : http://helloworld Dev. ServerInstance : BC

Files: http://helloworld:8080/al-4.0.176004.vsix

Initialization took 97 seconds Ready for connections! Reading CustomSettings.config from helloworld Copy from container helloworld (C:\Applications\TestFramework\TestLibraries\Any\Microsoft_Any.app) to C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Any.app Publishing C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Any.app WARNING: This license is not compatible with this version of Business Central. Synchronizing Any on tenant default Installing Any on tenant default WARNING: This license is not compatible with this version of Business Central. App successfully published Copy from container helloworld (C:\Applications\TestFramework\TestLibraries\Assert\Microsoft_Library Assert.app) to C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Library Assert.app Publishing C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Library Assert.app WARNING: This license is not compatible with this version of Business Central. Synchronizing Library Assert on tenant default Installing Library Assert on tenant default WARNING: This license is not compatible with this version of Business Central. App successfully published Copy from container helloworld (C:\Applications\TestFramework\TestRunner\Microsoft_Test Runner.app) to C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Test Runner.app Publishing C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Test Runner.app WARNING: This license is not compatible with this version of Business Central. Synchronizing Test Runner on tenant default Installing Test Runner on tenant default WARNING: This license is not compatible with this version of Business Central. App successfully published Copy from container helloworld (C:\Applications\System Application\Test\Microsoft_System Application Test Library.app) to C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_System Application Test Library.app Publishing C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_System Application Test Library.app WARNING: This license is not compatible with this version of Business Central. Synchronizing System Application Test Library on tenant default Installing System Application Test Library on tenant default WARNING: This license is not compatible with this version of Business Central. App successfully published Copy from container helloworld (C:\Applications\BaseApp\Test\Microsoft_Tests-TestLibraries.app) to C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Tests-TestLibraries.app Publishing C:\ProgramData\NavContainerHelper\Extensions\helloworld_Microsoft_Tests-TestLibraries.app WARNING: This license is not compatible with this version of Business Central. Synchronizing Tests-TestLibraries on tenant default Installing Tests-TestLibraries on tenant default WARNING: This license is not compatible with this version of Business Central. App successfully published TestToolkit successfully imported Container helloworld successfully created Backing up CRONUS to C:\ProgramData\NavContainerHelper\onprem-15.0.36560.0-W1-bakFolders\helloworld\database.bak

[section]Finishing: Create Build Container

[kine]

Hi Freddy, I am having sometimes same error on my pipelines, mostly re-run of the stage helps. Seems interesting for me, because it means there is no change in the commands, just running them again and it helps. I am looking into this and will try to compare the output of failing run and successful re-run. But I think it must be something what is not directly connected to the scripts, some instability with some timing or concurrency. Do not know. Problem exists, but how much it is because something in navcontainerhelper, hard to say.

[kine]

Ok, some observation: When running on Win2016, it is failing, when same running on Win2019 it runs ok. Seems like problem of the combination of OS and Image. My failing case is:

2019-11-28T07:24:08.4985764Z Host is Microsoft Windows Server 2016 Standard - ltsc2016
2019-11-28T07:24:09.0137419Z Docker Client Version is 19.03.4
2019-11-28T07:24:09.2682161Z Docker Server Version is 19.03.4
2019-11-28T07:24:09.6306626Z Using image bcinsider.azurecr.io/bcsandbox:latest-ltsc2016
2019-11-28T07:24:12.1858768Z Creating Container BCApps
2019-11-28T07:24:12.5163251Z Version: 15.2.38451.0-w1
2019-11-28T07:24:12.5169736Z Style: sandbox
2019-11-28T07:24:12.7449929Z Platform: 15.0.38362.0
2019-11-28T07:24:12.9741365Z Generic Tag: 0.0.9.96
2019-11-28T07:24:13.2018646Z Container OS Version: 10.0.14393.3274 (ltsc2016)
2019-11-28T07:24:13.2025571Z Host OS Version: 10.0.14393.2580 (ltsc2016)
2019-11-28T07:24:13.2125901Z WARNING: The container operating system matches the host operating system, but the revision is different.
2019-11-28T07:24:13.2133454Z If you encounter issues, you might want to specify -isolation hyperv
2019-11-28T07:24:13.2156982Z Using locale en-US
2019-11-28T07:24:13.2168246Z Using process isolation

Running case is:

2019-11-28T07:24:25.7115671Z Host is Microsoft Windows Server 2019 Standard - ltsc2019
2019-11-28T07:24:26.0429809Z Docker Client Version is 18.09.4
2019-11-28T07:24:26.2008154Z Docker Server Version is 18.09.4
2019-11-28T07:24:26.4998363Z Using image bcinsider.azurecr.io/bcsandbox:latest-ltsc2019
2019-11-28T07:24:27.2134878Z Creating Container BCApps
2019-11-28T07:24:27.3828177Z Version: 15.2.38312.0-w1
2019-11-28T07:24:27.3835506Z Style: sandbox
2019-11-28T07:24:27.5536227Z Platform: 15.0.38154.0
2019-11-28T07:24:27.7200394Z Generic Tag: 0.0.9.96
2019-11-28T07:24:27.8840796Z Container OS Version: 10.0.17763.805 (ltsc2019)
2019-11-28T07:24:27.8849098Z Host OS Version: 10.0.17763.107 (ltsc2019)
2019-11-28T07:24:27.8942448Z WARNING: The container operating system matches the host operating system, but the revision is different.
2019-11-28T07:24:27.8949008Z If you encounter issues, you might want to specify -isolation hyperv
2019-11-28T07:24:27.8973707Z Using locale en-US
2019-11-28T07:24:27.8985169Z Using process isolation

[kine]

Ok, seems like it is effect of low RAM available in my case. It make sense. If there is not enough physical RAM, the process is slow and will timeout when downloading symbols. Will watch more for next case.

[Spycoclown]

The RAM idea i never thought of, maybe this is even my probleme. I will have a look at the RAM next time i get this error. Thx

[mohamedkathman]

How much RAM would be ideal for the container?

[Spycoclown]

I use the Host as agent Pool with 5 Agents, one agent need only 2 - 3 GB RAM but one creating it can use more, so if all 5 Agents are in creating, i have not enough RAM on my server so it get slower.

[kine]

In my case, I am using 5GB RAM per container (the switch for creating the container, not reserving it when process isolation is used), but on the server there are other containers running, and more active are really using e.g. 5GB. The PC has 25GB RAM, but in cases when there was problem, there was only 3GB RAM available.

[freddydk]

Assuming that this was a memory issue - thanks @kine and @Spycoclown - closing this issue.