Closed evjrr closed 7 months ago
So, you are saying that this:
$Artifact = Get-BCArtifactUrl -version 23 -type Sandbox -country w1 -select Latest
Download-Artifacts -artifactUrl $artifact -includePlatform
Takes 1-2 hours???
or is it the container creation?
I can see that you are not running as administrator, yet UsePsSession is true - that is strange, but would not cause any issues with artifact download.
It is the download part there takes 1-2 hours It's the same if I just download the artifact from a browser
Artifacts are not fast and we are investigating ways to make them faster. Having said that - I can (from my home with a similar internet connection as yours) get the BC artifact url in 7 seconds and download them in 1 minute and 40 seconds.
Measure-Command {
$Artifact = Get-BCArtifactUrl -version 23 -type Sandbox -country w1 -select Latest
}
Measure-Command {
Download-Artifacts -artifactUrl $artifact -includePlatform
}
Days : 0
Hours : 0
Minutes : 0
Seconds : 7
Milliseconds : 120
Ticks : 71209458
TotalDays : 8,24183541666667E-05
TotalHours : 0,0019780405
TotalMinutes : 0,11868243
TotalSeconds : 7,1209458
TotalMilliseconds : 7120,9458
Downloading artifact /sandbox/23.4.15643.16235/w1
Downloading C:\Users\freddyk\AppData\Local\Temp\2a98828f-9533-4d24-9265-aa94f2ebc4cd.zip
Downloading using WebClient
Unpacking artifact to tmp folder using 7zip
Downloading platform artifact /sandbox/23.4.15643.16235/platform
Downloading C:\Users\freddyk\AppData\Local\Temp\0ecd86a8-a4b9-41a2-8187-44ccf510729b.zip
Downloading using WebClient
Unpacking artifact to tmp folder using 7zip
Days : 0
Hours : 0
Minutes : 1
Seconds : 39
Milliseconds : 530
Ticks : 995307043
TotalDays : 0,00115197574421296
TotalHours : 0,0276474178611111
TotalMinutes : 1,65884507166667
TotalSeconds : 99,5307043
TotalMilliseconds : 99530,7043
The problem seems to be local at your place - maybe you have a proxy or a virus scanner that causes this - it isn't something I can reproduce nor something I have seen from other people.
It is running fine now.
but the problem was not only on my work computer and in the office I tested at work and from home and at home I tested it on my private PC and on my work PC same result so it is not a proxy or virus scanner problem I think it is a Azure Blob Storage problem.
It is not the first time I have seen this problem.
I work at the same company as @evjrr and have tried the same command as you just posted @freddydk.
The results are quite a bit slower (~25 minutes to download artifacts). That is bad news for us since we have a timeout of 20 minutes to download an artifact.
In order to test on some other ISP, I had a couple of friends test the same URL (https://bcartifacts.azureedge.net/sandbox/23.4.15643.16247/w1). They reported as follows:
Is it possible that the Azure Storage Account bcartifacts could be reaching some throughput limit at peak times? We do not have experience with large public Azure Storage Accounts, but I think I would personally start at the following article and see if the metrics in Azure Storage reported any issues. https://learn.microsoft.com/en-us/troubleshoot/azure/azure-storage/troubleshoot-storage-performance
The simple solution for us would be to increase the timeout to 30 minutes or more, but I only see more pipelines/developers coming to Business Central.
PS C:\Users\jsa> Measure-Command {
>> $Artifact = Get-BCArtifactUrl -version 23 -type Sandbox -country w1 -select Latest
>> }
BcContainerHelper version 6.0.3
BC.HelperFunctions emits usage statistics telemetry to Microsoft
Days : 0
Hours : 0
Minutes : 0
Seconds : 18
Milliseconds : 353
Ticks : 183530401
TotalDays : 0,000212419445601852
TotalHours : 0,00509806669444444
TotalMinutes : 0,305884001666667
TotalSeconds : 18,3530401
TotalMilliseconds : 18353,0401
PS C:\Users\jsa> Measure-Command {
>> Download-Artifacts -artifactUrl $artifact -includePlatform
>> }
Downloading artifact /sandbox/23.4.15643.16247/w1
Downloading C:\Users\jsa\AppData\Local\Temp\e3e8a53e-55fb-4d1a-943f-1171576de634.zip
Downloading using WebClient
Unpacking artifact to tmp folder using 7zip
Downloading platform artifact /sandbox/23.4.15643.16247/platform
Downloading C:\Users\jsa\AppData\Local\Temp\9c2f87a8-657a-48a3-b481-3c4d906a0a3c.zip
Downloading using WebClient
Unpacking platform artifact to tmp folder using 7zip
Downloading Prerequisite Components
Downloading c:\bcartifacts.cache\sandbox\23.4.15643.16247\platform\Prerequisite Components\IIS URL Rewrite Module\rewrite_2.0_rtw_x64.msi
Downloading using WebClient
Downloading c:\bcartifacts.cache\sandbox\23.4.15643.16247\platform\Prerequisite Components\DotNetCore\DotNetCore.1.0.4_1.1.1-WindowsHosting.exe
Downloading using WebClient
Days : 0
Hours : 0
Minutes : 24
Seconds : 43
Milliseconds : 778
Ticks : 14837780411
TotalDays : 0,0171733569571759
TotalHours : 0,412160566972222
TotalMinutes : 24,7296340183333
TotalSeconds : 1483,7780411
TotalMilliseconds : 1483778,0411
I have tested the direct download of weekly sandbox artifacts for DK (https://bcartifacts.azureedge.net/sandbox/23.4.15643.16092/dk) from home with a 1000 mbit connection - I have an extremely poor download speed at 16-24 kb per second which is back to analog modem "speed". Download ETA is 2+ hours!. I know this is not containerhelper issue but something in Entra - please engage relevant people at Microsoft as all our pipelines are currently timing out and therefore failing.
Same here, I have actions in github taking forever, tried a local download just to check: A part of it downloads and then it grinds to a halt, Chrome gave up finally. Gigabit fiber, so it should not be local.
I think somebody fixed the issue during the night, 7 pipelines executed this morning and everything is now back to normal speed :-)
Similar thing (but not a such extreme) was discussed here: #3320
I don't feel it's fixed, it comes and go... I'm still facing timeout issues using the latest bccontainerhelper... I don't face any on github runner, only on my self hosted runners with a 300Mbs internet speed.
@ronnykwon please create a new issue with the version of ContainerHelper used and artifact Url + the time spend when timing out
Slow Artifact Download it takes about 2 hours to download and my connection is not the problem I have 560 Mbps Download and 940 Mbps Upload when I performed the test.
Normally it takes about 7-9 minutes to build a new container from scratch.
But with this issue all our pipelines fails if it does not have the artifact.cache because of a timeout on 20 min for downloading artifacts.
Scripts used to create container and cause the issue
Full output of scripts