search

microsoft / AL-Go

The plug-and-play DevOps solution for Business Central app development on GitHub
MIT License
270 stars 113 forks source link

Deployments to environments are behaving strange recently #1056

Open gntpet opened 4 months ago

gntpet commented 4 months ago

Hey @freddydk,

We are seeing some strange issues with deployments to our PROD environments.

Firstly, it takes really long time now 1h+ for the minor changes. Normally it runs between 15..30 minutes.

Eventually it errors out.

image

After restarting the job, it tells that my first extension is already in, and quickly deploys the second one.

image

If i check installation status, it shows Failed with timeout error.

image

But, it is actually installed

image

This behavior started this week.

Best Regards, Gintautas

freddydk commented 4 months ago

Is this on V24? I think there has been problems on 24 like this - but they should be fixed yesterday or today

gntpet commented 4 months ago

No, it's v23

gntpet commented 3 months ago

Again, issues with deployments. Deployments stopped working to our QA environment since yesterday.

Pipeline fails with useless error:

##[error]Deploying to QA failed. Unable to publish app. Please open the Extension Deployment Status Details page in Business Central to see the detailed error message.

Installation status shows another useless error

image

App insights shows another useless error

image

Just tried to deploy it manually, withouth AL-Go and it succeeded. Previously AL-GO failed multiple times

image

'Operations' shown in Admin portal does not match Extension Installation Status. It shows only two successfully installed apps that i installed manually. Failed entries from yesterday, & today are not to be seen. image

Is there something you could see on your side, @freddydk. There's must be something wrong with platform or AL-GO, since we are coming back to the deployment issues from time to time.

freddydk commented 3 months ago

What does the log in AL-Go show for the publishing of this app?

gntpet commented 3 months ago

Nothing special:

2024-05-07T06:57:54.3821107Z Downloading BcContainerHelper latest version from Blob Storage
2024-05-07T06:57:56.2054798Z Import from C:\ProgramData\BcContainerHelper\6.0.17\BcContainerHelper\BcContainerHelper.ps1
2024-05-07T06:57:56.4988390Z BcContainerHelper version 6.0.17
2024-05-07T06:57:56.6867654Z Setting useCompilerFolder = True
2024-05-07T06:57:56.8921477Z BC.HelperFunctions emits usage statistics telemetry to Microsoft
2024-05-07T06:57:56.9789664Z Running on Windows, PowerShell 5.1.20348.2400
2024-05-07T06:57:58.7788206Z project 'SRS.nl'
2024-05-07T06:57:58.7827819Z Apps to deploy
2024-05-07T06:57:58.7841440Z C:\a\srs-bc\srs-bc\.artifacts\SRS.nl-main-Apps-1.0.1565.0
2024-05-07T06:57:58.8667865Z Attempting authentication to https://api.businesscentral.dynamics.com/.default using clientCredentials...
2024-05-07T06:57:59.4831614Z Authenticated as app ***
2024-05-07T06:57:59.4971695Z EnvironmentUrl: https://businesscentral.dynamics.com/db1e96a8-a3da-442a-930b-235cac24cd5c/QA
2024-05-07T06:57:59.9489260Z Publishing apps using automation API
2024-05-07T06:57:59.9880900Z https://api.businesscentral.dynamics.com/v2.0/QA/api/microsoft/automation/v2.0/companies
2024-05-07T06:58:00.7722785Z Company 'CRONUS NL' has id 3b1e9576-41bc-ee11-907d-6045bde9cc0e
2024-05-07T06:58:00.7730130Z https://api.businesscentral.dynamics.com/v2.0/QA/api/microsoft/automation/v2.0/companies(3b1e9576-41bc-ee11-907d-6045bde9cc0e)/extensions
2024-05-07T06:58:01.9755659Z Extensions before:
....
....
2024-05-07T06:58:04.1986229Z WARNING: Dependency 6da8dd2f-e698-461f-9147-8e404244dd85:Continia Software_Continia Document Capture_12.0.0.0).app not 
2024-05-07T06:58:04.1987557Z found
2024-05-07T06:58:04.5059508Z SRS_SRS.base_1.0.1565.0.app - Downloading AL Language Extension to C:\ProgramData\BcContainerHelper\alLanguageExtension\13.0.1007491.zip
2024-05-07T06:58:09.8638045Z using 7zip
2024-05-07T07:08:12.8888397Z upgrading..............................................................Failed
2024-05-07T07:08:12.9107797Z ERROR: Unable to publish app. Please open the Extension Deployment Status Details page in Business Central to see the detailed error message. [System.Management.Automation.RuntimeException]
2024-05-07T07:08:13.5983525Z 
2024-05-07T07:08:13.5992372Z Extensions after:
....
....
freddydk commented 3 months ago

The latest succeeding publish - how much time did that take? This one fails after 10 minutes 6:58:09 -> 7:08:12

gntpet commented 3 months ago

here are screenshot for the last two succesful ones:

Base app takes 6, & 8.5 minutes to deploy

image image

freddydk commented 3 months ago

Will investigate whether there are any timeouts on 10 minutes

Sendt fra Outlook til iOShttps://aka.ms/o0ukef

Fra: Gintautas @.> Sendt: Wednesday, May 8, 2024 9:17:38 AM Til: microsoft/AL-Go @.> Cc: Freddy Kristiansen @.>; Mention @.> Emne: Re: [microsoft/AL-Go] Deployments to environments are behaving strange recently (Issue #1056)

here are screenshot for the last two succesful ones:

Base app takes 6, & 8.5 minutes to deploy

image.png (view on web)https://github.com/microsoft/AL-Go/assets/16342128/dcfe2bc4-052f-4b76-9be9-ca67547584c7 image.png (view on web)https://github.com/microsoft/AL-Go/assets/16342128/fa91ab10-b55e-49dd-88de-f3c8272d68ce

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/AL-Go/issues/1056#issuecomment-2100013128, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSGUA2V46DZLRSZQB247ZTZBHNSFAVCNFSM6AAAAABG26EMQCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBQGAYTGMJSHA. You are receiving this because you were mentioned.Message ID: @.***>

gntpet commented 3 months ago

Just happened again

This time it was running <10min

image

image