search

HodorNV / ALOps

ALOps
59 stars 24 forks source link

BC14 with FOB Import: Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (503) Server Unavailable." #250

Closed DanielGoehler closed 3 years ago

DanielGoehler commented 3 years ago

Describe the bug When I import a larger number of C/AL objects in BC 14 the service tier cannot provide Application symbols. I could provide an example via email.

In our docker environment I have run Sync-NAVTenant after importing the objects.

PS C:\Windows\system32> Import-ObjectsToNavContainer -containerName test -objectsFile "C:\Temp\Objects.fob" -ImportAction Overwrite -SynchronizeSchemaChanges Yes

Copy C:\Temp\Objects.fob to container test (c:\run\Objects.fob)
Importing Objects from c:\run\Objects.fob (container path)
Objects successfully imported

PS C:\Windows\system32> Enter-BCContainer test

[DOCKER]: PS C:\Run> Sync-NAVTenant nav -force

the used yaml

name: $(Build.BuildId)

variables:
- name: 'AppVersion'
  value: '1.1.[yyyyWW].*'
- group: 'ALOps Settings'

pool:
  name: BusinessCentralBuildAgents

trigger:
- master

steps:
- checkout: self
  clean: true

- task: ALOpsDockerCreate@1
  inputs:
    artifactversion: '14.6'
    artifactcountry: 'de'
    licensefile: '\\files\DevOps\Common\BC.flf'

- task: ALOpsDockerStart@1
  inputs:
    enable_symbol_loading: true
    memory_gb: '8'
    container_restart: 'always'
    docker_parameters: '--isolation=hyperv'

- task: ALOpsDockerWait@1
  inputs:
    search_string: 'Ready for connections!'

- task: ALOpsFobImport@1
  displayName: 'Import C/SIDE Customisations'
  inputs:
    usedocker: true
    filePath: 'Objects.fob'
    import_action: 'Overwrite'
    generate_symbol_reference: true

- task: ALOpsAppCompiler@1
  inputs:
    usedocker: true
    targetproject: 'MainApp/app.json'
    al_analyzer: 'CodeCop,PerTenantExtensionCop,UICop'
    nav_app_version: '$(AppVersion)'

- task: ALOpsAppPublish@1
  inputs:
    usedocker: true
    nav_artifact_app_filter: '*.app'

- task: ALOpsDockerRemove@1
  enabled: true
  condition: always()

the output

##[section]Starting: ALOpsAppCompiler
==============================================================================
Task         : ALOps App Compiler
Description  : Compile AL Extensions for Business Central
Version      : 1.435.1904
Author       : Hodor
Help         : Compile a Business Central extension from AL code.
==============================================================================
*** Validate configuration
*** Task Inputs:

name                  defaultValue required                                 value
----                  ------------ --------                                 -----
usedocker                    False    False                                  True
fixed_tag                             False                                      
targetproject           ./app.json    False C:\agent1\_work\21\s\MainApp\app.json
nav_computername                       True                                      
nav_serverinstance           BC140     True                                 BC140
nav_tenant                 default     True                               default
nav_ports_dev                 7049     True                                  7049
bc_username                           False                                      
bc_password                           False                                      
al_compiler_version     0.12.15355     True                            0.12.15355
al_analyzer                           False   CodeCop,PerTenantExtensionCop,UICop
nav_app_version            1.0.*.0     True                        1.1.[yyyyWW].*
vsix_download_path                    False                                      
use_ssl                      False    False                                 False
download_test_symbols        False    False                                 False
usecompression                True    False                                  True
publish_artifact              True    False                                  True
failed_on_warnings           False    False                                 False
app_file_suffix                       False                                      
updatebuildnumber             True    False                                  True
setup_working_folder         False    False                                 False

*** For documentation, please visit   : https://www.alops.be/documentation

*** Importing required PS-Functions
*** Initiate Docker Session
*** Set Docker Container ErrorActionPreference = Stop
*** Import NAV/BC Management DLL's
*** Loading assemblies for ServiceTier [NAV].
*** Loading assemblies from: [C:\program files\microsoft dynamics nav\140\service]
*** Selected module: [C:\program files\microsoft dynamics nav\140\service\Microsoft.Dynamics.Nav.Apps.Management.dll]
*** Loading module: [C:\program files\microsoft dynamics nav\140\service\Microsoft.Dynamics.Nav.Apps.Management.dll]
*** Selected module: [C:\program files\microsoft dynamics nav\140\service\Microsoft.Dynamics.Nav.Management.dll]
*** Loading module: [C:\program files\microsoft dynamics nav\140\service\Microsoft.Dynamics.Nav.Management.dll]
*** Setting up AL Compiler [ALLanguage]
*** Downloading AL Compiler [ALLanguage]
*** Get Extension from Local-Disk...[C:\Run\ALLanguage.vsix]
*** Resolving Path [C:\Run\ALLanguage.vsix]
*** VSIX Path Resolved: [C:\Run\ALLanguage.vsix]
*** Unpacking Extension to folder...
*** Loading Json Object: [c:\Run\Microsoft.al\al-ALLanguage\extension\package.json]
*** Using AL Compiler [vALLanguage] located [C:\Run\Microsoft.al\al-ALLanguage\extension\bin\alc.exe]
***
*** Working folder: C:\agent1\_work\21\s
***
*** Copy Working folder to Docker Container: ed7dca391661f9e78c0567e27ebfc89b77ea4450cea0e01fc8a3b3e6959b1e49
*** Docker Working Folder: c:\Run\DevOps\
*** Copy from [C:\agent1\_work\21\s] to [c:\Run\DevOps\]
*** Source File setup completed
*** Target Project: \MainApp\app.json
*** Check for app.json files: c:\Run\DevOps\
*** Multiple projects found:
   - C:\Run\DevOps\MainApp
   - C:\Run\DevOps\TestApp
*** Retrieving App-Info [\MainApp\app.json]
*** Loading Json Object: [c:\Run\DevOps\MainApp\app.json]
  * App.ID          = 1892fdd2-f824-4dc3-bae2-64c458aee732
  * App.Name        = CustomApp
  * App.Publisher   = My Company
  * App.Version     = 1.0.0.44
  * App.Platform    = 14.0.0.0
  * App.Application = 14.0.0.0
***
*** Match: yyyyWW
*** Setting Managed App Version [c:\Run\DevOps\MainApp\app.json] =>  1.1.202045.2649
***
*** Working Direcotry: [c:\Run\DevOps\]
*** Target Project: [\MainApp\app.json]
*** Project Directory: [c:\Run\DevOps\MainApp]
*** VSCode Settings: [c:\Run\DevOps\MainApp\.vscode\settings.json]
*** VSCode-Settings json found.
*** Loading Json Object: [c:\Run\DevOps\MainApp\.vscode\settings.json]
*** No AL-RuleSet json found.
***
*** Resolve Docker Credentials
*** Setup BC Username/Password from Container info
*** Download Symbols
*** Working Dir: c:\Run\DevOps\
*** Loading assemblies from: [C:\program files\microsoft dynamics nav\140\service]
*** PackageCache path created [c:\Run\DevOps\PackageCache]
***
*** Download System-Symbols for [14.0.0.0]
*** Downloading NAV Symbols for [System v14.0.0.0]
*** Downloading NAV Symbols from [https://ed7dca391661:7049/NAV/dev/packages?publisher=Microsoft&appName=System&versionText=14.0.0.0&tenant=default]
*** Downloading NAV Symbols with BasicAuthentication [admin]
##[command]Invoke-RestMethod -Method Get -Uri https://ed7dca391661:7049/NAV/dev/packages?publisher=Microsoft&appName=System&versionText=14.0.0.0&tenant=default
*** Content-Disposition: attachment; filename=Microsoft_System_14.0.36457.0.app
*** Saving symbol file [Microsoft_System_14.0.36457.0.app] to [c:\Run\DevOps\PackageCache]
*** Read App Manifest: [c:\Run\DevOps\PackageCache\Microsoft_System_14.0.36457.0.app].
***
*** Download Application-Symbols for [14.0.0.0]
*** Downloading NAV Symbols for [Application v14.0.0.0]
*** Downloading NAV Symbols from [https://ed7dca391661:7049/NAV/dev/packages?publisher=Microsoft&appName=Application&versionText=14.0.0.0&tenant=default]
*** Downloading NAV Symbols with BasicAuthentication [admin]
##[command]Invoke-RestMethod -Method Get -Uri https://ed7dca391661:7049/NAV/dev/packages?publisher=Microsoft&appName=Application&versionText=14.0.0.0&tenant=default
##[error]Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (503) Server Unavailable."
##[section]Async Command Start: Add Build Tag
Build '2649' has following tags now: myimage-onprem-14.6.36463.0-de-10.0.17763.1518-ltsc2019, BC.Platform.14.0.36457.0, BC.Version.14.6.36463.0, AL.Compiler.ALLanguage
##[section]Async Command End: Add Build Tag
##[section]Async Command Start: Update Build Number
Update build number to 1.1.202045.2649 for build 2649
##[section]Async Command End: Update Build Number
##[section]Finishing: ALOpsAppCompiler
waldo1001 commented 3 years ago

I try to understand this. Few questions:

waldo1001 commented 3 years ago

I reached out for an example (to repro this)

DanielGoehler commented 3 years ago

I try to understand this. Few questions:

  • only doesn't work with "big" fob file? It does work with a "smaller" fob file

This also occurs if I split the objects into mutiple files.

  • the import doesn't fail, but the compile fails.. .

Compilation works fine. Without errors or warnings.

  • when you test it yourself, on the same server (hardware), same VM, with same fob-file, with container with same amount of memory, ... it does work?

Different Hardware Server but same docker image, I have to run Sync-NAVTenant afterwards and in some cases even Generate Symbols to get the database to work again. Imcreasing RAM to 16 GB doesn't help.

I guess the PowerShell CmdLet has issues with larger FOB, imports them but timeouts or something else happens while syncing with SQL Server.

waldo1001 commented 3 years ago

I guess I need the exact fob to repro this ..

DanielGoehler commented 3 years ago

Should be in your mailbox (send yesterday 10:53). If not please reach out.

I have found something which might be related. Here is also a ready to go example, where I only change one table (5330 "CRM Connection Setup"). Despite enable_symbol_loading = true and generate_symbol_reference = true the Al Compiler says:

Table 'CRM Connection Setup' is missing

AL Example.zip

the used yaml

[..]
- task: ALOpsDockerStart@1
  inputs:
    enable_symbol_loading: true
    memory_gb: '8'

- task: ALOpsDockerWait@1
  inputs:
    search_string: 'Ready for connections!'

- task: ALOpsFobImport@1
  displayName: 'Import C/SIDE Customisations'
  inputs:
    usedocker: true
    filePath: 'Customisations.fob'
    import_action: 'Overwrite'
    generate_symbol_reference: true
[..]
waldo1001 commented 3 years ago

Internal ref: https://devops.hodor.be/Waldo/ALOpsExamples/_build/results?buildId=248&view=results

waldo1001 commented 3 years ago

Hi @DanielGoehler ,

It seems you did a wrong reference. Can you consider this project: FOBProject.zip

This pipeline just runs, and takes the customization of the fob into account..

DanielGoehler commented 3 years ago

Hi @waldo1001,

the pipeline works without ALOpsFobImport. Why do I have to change Record "CRM Connection Setup" to Record "Microsoft Dynamics 365 Connection Setup" althougth this table has the name CRM Connection Setup?

Record "CRM Connection Setup" works in dev, test and production environment.

Screenshot

waldo1001 commented 3 years ago

Well, I don't know what happened with the fob, or what is wrong with BC version 14 (besides A LOT is wrong with that version) .. but apparently, when you reference the caption of the table, it works. This means that the symbols are wrong. image We figured that out by downloading the symbols that the compiler was using during compile. Table 5330 was in those symbols (although it said it wasn't), but with the name equal to the caption.

It could be that there is an issue with the alc.exe of version 14. It could also be that the NST in version 14 generates the wrong symbols.

All I can say is that ALOps is calling the default tools that are available. There's not a lot more we can do :-/

We'll try some more to try to see why VSCode behaves differently than the pipeline.. .

waldo1001 commented 3 years ago

We found the cause. As expected: it's the version of Business Central that contains a bug. You can replicate the bug when you set up the container with "enableSymbolLoading" enabled, and THEN import your fob file with "generatesymbol". At that point, the symbols are corrupt, using the caption as a name.

To mitigate this, we need to "generatesymbols for the entire application".

We'll see we can provide a quickfix to mitigate this BC Bug in ALOps - probably by a "ALOpsDockerExec" step after the fob import.. .

DanielGoehler commented 3 years ago

Thanks! Oh boy, I didn't expected such an issue here. A ALOpsDockerExec solution would be nice for already released unfixed BC14 versions.

I try to reproduce this and create a Microsoft Support Case, so at least this gets fixed for upcoming BC14 CUs.

waldo1001 commented 3 years ago

We'll build a new parameter in the fobimport to be able to "force" the "generate symbols". It should not be necessary in newer versions .. but it will fix the issue if necessary (it slows down the pipiline..)

AdminHodor commented 3 years ago

Dear @DanielGoehler ,

We just release v1.437 which contains the update for this. => ALOpsFobImport: New boolean parameter [force_full_generate_symbol_reference], will trigger a full generate symbols after FOB Import.

Kind regards,

DanielGoehler commented 3 years ago

Just a quick update: I just got feedback from Microsoft support that this issue will not be not resolved, but they will update the documentation that this does not work.

The import with force_full_generate_symbol_reference takes a little longer, but works fine. So thank you for this feature!

[..]
- task: ALOpsFobImport@1
  displayName: 'Import C/SIDE Customisations'
  inputs:
    usedocker: true
    filePath: 'Customisations.fob'
    import_action: 'Overwrite'
    force_full_generate_symbol_reference: true
[..]