Update BizTalk Server Project Build Component

fabsenet commented 2 years ago

Tool name

BizTalk Server Project Build Component

Tool license

Developer license

Add or update?

[ ] Add
[X] Update

Desired version

3.13.785.0 / latest / CU2

Approximate size

No response

Brief description of tool

This is a continuation of the talk over here: https://github.com/actions/virtual-environments/issues/1865#issuecomment-960596247

URL for tool's homepage

https://docs.microsoft.com/en-us/biztalk/core/developer-tools

Provide a basic test case to validate the tool's functionality.

No response

Virtual environments affected

[ ] Ubuntu 18.04
[ ] Ubuntu 20.04
[ ] macOS 10.15
[ ] macOS 11
[ ] Windows Server 2016
[X] Windows Server 2019
[ ] Windows Server 2022

Can this tool be installed during the build?

No response

Tool installation time in runtime

No response

Are you willing to submit a PR?

No, I cannot! The component seems to be made specifically by Microsoft for DevOps.

al-cheb commented 2 years ago

Hey, @fabsenet We already pre-installed the latest available version of BizTalk Server Visual Studio extension(ms-biztalk.BizTalk) on Windows Server 2019 - https://github.com/actions/virtual-environments/blob/main/images/win/toolsets/toolset-2019.json

PS > Get-VsixExtenstionFromMarketplace -ExtensionMarketPlaceName ms-biztalk.BizTalk

ExtensionName : ms-biztalk.BizTalk
VsixId        : BizTalkProjectSystem.BD1D0FA0-E48A-4270-995F-F110DD9F7838
FileName      : BizTalkVsix.vsix
DownloadUri   : https://ms-biztalk.gallerycdn.vsassets.io/extensions/ms-biztalk/biztalk/3.13.2.0/1595970968123/BizTalkV
                six.vsix

3.13.785.0 - It's the version of BizTalk Server 2020 which we don't install at all. https://support.microsoft.com/en-us/topic/kb5003151-cumulative-update-2-for-microsoft-biztalk-server-2020-324a3d5d-8d87-49aa-b7e4-1f0733acbb09

For BizTalk Developer Tools If you have BizTalk Developer Tools installed, you must install the latest version of BizTalk Server Visual Studio extension (build 3.13.2.0) in addition to this CU package. The extension can be installed from https://marketplace.visualstudio.com/items?itemName=ms-biztalk.BizTalk or from within Visual Studio through Manage Extensions.

fabsenet commented 2 years ago

The actual tool download comes from https://aka.ms/BuildComponentSetup.EN and is a zip file containing 2 MSIs and a bunch of BizTalk-DLLs but not that vsix. the vsix is neither included nor needed.

this zip file is outdated.

miketimofeev commented 2 years ago

@fabsenet thanks for the clarification. The vsix is installed by one of those MSIs. We have engaged the BizTalk team.

al-cheb commented 2 years ago

@fabsenet, The test version has been released https://aka.ms/TestBuildComponentSetup.EN.zip with build number 3.13.812.0. Could you check if the problem still persists?

fabsenet commented 2 years ago

hi @al-cheb ,

I am sorry but I cannot test this. I never had the build error on one of my own machines. It only surfaces on the hosted devops build agents and this does not seem to have this update yet.

Is prerelease hosted agents a thing? I could change the hosted agent and simply let it run. At the moment with the official hosted agent, the error is still there.

al-cheb commented 2 years ago

@fabsenet , You could install the new version in runtime and check:

      - name: Install BizTalk
        run: |
          $bizTalkBuildComponentUri = "https://aka.ms/BuildComponentSetup.EN"
          $setupZipFile = Start-DownloadWithRetry -Url $bizTalkBuildComponentUri -Name "BuildComponentSetup.EN.zip"
          $setupPath = "C:\BizTalkBuildComponent"
          if (-not (Test-Path -Path $setupPath)) {
              $null = New-Item -Path $setupPath -ItemType Directory -Force
          }
          Extract-7Zip -Path $setupZipFile -DestinationPath $setupPath
          Start-Process msiexec.exe -ArgumentList "/qn /x C:\BizTalkBuildComponent\Bootstrap.msi" -Wait
          Start-Process msiexec.exe -ArgumentList "/qn /x C:\BizTalkBuildComponent\BuildComponentSetup.msi" -Wait
          $bizTalkBuildComponentUri = "https://aka.ms/TestBuildComponentSetup.EN.zip"
          $setupZipFile = Start-DownloadWithRetry -Url $bizTalkBuildComponentUri -Name "BuildComponentSetup.EN.zip"
          Extract-7Zip -Path $setupZipFile -DestinationPath $setupPath
          $ArgumentList = ('/i', "C:\BizTalkBuildComponent\BuildComponentSetup.EN\Bootstrap.msi", '/QN', '/norestart')
          Start-Process -FilePath msiexec.exe -ArgumentList $ArgumentList -Wait
          $ArgumentList = ('/i', "C:\BizTalkBuildComponent\BuildComponentSetup.EN\BuildComponentSetup.msi", '/QN', '/norestart')
          Start-Process -FilePath msiexec.exe -ArgumentList $ArgumentList -Wait 

          $bizTalkReg = Get-ItemProperty "HKLM:\SOFTWARE\WOW6432Node\Microsoft\BizTalk Server\3.0"
          return "$($bizTalkReg.ProductName) $($bizTalkReg.ProductVersion)"

fabsenet commented 2 years ago

I changed the pipeline and added a first step:

and installing the component seems to be succesful:

the actual build error remains unchanged :-(

[error]\SomeEdifactBiztalkMap.btm(0,0): Error btm1023: Exception Caught: The map contains a reference to a schema node that is not valid. Perhaps the schema has changed. Try reloading the map in the BizTalk Mapper. The XSD XPath of the node is: /[local-name()='']/[local-name()='EFACT_D18A_IFTSTA_2.0c']/[local-name()='CNILoop1']/[local-name()='STSLoop2']/[local-name()='NADLoop2']/[local-name()='CTALoop2']/[local-name()='CTA']/[local-name()='CTA01']

so technically you could go ahead and update the hosted agent with the new tools version but it does not fix my issue.

al-cheb commented 2 years ago

@fabsenet , Could you share a repo with an app to reproduce the issue ? Does it work without BizTalk Server 2020?

fabsenet commented 2 years ago

please see https://github.com/fabsenet/bts-hosted-agent-minimum-sample

I tried to remove as much stuff as possible. this sample compiles just fine on my dev machine as well as my self-hosted build agent. But it fails to compile on a hosted agent.

al-cheb commented 2 years ago

@fabsenet, Do you have on your local dev machine this C:\Program Files (x86)\Microsoft BizTalk Server\Developer Tools\Microsoft.VisualStudio.BizTalkProject.dll library?

fabsenet commented 2 years ago

yes, this files exists on my dev machine as well as our self-hosted build machine. I am wondering how you got there to ask for this specific file?

al-cheb commented 2 years ago

@fabsenet, The Microsoft.VisualStudio.BizTalkProject.dll library is not a part of build component, it is a part of BizTalk Server, which is not installed on Windows Server 2019.

https://techcommunity.microsoft.com/t5/biztalk-server-team-blog/biztalk-server-2020-setup-developer-tools-installation/ba-p/1225906

Note that this extension only installs necessary components and templates required for a Visual Studio extension, and is dependent on BizTalk Server installation. BizTalk Server 2020 Developer Tools feature installation is required for it to work.

fabsenet commented 2 years ago

the sample solution is sooo scaled down, it is a simple project containing schemas and maps. you cannot get more basic than this in the biztalk world.

If I understand you correctly, you are trying to say, that this is not expected to compile on the hosted agent? Please tell me, what specific action is enabled by installing the BizTalk Server Project Build Component then in the environment?

Also before making this small sample, I had a solution with 54 large projects in them and this map is the only one failing to compile.

al-cheb commented 2 years ago

If I understand you correctly, you are trying to say, that this is not expected to compile on the hosted agent? - Yep, that's right, because the library is the part of BizTalk Server. For e.g. if we look at CU update https://support.microsoft.com/en-us/topic/cumulative-update-6-for-microsoft-biztalk-server-2016-de7c2936-58d7-c75a-55f8-b7cc3350fab9 for BizTalk Server,we can see this lib:

In that case, nothing we can do from our side, you should continue to use a self-hosted agent.

Please tell me, what specific action is enabled by installing the BizTalk Server Project Build Component then in the environment? - What actions do you mean?

fabsenet commented 2 years ago

I cannot follow how this single dll is of any importance. I can also not follow the source of your screenshots. The error on the hosted build agent is:

[error]\BROKEN.btm(0,0): Error btm1023: Exception Caught: The map contains a reference to a schema node that is not valid. Perhaps the schema has changed. Try reloading the map in the BizTalk Mapper. The XSD XPath of the node is: /[local-name()='']/[local-name()='EFACT_D18A_IFTSTA_2.0c']/[local-name()='CNILoop1']/[local-name()='STSLoop2']/[local-name()='NADLoop2']/[local-name()='CTALoop2']/[local-name()='CTA']/[local-name()='CTA01']

There is no assembly load error anywhere in this message. And just to make it clear, i tried the suggestion in the error message and it is not making it better.

Here is my opinion:

The BizTalk Server Project Build Component is installed on the hosted build agent to allow compilation of biztalk projects. I cannot come up with any other reason for its existence!
my project is a biztalk project, so I expect it to compile
My complete solution consists of 54 projects and all others happily compile except one, which I distilled into the mini-sample repo.

You may be right regarding that this is not an error in the virtual build environment but in biztalk itself. But strangely, it does not surface outside of the hosted agent. But please, get someone from the BizTalk-Team involved.

al-cheb commented 2 years ago

I cannot follow how this single dll is of any importance. I can also not follow the source of your screenshots. The error on the hosted build agent is:

[error]\BROKEN.btm(0,0): Error btm1023: Exception Caught: The map contains a reference to a schema node that is not valid. Perhaps the schema has changed. Try reloading the map in the BizTalk Mapper. The XSD XPath of the node is: /[local-name()='']/[local-name()='EFACT_D18A_IFTSTA_2.0c']/[local-name()='CNILoop1']/[local-name()='STSLoop2']/[local-name()='NADLoop2']/[local-name()='CTALoop2']/[local-name()='CTA']/[local-name()='CTA01']

There is no assembly load error anywhere in this message. And just to make it clear, i tried the suggestion in the error message and it is not making it better.

Here is my opinion:

The BizTalk Server Project Build Component is installed on the hosted build agent to allow compilation of biztalk projects. I cannot come up with any other reason for its existence!

my project is a biztalk project, so I expect it to compile

My complete solution consists of 54 projects and all others happily compile except one, which I distilled into the mini-sample repo.

You may be right regarding that this is not an error in the virtual build environment but in biztalk itself. But strangely, it does not surface outside of the hosted agent. But please, get someone from the BizTalk-Team involved.

The BizTalk team is able to reproduce the issue in their local test environments They will look at this in the scope of our next CU, but no ETA has been provided.

al-cheb commented 2 years ago

@fabsenet , BizTalk team has published a new test version of the BizTalk Component.

I was able to build the project from https://github.com/fabsenet/bts-hosted-agent-minimum-sample repo :

- powershell: |
   $ArgumentList = ('/QN', '/x', '{32DD9048-2E5F-45F9-BD31-7D51CE5C2F6C}')
   Start-Process -FilePath msiexec.exe -ArgumentList $ArgumentList -Wait
  displayName: 'Uninstall BizTalk'

- powershell: |
   $setupPath = "C:\BizTalkBuildComponent"
   $bizTalkBuildComponentUri = "https://aka.ms/TestBuildComponentSetup.EN.zip"
   $setupZipFile = Start-DownloadWithRetry -Url $bizTalkBuildComponentUri -Name "BuildComponentSetup.EN.zip"
   Extract-7Zip -Path $setupZipFile -DestinationPath $setupPath
   $ArgumentList = ('/i', "$setupPath\Bootstrap.msi", '/QN', '/norestart')
   Start-Process -FilePath msiexec.exe -ArgumentList $ArgumentList -Wait
   $ArgumentList = ('/i', "$setupPath\BuildComponentSetup.msi", '/QN', '/norestart')
   Start-Process -FilePath msiexec.exe -ArgumentList $ArgumentList -Wait 

   $bizTalkReg = Get-ItemProperty "HKLM:\SOFTWARE\WOW6432Node\Microsoft\BizTalk Server\3.0"
   return "$($bizTalkReg.ProductName) $($bizTalkReg.ProductVersion)"

  displayName: 'Install BizTalk update'

fabsenet commented 2 years ago

This are some really good news!

I tested this with our full solution and it builds fine as well now. From my point of view, this can be released to the hosted agent.

al-cheb commented 2 years ago

@fabsenet, The deployment with BizTalk Server Project Build Component 3.13.832.0 has been completed.

actions / runner-images