Azure / Azure-Data-Factory-Integration-Runtime-in-Windows-Container

Azure Data Factory Integration Runtime in Windows Container Sample
MIT License
25 stars 36 forks source link

Upgrade shir to current version #24

Closed magnusakselvoll closed 3 months ago

magnusakselvoll commented 5 months ago

Hi,

What is the policy for upgrading the shir version in this container? I would like to have version 5.37.8767.4, but the setup script explicitly downloads 5.34.8675.1.

Could anyone bring some insight or workaround, or should I produce a pull request?

jikuja commented 5 months ago

Workaround: Download SHIR MSI into SHIR folder: https://github.com/Azure/Azure-Data-Factory-Integration-Runtime-in-Windows-Container/blob/eefd1426b30afc9ca95f57b7576df1116c7c2bca/SHIR/build.ps1#L11-L15

Insights from https://github.com/Azure/Azure-Data-Factory-Integration-Runtime-in-Windows-Container/issues/18

The link is by design redirected to the auto-upgraded version (that is, the stablest version), instead of the latest version. For instance, we now start to push version 5.32.8600.2, and the fwlink will help you download this version.

magnusakselvoll commented 3 months ago

Thank you @jikuja, that seems to work. Would you or anyone happen to know how to get the correct URL for a given version of shir? I see in build.ps1 this url: "https://go.microsoft.com/fwlink/?linkid=839822&clcid=0x409", but the only URL I can find for the latest version is "https://download.microsoft.com/download/E/4/7/E4771905-1079-445B-8BF9-8A1A075D8A10/IntegrationRuntime_5.39.8818.1.msi".

Sorry if this is outside the scope of this repo, but I thought perhaps you would know.

byran77 commented 3 months ago

Thank you @jikuja, that seems to work. Would you or anyone happen to know how to get the correct URL for a given version of shir? I see in build.ps1 this url: "https://go.microsoft.com/fwlink/?linkid=839822&clcid=0x409", but the only URL I can find for the latest version is "https://download.microsoft.com/download/E/4/7/E4771905-1079-445B-8BF9-8A1A075D8A10/IntegrationRuntime_5.39.8818.1.msi".

Sorry if this is outside the scope of this repo, but I thought perhaps you would know.

hi @magnusakselvoll, just replace 5.39.8818.1 with the needed version in url "https://download.microsoft.com/download/E/4/7/E4771905-1079-445B-8BF9-8A1A075D8A10/IntegrationRuntime_5.39.8818.1.msi"

magnusakselvoll commented 3 months ago

Ok, thanks, I found the correct URL then. Thanks a lot for your help, this resolves my issue.

For anyone else having this issue, what I basically do is that after I clone the github repository to my build server, and before I run docker build (in an azure devops pipeline), I download the desired version of SHIR to the ./SHIR/ folder. Example code:

trigger: none

stages:
- stage: 'create_container'
  jobs:
  - deployment: 'build_push_container'
    displayName: 'Build container and push to ACR'
    variables:
      containerName: 'adf-self-hosted-integration-runtime'
      artifactName: 'dockerimage'
    environment: 'adf-shir-container'
    pool:
      vmImage: 'windows-2019'
    workspace:
      clean: 'all'
    strategy:
      runOnce:
        deploy:
          steps:
          - script: 'git clone --depth 1 -b main https://github.com/Azure/Azure-Data-Factory-Integration-Runtime-in-Windows-Container.git'
            workingDirectory: '$(Build.SourcesDirectory)'
            displayName: 'Clone github repository with dockerfile'
          - pwsh: 'Invoke-WebRequest -Uri "https://download.microsoft.com/download/E/4/7/E4771905-1079-445B-8BF9-8A1A075D8A10/IntegrationRuntime_5.39.8818.1.msi" -OutFile "$(Build.SourcesDirectory)/Azure-Data-Factory-Integration-Runtime-in-Windows-Container/SHIR/IntegrationRuntime_5.39.8818.1.msi"'
            displayName: 'Download specific version of ADF Shir'
          - task: DockerInstaller@0
            inputs:
              dockerVersion: '17.09.0-ce'
            displayName: 'Ensure that docker command line is installed'
          - task: 'Docker@2'
            displayName: 'Build container image'
            inputs:
              command: 'build'
              containerRegistry: 'YOUR_CONTAINER_REGISTRY'
              repository: '$(containerName)'
              Dockerfile: '$(Build.SourcesDirectory)/Azure-Data-Factory-Integration-Runtime-in-Windows-Container/Dockerfile'
              tags: |
                $(Build.BuildNumber)
                latest
              arguments: '--build-arg INSTALL_JDK=true'
          - task: Docker@2
            displayName: 'Push docker image'
            inputs:
              command: 'push'
              containerRegistry: 'YOUR_CONTAINER_REGISTRY'
              repository: '$(containerName)'
              Dockerfile: '$(Build.SourcesDirectory)/Azure-Data-Factory-Integration-Runtime-in-Windows-Container/Dockerfile'
              tags: |
                $(Build.BuildNumber)
                latest