search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.28k stars 21.46k forks source link

YAML Build/Deployment Pipeline. #108365

Closed JFolberth closed 1 year ago

JFolberth commented 1 year ago

I once flagged that this release content is outdates since Classic Releases can now be disabled in ADO.

Below is a YAML pipeline which can be used for any data factory. I do have templates available; however, for walkthrough this is a fully expanded one where user just needs to substitute the variables. Assumption is the package.json is in an adf_scripts folder and adf pipelines are mapped to adf folder.

More then happy to collaborate further.

trigger:
  branches:
    include:
    - main
pool:
  vmImage: 'ubuntu-latest'

variables:
- name: adfResourceGroupName
  value: 'rg-datafactory'
- name: adfName
  value: 'adf-datafactory'
- name: azureSubscription
  value: '0000000-0000-0000-0000-0000000000000'
- name: azureServiceConnectionName
  value: 'AzureDevServiceConnection'
stages:
- stage: adfdemo_build
  jobs:
  - job: adf_datafactoryName_build
    steps:
    - task: NodeTool@0
      inputs:
        versionSpec: 16.x
      displayName: Installing Node Version 16.x
    - task: Npm@1
      inputs:
        command: 'install'
        verbose: False
        workingDir: $(Build.Repository.LocalPath)/adf_scripts
      displayName: 'Install npm package'
    - task: Npm@1
      inputs:
        command: 'custom'
        customCommand: run build export $(Build.Repository.LocalPath)/adf /subscriptions/0000000-0000-0000-0000-0000000000000/resourceGroups/$(adfResourceGroupName)/providers/Microsoft.DataFactory/factories/$(adfName)
        workingDir: $(Build.Repository.LocalPath)/adf_scripts
    - task: PublishPipelineArtifact@1
      displayName: 'Publish Pipeline Artifact ADFTemplates '
      inputs:
        targetPath: $(Build.Repository.LocalPath)/adf_scripts
        artifact: ADFTemplates
- stage: adfdemo_deploy
  jobs:
  - deployment: adfdemo_infrastructure
    environment:
      name: dev
    - name: powerShellScriptPath
      value: ../ADFTemplates/PrePostDeploymentScript.ps1
    - name: ARMTemplatePath
      value: ADFTemplates/ARMTemplateForFactory.json
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzurePowerShell@5
            displayName: Stop ADF Triggers
            inputs:
              scriptType: 'FilePath'
              ConnectedServiceNameARM: AzureDevServiceConnection
              scriptPath: ../ADFTemplates/PrePostDeploymentScript.ps1
              ScriptArguments: -armTemplate "ADFTemplates/ARMTemplateForFactory.json" -ResourceGroupName $(adfResourceGroupName) -DataFactoryName $(adfName) -predeployment $true -deleteDeployment $false
              errorActionPreference: stop
              FailOnStandardError: False
              azurePowerShellVersion: azurePowerShellVersion
              preferredAzurePowerShellVersion: 3.1.0
              pwsh: False
              workingDirectory: ../
          - task: AzureResourceManagerTemplateDeployment@3
            inputs:
              deploymentScope: Resource Group
              azureResourceManagerConnection: AzureDevServiceConnection
              action: Create Or Update Resource Group
              resourceGroupName: $(adfResourceGroupName) 
              location: eastus
              csmFile: $(Agent.BuildDirectory)/ADFTemplates/ARMTemplateForFactory.json
              csmParametersFile: $(Agent.BuildDirectory)/ADFTemplates/parameters/dev.eus.parameters.json
              overrideParameters: ''
              deploymentMode: Incremental
          - task: AzurePowerShell@5
            displayName: Start ADF Triggers
            inputs:
              scriptType: 'FilePath'
              ConnectedServiceNameARM: AzureDevServiceConnection
              scriptPath: ../ADFTemplates/PrePostDeploymentScript.ps1
              ScriptArguments: -armTemplate "ADFTemplates/ARMTemplateForFactory.json" -ResourceGroupName $(adfResourceGroupName)  -DataFactoryName $(adfName) -predeployment $false -deleteDeployment $true
              errorActionPreference: stop
              FailOnStandardError: False
              azurePowerShellVersion: azurePowerShellVersion
              preferredAzurePowerShellVersion: 3.1.0
              pwsh: False
              workingDirectory: ../`

The templated version is available at https://github.com/JFolberth/TheYAMLPipelineOne/tree/main/stages

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

RamanathanChinnappan-MSFT commented 1 year ago

@JFolberth

Thanks for your feedback! We will investigate and update as appropriate.

JFolberth commented 1 year ago

@Joe-Code for visibility as well.

RamanathanChinnappan-MSFT commented 1 year ago

@nabhishek

could you please review this and update as appropriate.

RamanathanChinnappan-MSFT commented 1 year ago

@JFolberth

I've delegated this to @nabhishek, a content author, to review and share their valuable insights.

JFolberth commented 1 year ago

@nabhishek any update on this?

jonburchel commented 1 year ago

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, and the timeline for resolution may vary based on resourcing, so we've created an internal work item to incorporate your suggestions. We are closing this issue for now, but feel free to comment here as necessary. #please-close