azurerm_data_factory_integration_runtime_self_hosted not present in DF backed by DevOps Git Repository

[piotr-gbyliczek]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.28

• provider.azurerm v2.16.0

Affected Resource(s)

azurerm_data_factory azurerm_data_factory_integration_runtime_self_hosted

Terraform Configuration Files

resource "azurerm_data_factory" "data_factory" {
  name                        = var.resource_name
  location                    = var.resource_location
  resource_group_name         = var.resource_group_name
  identity {
    type = "SystemAssigned"
  }
  vsts_configuration {
    account_name                = var.resource_vsts_acc_name
    branch_name                 = var.resource_vsts_branch_name
    repository_name         = var.resource_vsts_repo_name
    project_name                = var.resource_vsts_proj_name
    root_folder                 = var.resource_vsts_root_folder
    tenant_id                     = var.resource_tenant_id
  }
  tags = var.resource_tags
}
resource "azurerm_data_factory_integration_runtime_self_hosted" "integration_runtime" {
  name                = "${var.resource_name}-integration-runtime"
  resource_group_name = var.resource_group_name
  data_factory_name   = var.resource_name
}

Debug Output

Panic Output

Expected Behavior

Integration Runtimes connection should be created in Data Factory and visible in "Azure DevOps GIT" mode, also, a connection file should be present in git repository.

Actual Behavior

Integration Runtimes connection is created in Data Factory mode, and not present in git repository as a file.

Steps to Reproduce

Create resources.

1. terraform apply

Important Factoids

References

[SebastianBalle]

Any progress on this issue? I am facing similar issues.

[pindey]

Anyone know of a workaround in the meantime?

[ragzzy]

is there an update to this please - thank you

[drjwelch]

I have the same issue. If your factory is going to git mode then you need the IR description in JSON in the factory's repo otherwise it exists but doesn't show unless you switch back to live mode (very confusing!).

The same issue will arise if you create any other adf resources like datasets or linked services in terraform.

As workarounds:

• we could probably inject the files with a copy task in the build pipeline - but then it won't know (without some kind of jiggery-pokery in the pipeline) when terraform applys/destroys the IR to keep the repo in sync with the tf resources;
• or create the repo with azuredevops provider and initialise it from a static copy elsewhere that contains the "baseline" resources that tf creates, but again a sync problem.

Ideally we could do with a terraform resource maybe in azuredevops provider that you could use to check a new file into a git repo - for the use case here it's only 4 lines of JSON so could be inline in the resource definition. Something like:

resource "azuredevops_git_repository_file" "new_file" {
  project_id           = azuredevops_project.project.id
  repository_id           = azuredevops_git_repository.repo.id
  content = <<CONTENT
{
    "name": "myIntegrationRuntime",
    "properties": {
        "type": "SelfHosted"
    }
}
CONTENT
}

Maybe optionally a fileURI = ... if we didn't want the file inline in a content key, perhaps?

And/or maybe an optional repository_file {} block available in all the adf contained resources with the inline content or fileURI

We could create a feature request if this is useful to several folks (I'd write it myself if I knew how to do it!)

[favoretti]

HI there and thank you for reporting this issue. As @drjwelch rightfully states - the self-hosted IR has to be present both in TF and and ADF's git (yuck). We are also doing something in the lines of what @drjwelch suggests. Unfortunately there's no other way around it from what I know.