search

microsoft / terraform-provider-azuredevops

Terraform Azure DevOps provider
https://www.terraform.io/docs/providers/azuredevops/
MIT License
385 stars 275 forks source link

provider "azuredevops". Error: The resource cannot be found. #449

Open ngeegoh opened 3 years ago

ngeegoh commented 3 years ago

Terraform (and Azure DevOps Provider) Version: 0.12.19

Affected Resource(s)

Terraform Configuration Files

#main.tf
terraform {
   backend "azurerm" {
   }
}

provider "azurerm" {
  version = "=2.46.0"
  features {}
}

provider "azuredevops" {
  version = ">=0.1.0"
  org_service_url = "https://dev.azure.com/<org>"
  personal_access_token = "<pat>"
}

Error Output

Error: The resource cannot be found.

on main.tf line 32, in provider "azuredevops": 32: provider "azuredevops" {

Actual Behavior

It was okay a few hours ago. Then suddenly the resource cannot be found for some reason.

Steps to Reproduce

I run the terraform through azure devops pipeline to create azure devops k8s service connection. It was fine a few hours ago then it suddenly failed to find the provider. I am sure sure the ado k8s service connection terrafrom script works fine as it did work before and i didnt change anything.

ngeegoh commented 3 years ago

Just wondering if this merged PR might potentially cause this issue? https://github.com/microsoft/terraform-provider-azuredevops/commit/473a1c7d5b94511a4aa74212f9535aee2a1c0f76

ngeegoh commented 3 years ago

I even tried to upgrade Terraform version to 1.0.5, but it still complains about provider azuredevops not found in terraform registry. 

 Error: The resource cannot be found.
│ 
│   with provider["registry.terraform.io/microsoft/azuredevops"],
│   on main.tf line 47, in provider "azuredevops":
│   47: provider "azuredevops" {
│
xuzhang3 commented 3 years ago

Just wondering if this merged PR might potentially cause this issue? 473a1c7

This feature is not released.

ngeegoh commented 3 years ago

Thanks @xuzhang3, do you know what might go wrong with the terraform azuredevops package that can not be found in the registry?

The provider name is azuredevops not azurerm

xuzhang3 commented 3 years ago

@ngeegoh You should add the azure devops configuration: 

terraform {
  required_providers {
    azuredevops = {
      source = "microsoft/azuredevops"
      version = "=0.1.7"
    }
  }
}
ngeegoh commented 3 years ago

I have tried that as well but it doesn't work, everything was fine last night, not sure why.

xuzhang3 commented 3 years ago

@ngeegoh You are using the legacy configuration, you should upgrade the configuration to new format(terraform registry):

terraform {
  required_providers {
    azuredevops = {
      source = "microsoft/azuredevops"
      version = "=0.1.7"
    }
    azurerm = {
      source = "hashicorp/azurerm"
      version = "=2.46.0"
    }
  }
}

terraform {
  backend "azurerm" {
  }
}

provider "azurerm" {
  features {}
}
ngeegoh commented 3 years ago

@xuzhang3 I needed to add provider "azuredevops",

terraform {
  required_providers {
    azuredevops = {
      source = "microsoft/azuredevops"
      version = "=0.1.7"
    }
    azurerm = {
      source = "hashicorp/azurerm"
      version = "=2.75.0"
    }
  }
}

terraform {
  backend "azurerm" {
  }
}

provider "azurerm" {
  features {}
}

provider "azuredevops" {
  org_service_url = "https://dev.azure.com/<org>"
  personal_access_token = "<AZDO_PERSONAL_ACCESS_TOKEN>"
}

otherwise, it will complain with this error :

╷
│ Error: Invalid provider configuration
│ 
│ Provider "registry.terraform.io/microsoft/azuredevops" requires explicit
│ configuration. Add a provider block to the root module and configure the
│ provider's required arguments as described in the provider documentation.
│ 
╵
╷
│ Error: the personal access token is required
│ 
│   with provider["registry.terraform.io/microsoft/azuredevops"],
│   on <empty> line 0:
│   (source code not available)
│

But even if I added the config provider block for azuredevops, its still the same issue:

╷
│ Error: The resource cannot be found.
│ 
│   with provider["registry.terraform.io/microsoft/azuredevops"],
│   on main.tf line 48, in provider "azuredevops":
│   48: provider "azuredevops" {
│
xuzhang3 commented 3 years ago

@ngeegoh I can initial the configuration with terraform init(terraform v1.0.5), can you delete the .terraform folder and retry? The .terraform structure is different between the legacy and new format.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding microsoft/azuredevops versions matching "0.1.7"...
- Finding hashicorp/azurerm versions matching "2.46.0"...
- Installing microsoft/azuredevops v0.1.7...
- Installed microsoft/azuredevops v0.1.7 (signed by a HashiCorp partner, key ID 6F0B91BDE98478CF)
- Installing hashicorp/azurerm v2.46.0...
- Installed hashicorp/azurerm v2.46.0 (signed by HashiCorp)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
ngeegoh commented 3 years ago

@xuzhang3 I can initial the configure with terraform init as well, but fails at terraform plan when it couldnt find the resource and provider of azuredevops.

This is my terraform init output

Initializing the backend...

Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding microsoft/azuredevops versions matching "0.1.7"...
- Finding hashicorp/azurerm versions matching "2.75.0"...
- Installing microsoft/azuredevops v0.1.7...
- Installed microsoft/azuredevops v0.1.7 (signed by a HashiCorp partner, key ID 6F0B91BDE98478CF)
- Installing hashicorp/azurerm v2.75.0...
- Installed hashicorp/azurerm v2.75.0 (signed by HashiCorp)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Then it failed at terraform plan:

/opt/hostedtoolcache/terraform/1.0.5/x64/terraform providers

Providers required by configuration:
.
├── provider[registry.terraform.io/microsoft/azuredevops] 0.1.7
└── provider[registry.terraform.io/hashicorp/azurerm] 2.75.0

/opt/hostedtoolcache/terraform/1.0.5/x64/terraform plan -out=tfplan
Acquiring state lock. This may take a few moments...
╷
│ Error: The resource cannot be found.
│ 
│   with provider["registry.terraform.io/microsoft/azuredevops"],
│   on main.tf line 48, in provider "azuredevops":
│   48: provider "azuredevops" {
│

This is my tf file that has azuredevops resource

data "azuredevops_project" "p" {
  name = "app-api"
}

resource "azuredevops_serviceendpoint_kubernetes" "se_azure_sub" {
  project_id            = data.azuredevops_project.p.id
  service_endpoint_name = "app-api-ado-svc-connection"
  apiserver_url         = "https://${azurerm_kubernetes_cluster.app_api.fqdn}"
  authorization_type    = "AzureSubscription"

  azure_subscription {
    subscription_id   = "****"
    subscription_name = "<mysub>"
    tenant_id         = "****"
    resourcegroup_id  = azurerm_resource_group.app_api.name
    namespace         = "default"
    cluster_name      = azurerm_kubernetes_cluster.app_api.name
  }
}
xuzhang3 commented 3 years ago

@ngeegoh This strange, if you can initial the providers, then during the plan phase, terraform should find the providers. Is this backend state shared with some who may modify these files? The log shows that the error occurred when terraform handling the state.

ngeegoh commented 3 years ago

This tfstate file is not shared with anyone, it was locked when terraform plan is running

ngeegoh commented 3 years ago

I even tried to delete the whole tfstate file and re-ran, it still failed with same error at terraform plan.

xuzhang3 commented 3 years ago

Does terraform plan run under the right permission?

ngeegoh commented 3 years ago

Yes it does as it was working fine before with azuredevops, and even without azuredevops provider its still working fine.

ngeegoh commented 3 years ago

Hmm could I miss anything in the above tf file that has azuredevops resource (azuredevops service connection resource)?? I didnt change anything since it worked before.

xuzhang3 commented 3 years ago

@ngeegoh Can you run terraoform plan with local state? Disable the backend configuration.

terraform {
  backend "azurerm" {
  }
}
ngeegoh commented 3 years ago

@xuzhang3 I am running terraform in AzureDevOps pipeline by using terraform backend, its working fine, but not with azuredevops provider. I tried with local same error with resource not found. Are you able to reproduce this error from your end?

xuzhang3 commented 3 years ago

@ngeegoh Can you share full TF script and which Terraform version you use?

ngeegoh commented 3 years ago

@xuzhang3 I am using terraform version 1.0.5 in AzureDevOps pipeline. These are my tf scripts that I used (4):

#main.tf
terraform {
  required_providers {
    azuredevops = {
      source = "microsoft/azuredevops"
      version = "=0.1.7"
    }
    azurerm = {
      source = "hashicorp/azurerm"
      version = "=2.75.0"
    }
  }
}

terraform {
  backend "azurerm" {
  }
}

provider "azurerm" {
  features {}
}

provider "azuredevops" {
  org_service_url = "https://dev.azure.com/<org>"
  personal_access_token = "__AZDO_PERSONAL_ACCESS_TOKEN__"
}
#ado-aks-service-connection.tf
data "azuredevops_project" "p" {
  name = "app-api"
}

resource "azuredevops_serviceendpoint_kubernetes" "se_azure_sub" {
  project_id            = data.azuredevops_project.p.id
  service_endpoint_name = "app-api-ado-svc-connection"
  apiserver_url         = "https://${azurerm_kubernetes_cluster.app_api.fqdn}"
  authorization_type    = "AzureSubscription"

  azure_subscription {
    subscription_id   = "****"
    subscription_name = "az-subscription"
    tenant_id         = "****"
    resourcegroup_id  = azurerm_resource_group.app_api.name
    namespace         = "default"
    cluster_name      = azurerm_kubernetes_cluster.app_api.name
  }
}
#azure-kubernetes-service.tf
resource "azurerm_resource_group" "app_api" {
  name     = "app-api-rg"
  location = "East US"
}

resource "azurerm_kubernetes_cluster" "app_api" {
  name                = "app-api-aks"
  location            = azurerm_resource_group.app_api.location
  resource_group_name = azurerm_resource_group.app_api.name
  dns_prefix          = "app-api-aks"

  default_node_pool {
    name       = "default"
    node_count = 1
    vm_size    = "Standard_D2_v2"
  }

  identity {
    type = "SystemAssigned"
  }

  addon_profile {
    http_application_routing {
        enabled = true
    }
  }
}
#output.tf
output "http_application_routing_zone_name" {
  value = azurerm_kubernetes_cluster.app_api.addon_profile[0].http_application_routing[0].http_application_routing_zone_name
}

output "fqdn" {
  value = azurerm_kubernetes_cluster.app_api.fqdn
}

output "service_endpoint_name" {
  value = azuredevops_serviceendpoint_kubernetes.se_azure_sub.service_endpoint_name
}
xuzhang3 commented 3 years ago

@ngeegoh I cannot reproduce your error., no matter AzureDevOps pipeline, mac, linux or windows , all works for me.

anthonytps commented 2 years ago

got same issue here, I'm using terraform 1.1.4 on Ubuntu 18.04

Radical-Dave commented 2 years ago

What I'm running can be found here: https://github.com/Radical-Dave/Auto/tree/main/data/providers/tf/tasks/core-devops-azure

For the most part, its the same. Just playing with different backend settings, etc. But no matter what I've tried, same error. Additional oddity, is that it HAS successfully generated a plan a few times and even successfully applied once, maybe twice.

pdarji-bakkt commented 2 years ago

I'm facing the same issue. Does anyone have a fix for this issue yet? Reported the issue here too: https://github.com/microsoft/terraform-provider-azuredevops/issues/652

brezina-petr commented 1 year ago

One note only: I use provider registry.terraform.io/microsoft/azuredevops v0.4.0. on Ubuntu. It works nice. I gave error 

Error: The resource cannot be found.
│ 
│   with provider["registry.terraform.io/microsoft/azuredevops"],

I found that this error gives me incorrect value in parameters in org_service_url 

provider "azuredevops" { 
    org_service_url       = "https://dev.azure.com/xxx"
    personal_access_token = "xxxxxxxyyyyyyyxxxxxxxnyyyyyyyyyyyxxxxxx"
}

May be could check correct value in org_service_url as first in this error type.

seblatre commented 6 months ago

Thanks @brezina-petr, I faced the same issue and indeed the solution was to fix the org_service_url value in the provider configuration. Thanks a lot for your comment!