hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.31k stars 9.49k forks source link

Use private endpoints for Azure backend #28048

Open jelisejev opened 3 years ago

jelisejev commented 3 years ago

Current Terraform Version

0.14.7.

Use-cases

Azure storage accounts have a private endpoint feature, that allows creating a private connection between a VNET and the Storage Account, without having to go over the public Internet. Containers created using a private endpoint become available using an IP address in the RFC 1918 IP ranges. You can also create a private DNS to access these endpoints for convenience.

This is an important security feature for many companies.

Attempted Solutions

To use private endpoints, we need to be able to define a custom endpoint for the blob container. As far as I understood, this is not currently supported. I'm not even sure if it's possible in the underlying library used to work with Azure blobs.

Proposal

I guess there should be an additional parameter to the backend configuration, that will allow overwriting the endpoint used to access the blob container.

Tbohunek commented 3 years ago

Yes, you can create a Private Endpoint for Storage Account. You also need to register it in privatelink.blob.core.windows.net DNS zone that is resolvable from the client.

resource "azurerm_private_endpoint" "storage_blob" {
  name                = azurerm_storage_account.example.name
  location            = azurerm_storage_account.example.location
  resource_group_name = azurerm_storage_account.example.resource_group_name
  subnet_id           = data.azurerm_subnet.example.id

  private_service_connection {
    name                           = "pe-${azurerm_storage_account.example.name}"
    private_connection_resource_id = azurerm_storage_account.example.id
    is_manual_connection           = false
    subresource_names               = ["blob"]
  }
}

# I will add the DNS config later
# Vnet network policies also need to be enabled

However, if you run your terraform apply from a VM in Azure, it can leverage Virtual network service Endpoints so that you don't need Private Endpoints at all.

krzkal91 commented 1 year ago

Hi,

What is the status of this issue?

Quite a significant feature from the security perspective

Keltirion commented 1 year ago

Hi, i'm also interested about the status of this. Disabling public endpoints is quite common.

krzkal91 commented 1 year ago

@Keltirion it seems to be working well after all - as long as your dns configuration (private dns zone or else) is correct

Keltirion commented 1 year ago

@krzkal91 that is fine. There is an issue for example if you run over P2S VPN and you do not replace DNS setting on the client, then you will be hitting public endpoint of the storage account because Azure is doing cname record for privatelink.blob.core.windows.net in public dns. I have second resolver in my vpn client for internal domain but i cannot force backend in terraform to use that fqdn because by design it hits always the same one which results in public endpoint.