search

hashicorp / terraform-provider-azurerm

Terraform provider for Azure Resource Manager
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
Mozilla Public License 2.0
4.62k stars 4.66k forks source link

Public IP returns empty, but visible from azure portal #3985

Closed rukawata closed 5 years ago

rukawata commented 5 years ago

Community Note

Terraform (and AzureRM Provider) Version

Affected Resource(s)

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

resource "azurerm_resource_group" "rp" {
  name     = "nancyc-linuxvm-empty-ip"
  location = "westus2"
}

# Prepare the network and public IP for linux VM
resource "azurerm_virtual_network" "vnet" {
  name                = "nancyc-linuxvm-empty-ip-vnet"
  address_space       = ["10.0.0.0/21"]
  location            = azurerm_resource_group.rp.location
  resource_group_name = azurerm_resource_group.rp.name
}

resource "azurerm_subnet" "subnet" {
  name                      = "nancyc-linuxvm-empty-ip-subnet"
  resource_group_name       = azurerm_resource_group.rp.name
  virtual_network_name      = azurerm_virtual_network.vnet.name
  address_prefix            = "10.0.0.0/24"
  network_security_group_id = azurerm_network_security_group.sap_nsg.id
}

resource "azurerm_network_security_group" "sap_nsg" {
  name                = "nancyc-linuxvm-empty-ip-nsg"
  resource_group_name = azurerm_resource_group.rp.name
  location            = azurerm_resource_group.rp.location

  security_rule {
    name                       = "SSH"
    priority                   = 101
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_range     = 22
    source_address_prefixes    = ["131.107.0.0/16", "67.160.0.0/16", "167.220.0.0/16"]
    destination_address_prefix = "*"
  }
}

resource "azurerm_public_ip" "pip" {
  name                = "nancyc-linuxvm-empty-ip-pip"
  location            = azurerm_resource_group.rp.location
  resource_group_name = azurerm_resource_group.rp.name
  allocation_method   = "Dynamic"
}

resource "azurerm_network_interface" "pip-nic" {
  name                      = "nancyc-linuxvm-empty-ip-pip-nic"
  location                  = azurerm_resource_group.rp.location
  resource_group_name       = azurerm_resource_group.rp.name
  network_security_group_id = azurerm_network_security_group.sap_nsg.id

  ip_configuration {
    name                          = "publicIP"
    subnet_id                     = azurerm_subnet.subnet.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.pip.id
  }
}

# A storage account to store boot diagnostics for a VM
resource "random_id" "randomId" {
  keepers = {
    # Generate a new id only when a new resource group is defined.
    resource_group = azurerm_resource_group.rp.name
  }

  byte_length = 8
}

resource "azurerm_storage_account" "bootdiagstorageaccount" {
  name                     = "diag${random_id.randomId.hex}"
  resource_group_name      = azurerm_resource_group.rp.name
  location                 = azurerm_resource_group.rp.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

# Create linux VM with Public IP
resource "azurerm_virtual_machine" "tmp-vm" {
  name                          = "nancyc-linuxvm-empty-ip-tmp-vm"
  location                      = azurerm_resource_group.rp.location
  resource_group_name           = azurerm_resource_group.rp.name
  network_interface_ids         = [azurerm_network_interface.pip-nic.id]
  vm_size                       = "Standard_DS1_v2"
  delete_os_disk_on_termination = true

  storage_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }

  storage_os_disk {
    name              = "myosdisk1"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }

  os_profile {
    computer_name  = "vmlinux"
    admin_username = "nancyc"
    admin_password = "Admin123!"
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }

  # Install ansible on linux VM
  connection {
    host     = azurerm_public_ip.pip.ip_address
    type     = "ssh"
    user     = "nancyc"
    password = "Admin123!"
  }
  provisioner "remote-exec" {
    inline = [
      "sudo apt update",
      "sudo apt install software-properties-common",
      "sudo apt-add-repository --yes --update ppa:ansible/ansible",
      "sudo apt install -y ansible > /home/nancyc/install.log",
    ]
  }

  depends_on = [azurerm_public_ip.pip]
}

Debug Output

terraform_debug_201907311053.log

Panic Output

Expected Behavior

The azurerm_public_ip should be able to pass the public IP to provisioner for the remote execution.

Actual Behavior

The public IP address is empty therefore remote execution fails. See below error example:

azurerm_virtual_machine.tmp-vm: Still creating... [6m10s elapsed] azurerm_virtual_machine.tmp-vm (remote-exec): Connecting to remote host via SSH... azurerm_virtual_machine.tmp-vm (remote-exec): Host: azurerm_virtual_machine.tmp-vm (remote-exec): User: nancyc azurerm_virtual_machine.tmp-vm (remote-exec): Password: true azurerm_virtual_machine.tmp-vm (remote-exec): Private key: false azurerm_virtual_machine.tmp-vm (remote-exec): Certificate: false azurerm_virtual_machine.tmp-vm (remote-exec): SSH Agent: false azurerm_virtual_machine.tmp-vm (remote-exec): Checking Host Key: false azurerm_virtual_machine.tmp-vm: Still creating... [6m20s elapsed] azurerm_virtual_machine.tmp-vm: Still creating... [6m30s elapsed]

Error: timeout - last error: dial tcp :22: connectex: No connection could be made because the target machine actively refused it.

Steps to Reproduce

  1. terraform apply

Below are different approaches I tested - none works:

  1. separate the provisioner from vm block, using a null_resource
  2. force a depends_on between the provisioner and public IP

Important Factoids

References

rukawata commented 5 years ago

If I try to put the public IP in the output, it's empty as well. Although the public IP is showing in the azure portal. This tells me that it is generated based on the dependency, just not being populated correctly.

output "tmp_vm_ip" {
  value = azurerm_public_ip.pip.ip_address
}
mcharriere commented 5 years ago

@rukawata Did you try to create a null_resource for the provisioner with a depends_on on the VM? Azure allocates the dynamic public IP when it is attached to the VM and not when the resource is created.

rukawata commented 5 years ago

null_resource

Hi @mcharriere, The example I gave does not involve null_resource. I was just saying, I also tried to separate the provisioner from the VM using null_resource, I did not see the public IP either.

Actually, even without remote exec, I see the public IP is empty (although from azure portal, the vm does have public IP address).

main.tf

resource "azurerm_resource_group" "rp" {
  name     = "nancyc-linuxvm-empty-ip"
  location = "westus2"
}

# Prepare the network and public IP for linux VM
resource "azurerm_virtual_network" "vnet" {
  name                = "nancyc-linuxvm-empty-ip-vnet"
  address_space       = ["10.0.0.0/21"]
  location            = azurerm_resource_group.rp.location
  resource_group_name = azurerm_resource_group.rp.name
}

resource "azurerm_subnet" "subnet" {
  name                      = "nancyc-linuxvm-empty-ip-subnet"
  resource_group_name       = azurerm_resource_group.rp.name
  virtual_network_name      = azurerm_virtual_network.vnet.name
  address_prefix            = "10.0.0.0/24"
  network_security_group_id = azurerm_network_security_group.sap_nsg.id
}

resource "azurerm_network_security_group" "sap_nsg" {
  name                = "nancyc-linuxvm-empty-ip-nsg"
  resource_group_name = azurerm_resource_group.rp.name
  location            = azurerm_resource_group.rp.location

  security_rule {
    name                       = "SSH"
    priority                   = 101
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_range     = 22
    source_address_prefixes    = ["131.107.0.0/16", "67.160.0.0/16", "167.220.0.0/16"]
    destination_address_prefix = "*"
  }
}

resource "azurerm_public_ip" "pip" {
  name                = "nancyc-linuxvm-empty-ip-pip"
  location            = azurerm_resource_group.rp.location
  resource_group_name = azurerm_resource_group.rp.name
  allocation_method   = "Dynamic"
}

resource "azurerm_network_interface" "pip-nic" {
  name                      = "nancyc-linuxvm-empty-ip-pip-nic"
  location                  = azurerm_resource_group.rp.location
  resource_group_name       = azurerm_resource_group.rp.name
  network_security_group_id = azurerm_network_security_group.sap_nsg.id

  ip_configuration {
    name                          = "publicIP"
    subnet_id                     = azurerm_subnet.subnet.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.pip.id
  }
}

# A storage account to store boot diagnostics for a VM
resource "random_id" "randomId" {
  keepers = {
    # Generate a new id only when a new resource group is defined.
    resource_group = azurerm_resource_group.rp.name
  }

  byte_length = 8
}

resource "azurerm_storage_account" "bootdiagstorageaccount" {
  name                     = "diag${random_id.randomId.hex}"
  resource_group_name      = azurerm_resource_group.rp.name
  location                 = azurerm_resource_group.rp.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

# Create linux VM with Public IP
resource "azurerm_virtual_machine" "tmp-vm" {
  name                          = "nancyc-linuxvm-empty-ip-tmp-vm"
  location                      = azurerm_resource_group.rp.location
  resource_group_name           = azurerm_resource_group.rp.name
  network_interface_ids         = [azurerm_network_interface.pip-nic.id]
  vm_size                       = "Standard_DS1_v2"
  delete_os_disk_on_termination = true

  storage_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }

  storage_os_disk {
    name              = "myosdisk1"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }

  os_profile {
    computer_name  = "vmlinux"
    admin_username = "nancyc"
    admin_password = "Admin123!"
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }
}

output "tmp_vm_ip" {
  value = azurerm_public_ip.pip.ip_address
}

Output:

Apply complete! Resources: 9 added, 0 changed, 0 destroyed.

Outputs:

tmp_vm_ip =

If this is also expected behavior, what would be the correct way to get the public IP address?

Many thanks! Ruka

mcharriere commented 5 years ago

According to the docs, with Dynamic publics IP you should use the data source azurerm_public_ip. But you wont be able to use the data source and the connection associated with the VM at the same time.

I think that the best way to achieve that is using the fqdn instead of the IP.

rukawata commented 5 years ago

According to the docs, with Dynamic publics IP you should use the data source azurerm_public_ip. But you wont be able to use the data source and the connection associated with the VM at the same time.

I think that the best way to achieve that is using the fqdn instead of the IP.

Thank you @mcharriere ! With fqdn does work.

ghost commented 5 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!