search

radekg / terraform-provisioner-ansible

Ansible with Terraform 0.14.x
Apache License 2.0
572 stars 100 forks source link

IP hostname fails to resolve in provisioner runs fine standalone. #128

Closed xophere closed 5 years ago

xophere commented 5 years ago

Steps to reproduce

Terraform apply.

Expected behavior

Ansible playbooks applied. ...

Actual behavior

Terraform apply fails with the below output. The host is available and you can ssh with that identity right after the failure. You can also run the ansible remote standalone and it applies just fine. This would happen occasionally before we moved from the default adapter in EC2. But now that we are using a defined adapter this happens every time. Our goals is to have dual nics configured and called out in the provisioning step. We are getting very close to giving up on the provider.

aws_instance.city_lb (ansible): Using /etc/ansible/ansible.cfg as config file

aws_instance.city_lb (ansible): PLAY [Post boot config of City Loadbalancers] **

aws_instance.city_lb (ansible): TASK [Gathering Facts] ***** aws_instance.city_lb (ansible): fatal: [54.149.150.46]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname [54.149.150.46]: Name or service not known", "unreachable": true}

aws_instance.city_lb (ansible): PLAY RECAP ***** aws_instance.city_lb (ansible): 54.149.150.46 : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0

The direct ssh to the EIP works. And the below works:

ANSIBLE_FORCE_COLOR=true ANSIBLE_ROLES_PATH=/usr/share/ansible/roles:/etc/ansible/roles:./aws/city_lb/roles ansible-playbook ./city_lb/city_lb.yml --inventory-file=./inventory --forks=5 --vault-id='/keybase/team/UUUUUUU/ansible-vault-password.txt' --user='UUUU' --ssh-extra-args='-p 22 -o ConnectTimeout=10 -o ConnectionAttempts=10' -vvv --extra-vars='{"aws_account_id":"AAAAAAAAAA","cluster_size":"3","dev_team":"userland","dmz_private_ip":"172.31.1.80","env":"chris","github_org":"CypherpunkArmory","output_directory":"/keybase/team/UUUUUUU/chris","vault_file":"/keybase/team/UUUUUUUU/ansible-vault.yml","vpc_active_subnet":"172.31.0.0","vpc_vpn_subnet":"172.16.0.0","vpn_domain":"chris.DDDDDDDD.com"}'

...

Configuration

Terraform version: me@localhost:~/work/grid/aws$ ansible --version ansible 2.8.1 config file = /etc/ansible/ansible.cfg configured module search path = [u'/home/UUUUU/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/dist-packages/ansible executable location = /usr/bin/ansible python version = 2.7.16 (default, Apr 6 2019, 01:42:57) [GCC 8.3.0] me@localhost:~/work/grid/aws$ terraform --version Terraform v0.11.14

Your version of Terraform is out of date! The latest version is 0.12.3. You can update by downloading from www.terraform.io/downloads.html

terraform-provisioner-ansible version/SHA:

Terraform file / provisioner configuration: provisioner "ansible" { when = "create"

connection {
  host = "${aws_eip.city_lb_ip.*.public_ip}"
  user = "XXXXXX"
  type = "ssh"
  timeout = "5m"
}

ansible_ssh_settings {
  host_key_checking = false
}

plays {
  playbook = {
    file_path = "${path.module}/city_lb/city_lb.yml"
    roles_path = ["${path.module}/city_lb/roles"]
  }

  hosts = ["${self.public_ip}"]
  vault_id = ["${var.output_directory}/ansible-vault-password.txt"]
  verbose = true
  extra_vars = {
    env               = "${terraform.workspace}"
    dmz_private_ip    = "${aws_instance.dmz.private_ip}"
    cluster_size      = "${var.city_hosts}"
    vault_file        = "${var.output_directory}/ansible-vault.yml"
    output_directory  = "${var.output_directory}/${terraform.workspace}"
    github_org        = "${var.github_org}"
    dev_team          = "XXXXXXXX"
    aws_account_id    = "${data.aws_caller_identity.current.account_id}"
    vpc_active_subnet = "172.31.0.0"
    vpc_vpn_subnet    = "172.16.0.0"
    vpn_domain        = "${terraform.workspace == "prod" ? XXXXX.io" : join(".", list(terraform.workspace, "XXXXXXXXXX.com"))}"
  }
}

}

Terraform run log: aws_instance.city_lb: Creating... ami: "" => "ami-XX" arn: "" => "" associate_public_ip_address: "" => "" availability_zone: "" => "" cpu_core_count: "" => "" cpu_threads_per_core: "" => "" ebs_block_device.#: "" => "" ephemeral_block_device.#: "" => "" get_password_data: "" => "false" host_id: "" => "" iam_instance_profile: "" => "lb_host_profile" instance_state: "" => "" instance_type: "" => "t2.micro" ipv6_address_count: "" => "" ipv6_addresses.#: "" => "" key_name: "" => "" monitoring: "" => "true" network_interface.#: "" => "1" network_interface.60705503.delete_on_termination: "" => "false" network_interface.60705503.device_index: "" => "0" network_interface.60705503.network_interface_id: "" => "eni-XXX" network_interface_id: "" => "" password_data: "" => "" placement_group: "" => "" primary_network_interface_id: "" => "" private_dns: "" => "" private_ip: "" => "" public_dns: "" => "" public_ip: "" => "" root_block_device.#: "" => "" security_groups.#: "" => "" subnet_id: "" => "" tags.%: "" => "5" tags.District: "" => "city" tags.Environment: "" => "chris" tags.Name: "" => "city_lbchris" tags.Role: "" => "lb" tags.Usage: "" => "app" tenancy: "" => "" user_data: "" => "CCC" volume_tags.%: "" => "" vpc_security_group_ids.#: "" => "" .... aws_route.vpn_route: Creation complete after 1s (ID: XXXXX) aws_route.private_vpn_route: Creation complete after 1s (ID: XXXX) aws_route.private_route: Creation complete after 1s (ID: r-XXXXX) aws_db_instance.city_rds: Still creating... (4m10s elapsed) aws_route53_record.dmz_wildcard: Still creating... (10s elapsed) aws_instance.city_lb: Still creating... (10s elapsed) aws_instance.city_host.2: Still creating... (10s elapsed) aws_instance.city_host.0: Still creating... (10s elapsed) aws_instance.city_host.1: Still creating... (10s elapsed) aws_db_instance.city_rds: Still creating... (4m20s elapsed) aws_route53_record.dmz_wildcard: Still creating... (20s elapsed) aws_instance.city_lb: Still creating... (20s elapsed) aws_instance.city_host.2: Still creating... (20s elapsed) aws_instance.city_host.0: Still creating... (20s elapsed) aws_instance.city_host.1: Still creating... (20s elapsed) aws_db_instance.city_rds: Still creating... (4m30s elapsed) aws_route53_record.dmz_wildcard: Still creating... (30s elapsed) aws_instance.city_lb: Still creating... (30s elapsed) aws_instance.city_host.2: Still creating... (30s elapsed) aws_instance.city_host.0: Still creating... (30s elapsed) aws_instance.city_host.1: Still creating... (30s elapsed) aws_instance.city_host[0]: Creation complete after 33s (ID: i-SSSSSS) aws_instance.city_host[1]: Creation complete after 33s (ID: i-SSSSSS) aws_instance.city_host[2]: Creation complete after 33s (ID: i-SSSSSS) aws_db_instance.city_rds: Still creating... (4m40s elapsed) aws_route53_record.dmz_wildcard: Still creating... (40s elapsed) aws_instance.city_lb: Still creating... (40s elapsed) aws_instance.city_lb: Provisioning with 'ansible'... aws_instance.city_lb (ansible): host key for '[54.XXX.XXX.46]' not received yet; retrying... aws_instance.city_lb (ansible): host key for '[54.XXX.XXX.46]' not received yet; retrying... aws_route53_record.dmzwildcard: Creation complete after 48s (ID: ZL4TTKYD8B75T*.chris.testinghole.com_A) aws_db_instance.city_rds: Still creating... (4m50s elapsed) aws_instance.city_lb: Still creating... (50s elapsed) aws_instance.city_lb (ansible): Generating temporary ansible inventory... aws_instance.city_lb (ansible): Writing temporary ansible inventory to '/tmp/temporary-ansible-inventory485255331'... aws_instance.city_lb (ansible): Ansible inventory written. aws_instance.city_lb (ansible): running local command: ANSIBLE_FORCE_COLOR=true ANSIBLE_ROLES_PATH=/home/XXXX/work/grid/aws/city_lb/roles ansible-playbook /home/schafer/work/grid/aws/city_lb/city_lb.yml --inventory-file='/tmp/temporary-ansible-inventory485255331' --extra-vars='{"aws_account_id":"RRRRRRRRRRR","cluster_size":"3","dev_team":"userland","dmz_private_ip":"172.31.1.92","env":"chris","github_org":"CypherpunkArmory","output_directory":"/keybase/team/EEEEEEEEE/chris","vault_file":"/keybase/team/EEEEEEEEE/ansible-vault.yml","vpc_active_subnet":"172.31.0.0","vpc_vpn_subnet":"172.16.0.0","vpn_domain":"chris.EEEEEEE.com"}' --forks=5 --vault-id='/keybase/team/SSSSSSS/ansible-vault-password.txt' --verbose --user='UUUUUU' --ssh-extra-args='-p 22 -o ConnectTimeout=10 -o ConnectionAttempts=10 -o UserKnownHostsFile=/tmp/5a7a86c5-6672-4388-b3b7-c533903b840d862464340' aws_instance.city_lb (ansible): Executing: ["/bin/sh" "-c" "ANSIBLE_FORCE_COLOR=true ANSIBLE_ROLES_PATH=/home/UUUUUU/work/grid/aws/city_lb/roles ansible-playbook /home/UUUUUU/work/grid/aws/city_lb/city_lb.yml --inventory-file='/tmp/temporary-ansible-inventory485255331' --extra-vars='{\"aws_account_id\":\"AAAAAAAA\",\"cluster_size\":\"3\",\"dev_team\":\"UUUUUUU\",\"dmz_private_ip\":\"172.31.1.92\",\"env\":\"MMMMM\",\"github_org\":\"CypherpunkArmory\",\"output_directory\":\"/keybase/team/userland/chris\",\"vault_file\":\"/keybase/team/UUUUUUUU/ansible-vault.yml\",\"vpc_active_subnet\":\"172.31.0.0\",\"vpc_vpn_subnet\":\"172.16.0.0\",\"vpn_domain\":\"chris.TTTTTTTTT.com\"}' --forks=5 --vault-id='/keybase/team/UUUUUUUUU/ansible-vault-password.txt' --verbose --user='alan' --ssh-extra-args='-p 22 -o ConnectTimeout=10 -o ConnectionAttempts=10 -o UserKnownHostsFile=/tmp/5a7a86c5-6672-4388-b3b7-c533903b840d862464340'"] aws_instance.city_lb (ansible): Using /etc/ansible/ansible.cfg as config file

aws_instance.city_lb (ansible): PLAY [Post boot config of City Loadbalancers] **

aws_instance.city_lb (ansible): TASK [Gathering Facts] ***** aws_instance.city_lb (ansible): fatal: [54.XXX.XXX.46]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname [54.149.150.46]: Name or service not known", "unreachable": true}

aws_instance.city_lb (ansible): PLAY RECAP ***** aws_instance.city_lb (ansible): 54.XXX.XXX.46 : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0

radekg commented 5 years ago

Hi @xophere, thank you for the report. Looking at your configuration and the logs:

connection {
  host = "${aws_eip.city_lb_ip.*.public_ip}"
  user = "XXXXXX"
  type = "ssh"
  timeout = "5m"
}
Could not resolve hostname [54.149.150.46].

I think what is happening, you are passing an array of hosts to connection.host (judging by the square brackets in the log output). connection.host takes a single host as a string. normally, one would expect something like:

connection {
  host = "${aws_eip.city_lb_ip.0.public_ip}"
  user = "XXXXXX"
  type = "ssh"
  timeout = "5m"
}

In your case, a list of hosts would becomes literal [54.149.150.46] instead of 54.149.150.46. Obviously, [54.149.150.46] is not a valid host name, hence the error you are seeing.

radekg commented 5 years ago

Closing due to no activity.