search

nusenu / ansible-relayor

An Ansible Role for Tor Relay Operators
GNU General Public License v3.0
247 stars 43 forks source link

TASK [nusenu.relayor : Use a single private IPv4 address if we have no public IPv4 address : Fails #212

Closed jakevis closed 4 years ago

jakevis commented 4 years ago

Describe the bug On a resent update we have started to see failures when running relayor against our HBSD hosts:

PLAY [tor1] *******************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************
ok: [tor1.eo]

TASK [nusenu.relayor : Check for min. ansible version requirement] ************************************************************************************************************
ok: [tor1.eo] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [nusenu.relayor : Check for local requirements] **************************************************************************************************************************
ok: [tor1.eo]

TASK [nusenu.relayor : Ensure tor_ContactInfo is set] *************************************************************************************************************************
ok: [tor1.eo] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [nusenu.relayor : Ensure we do not create more than two instances per IP] ************************************************************************************************
ok: [tor1.eo] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [nusenu.relayor : Ensure preconditions for tor_dedicatedExitIP are met (enough public IP addresses on the system)] *******************************************************
skipping: [tor1.eo]

TASK [nusenu.relayor : Abort if we use a centralized/common DNS resolver (Google, Quad9, CloudFlare, OpenDNS, Level3). See https://torproject.org/relay-guide#DNSonExitRelays (Exits only)] ***
ok: [tor1.eo]

TASK [nusenu.relayor : Set OS specific variables] *****************************************************************************************************************************
ok: [tor1.eo]

TASK [nusenu.relayor : Use a single private IPv4 address if we have no public IPv4 address] ***********************************************************************************
fatal: [tor1.eo]: FAILED! => {"msg": "The conditional check 'tor_v4ips == []' failed. The error was: An unhandled exception occurred while templating '{{ tor_available_public_ipv4s[0:tor_maxPublicIPs] }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Unexpected templating type error occurred on ({{ tor_available_public_ipv4s[0:tor_maxPublicIPs] }}): 'bool' object has no attribute '__getitem__'\n\nThe error appears to be in '/home/jake/.ansible/roles/nusenu.relayor/tasks/ip-list.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Use a single private IPv4 address if we have no public IPv4 address\n  ^ here\n"}

The config of the yaml file is:

---
- hosts: tor1
  vars:
    tor_ContactInfo: tech@emeraldonion.org
    tor_signingkeylifetime_days: 90
    tor_nicknamefile: /opt/emeraldonion/relayor/relayname.csv
    tor_maxPublicIPs: 18
    tor_ports:
        - { orport: 443, dirport: 80}
    tor_ExitRelay: True
    tor_ExitRelaySetting_file: /opt/emeraldonion/relayor/relaymode.csv
    tor_offline_masterkey_dir: /opt/emeraldonion/offlinekeys
    tor_Sandbox: 1
    tor_NoExec: 1
    tor_ExitPolicy:
        - accept *:*
    tor_ExitPolicy_file: /opt/emeraldonion/relayor/relayexit.csv
    tor_IPv6: True
    tor_dedicatedExitIP: False
    tor_ExitNoticePage: True
    tor_AbuseEmailAddress: abuse@emeraldonion.org
    tor_enableControlSocket: True
  roles:
   - nusenu.relayor

A similar config works against ubuntu fine, so I assume its an issue specific to *BSD

To Reproduce ansible-playbook tor1.yaml

Version information (please include the following information):

Playbook information See above

OS information FreeBSD 12.1-STABLE-HBSD (HARDENEDBSD)

Debug information Please add the output of your ansible-playbook run using "-vvv".

ansible-playbook -i hosts tor1.yaml -vvv
ansible-playbook 2.9.15
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/jake/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.17 (default, Sep 30 2020, 13:38:04) [GCC 7.5.0]
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /opt/emeraldonion/relayor/hosts as it did not pass its verify_file() method
script declined parsing /opt/emeraldonion/relayor/hosts as it did not pass its verify_file() method
auto declined parsing /opt/emeraldonion/relayor/hosts as it did not pass its verify_file() method
Parsed /opt/emeraldonion/relayor/hosts inventory source with ini plugin
statically imported: /home/jake/.ansible/roles/nusenu.relayor/tasks/ip-list.yml
statically imported: /home/jake/.ansible/roles/nusenu.relayor/tasks/configure.yml

PLAYBOOK: tor1.yaml ***********************************************************************************************************************************************************
1 plays in tor1.yaml

PLAY [tor1] *******************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************
task path: /opt/emeraldonion/relayor/tor1.yaml:2
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
<tor1.eo> (0, '/home/jake\n', '')
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/jake/.ansible/tmp `"&& mkdir "` echo /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545 `" && echo ansible-tmp-1604432938.85-12896-221715687314545="` echo /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545 `" ) && sleep 0'"'"''
<tor1.eo> (0, 'ansible-tmp-1604432938.85-12896-221715687314545=/home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545\n', '')
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/system/setup.py
<tor1.eo> PUT /home/jake/.ansible/tmp/ansible-local-128893PB5Cf/tmpEjNJam TO /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545/AnsiballZ_setup.py
<tor1.eo> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 '[tor1.eo]'
<tor1.eo> (0, 'sftp> put /home/jake/.ansible/tmp/ansible-local-128893PB5Cf/tmpEjNJam /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545/AnsiballZ_setup.py\n', '')
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'chmod u+x /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545/ /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545/AnsiballZ_setup.py && sleep 0'"'"''
<tor1.eo> (0, '', '')
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 -tt tor1.eo '/bin/sh -c '"'"'/usr/local/bin/python /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545/AnsiballZ_setup.py && sleep 0'"'"''
<tor1.eo> (0, 'UnboundLocalError("local variable \'pagesize\' referenced before assignment")\r\n\r\n{"ansible_facts": {"ansible_lsb": {}, "ansible_virtualization_type": "", "ansible_virtualization_role": "", "ansible_user_id": "jake", "ansible_user_uid": 1001, "ansible_user_gid": 1001, "ansible_user_gecos": "Jake", "ansible_user_dir": "/home/jake", "ansible_user_shell": "/bin/csh", "ansible_real_user_id": 1001, "ansible_effective_user_id": 1001, "ansible_real_group_id": 1001, "ansible_effective_group_id": 1001, "ansible_ssh_host_key_rsa_public": "AAAAB3NzaC1yc2EAAAADAQABAAABAQCwqQnwkjtwbDoxCJFmpIFLzCPdSxO94ByffYn9JAjPah4VoAe8Ov6wScLn56w8OZpNNQQRXOzHvFVnyoRcpJHUwS/+qhtvAWQ5Jt19+Z3Vg8z2dRvWhMZTnuP/FfqiAMw9hnayq7/Zb9HEvMigkpn/49cKZs3W6T9JiCOVh2a+tUBSEBi58xkBecN+dVurCsIBTkpYajCrAv4pDqnM+mKm/29c0rfgrrNmUN99Dx0UPlEemwkJnszNLAQ3psnAJ7e4OwmaFZMMejF6LqodC9Kf+AufB4mo2nJl9ifuJhgHUYM6g0tSuGsc86QO+Zy/HnA9uXJC3aT1Szq/ALSm75H7", "ansible_ssh_host_key_ecdsa_public": "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLRS9BzUaOa6+0KMWXDZSJHqaC3nDWVX/xypyybFx3+CZggs02leurAvO26e8KRGhCO5brTB8tL45yF8DhWiE0=", "ansible_ssh_host_key_ed25519_public": "AAAAC3NzaC1lZDI1NTE5AAAAIBkXbdY46obK1aDGcVriuakJrvodX4IY2IGnDmWDA6Zs", "ansible_python": {"version": {"major": 3, "minor": 7, "micro": 9, "releaselevel": "final", "serial": 0}, "version_info": [3, 7, 9, "final", 0], "executable": "/usr/local/bin/python", "has_sslcontext": true, "type": "cpython"}, "ansible_is_chroot": true, "ansible_interfaces": [], "ansible_default_ipv4": {}, "ansible_default_ipv6": {}, "ansible_all_ipv4_addresses": [], "ansible_all_ipv6_addresses": [], "ansible_system": "FreeBSD", "ansible_kernel": "12.1-STABLE-HBSD", "ansible_kernel_version": "FreeBSD 12.1-STABLE-HBSD #0 : Thu May 28 20:04:48 UTC 2020     root@updater-01.md.hardenedbsd.lan:/usr/obj/usr/src/amd64.amd64/sys/HARDENEDBSD", "ansible_machine": "amd64", "ansible_python_version": "3.7.9", "ansible_fqdn": "tor1.emeraldonion.org", "ansible_hostname": "tor1", "ansible_nodename": "tor1.emeraldonion.org", "ansible_domain": "emeraldonion.org", "ansible_userspace_bits": "64", "ansible_architecture": "amd64", "ansible_env": {"LC_CTYPE": "C.UTF-8", "VENDOR": "amd", "SSH_CLIENT": "103.47.63.125 45494 22", "LOGNAME": "jake", "PAGER": "less", "OSTYPE": "FreeBSD", "MACHTYPE": "x86_64", "MAIL": "/var/mail/jake", "PATH": "/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/jake/bin", "EDITOR": "vi", "HOST": "tor1.emeraldonion.org", "REMOTEHOST": "eo-bastion.emeraldonion.org", "PWD": "/home/jake", "GROUP": "jake", "TERM": "xterm", "SSH_TTY": "/dev/pts/1", "HOME": "/home/jake", "USER": "jake", "SSH_CONNECTION": "103.47.63.125 45494 23.129.64.180 22", "HOSTTYPE": "FreeBSD", "SHELL": "/bin/csh", "BLOCKSIZE": "K", "SHLVL": "1"}, "ansible_fips": false, "ansible_selinux": {"status": "Missing selinux Python library"}, "ansible_selinux_python_present": false, "ansible_dns": {"nameservers": ["1.0.0.1", "1.1.1.1", "2606:4700:4700::1111", "2606:4700:4700::1001"]}, "ansible_hostnqn": "", "ansible_distribution": "FreeBSD", "ansible_distribution_release": "12.1-STABLE-HBSD", "ansible_distribution_version": "12.1", "ansible_distribution_major_version": "12", "ansible_os_family": "FreeBSD", "ansible_date_time": {"year": "2020", "month": "11", "weekday": "Tuesday", "weekday_number": "2", "weeknumber": "44", "day": "03", "hour": "11", "minute": "49", "second": "07", "epoch": "1604432947", "date": "2020-11-03", "time": "11:49:07", "iso8601_micro": "2020-11-03T19:49:07.475360Z", "iso8601": "2020-11-03T19:49:07Z", "iso8601_basic": "20201103T114907475360", "iso8601_basic_short": "20201103T114907", "tz": "PST", "tz_offset": "-0800"}, "ansible_iscsi_iqn": "", "ansible_apparmor": {"status": "disabled"}, "ansible_fibre_channel_wwn": [], "ansible_local": {}, "ansible_pkg_mgr": "pkgng", "ansible_service_mgr": "service", "gather_subset": ["all"], "module_setup": true}, "invocation": {"module_args": {"gather_timeout": 10, "gather_subset": ["all"], "filter": "*", "fact_path": "/etc/ansible/facts.d"}}}\r\n', 'Shared connection to tor1.eo closed.\r\n')
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'rm -f -r /home/jake/.ansible/tmp/ansible-tmp-1604432938.85-12896-221715687314545/ > /dev/null 2>&1 && sleep 0'"'"''
<tor1.eo> (0, '', '')
ok: [tor1.eo]
META: ran handlers

TASK [nusenu.relayor : Check for min. ansible version requirement] ************************************************************************************************************
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/main.yml:3
ok: [tor1.eo] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [nusenu.relayor : Check for local requirements] **************************************************************************************************************************
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/main.yml:14
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: jake
<127.0.0.1> EXEC /bin/sh -c 'echo ~jake && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/jake/.ansible/tmp `"&& mkdir "` echo /home/jake/.ansible/tmp/ansible-tmp-1604432948.34-12911-148024620183477 `" && echo ansible-tmp-1604432948.34-12911-148024620183477="` echo /home/jake/.ansible/tmp/ansible-tmp-1604432948.34-12911-148024620183477 `" ) && sleep 0'
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<127.0.0.1> PUT /home/jake/.ansible/tmp/ansible-local-128893PB5Cf/tmpQD4SGP TO /home/jake/.ansible/tmp/ansible-tmp-1604432948.34-12911-148024620183477/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/jake/.ansible/tmp/ansible-tmp-1604432948.34-12911-148024620183477/ /home/jake/.ansible/tmp/ansible-tmp-1604432948.34-12911-148024620183477/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/jake/.ansible/tmp/ansible-tmp-1604432948.34-12911-148024620183477/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/jake/.ansible/tmp/ansible-tmp-1604432948.34-12911-148024620183477/ > /dev/null 2>&1 && sleep 0'
ok: [tor1.eo] => {
    "changed": false,
    "cmd": "command -V /bin/bash && command -V tor && command -V openssl && command -V sort && command -V uniq && command -V wc && command -V cut && command -V xargs && command -V sed\n",
    "delta": "0:00:00.001726",
    "end": "2020-11-03 11:49:08.536210",
    "invocation": {
        "module_args": {
            "_raw_params": "command -V /bin/bash && command -V tor && command -V openssl && command -V sort && command -V uniq && command -V wc && command -V cut && command -V xargs && command -V sed\n",
            "_uses_shell": true,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "warn": true
        }
    },
    "rc": 0,
    "start": "2020-11-03 11:49:08.534484",
    "stderr": "",
    "stderr_lines": [],
    "stdout": "/bin/bash is /bin/bash\ntor is /usr/sbin/tor\nopenssl is /usr/bin/openssl\nsort is /usr/bin/sort\nuniq is /usr/bin/uniq\nwc is /usr/bin/wc\ncut is /usr/bin/cut\nxargs is /usr/bin/xargs\nsed is /bin/sed",
    "stdout_lines": [
        "/bin/bash is /bin/bash",
        "tor is /usr/sbin/tor",
        "openssl is /usr/bin/openssl",
        "sort is /usr/bin/sort",
        "uniq is /usr/bin/uniq",
        "wc is /usr/bin/wc",
        "cut is /usr/bin/cut",
        "xargs is /usr/bin/xargs",
        "sed is /bin/sed"
    ]
}

TASK [nusenu.relayor : Ensure tor_ContactInfo is set] *************************************************************************************************************************
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/main.yml:25
ok: [tor1.eo] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [nusenu.relayor : Ensure we do not create more than two instances per IP] ************************************************************************************************
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/main.yml:33
ok: [tor1.eo] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [nusenu.relayor : Ensure preconditions for tor_dedicatedExitIP are met (enough public IP addresses on the system)] *******************************************************
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/main.yml:42
skipping: [tor1.eo] => {
    "changed": false,
    "skip_reason": "Conditional result was False"
}

TASK [nusenu.relayor : Abort if we use a centralized/common DNS resolver (Google, Quad9, CloudFlare, OpenDNS, Level3). See https://torproject.org/relay-guide#DNSonExitRelays (Exits only)] ***
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/main.yml:54
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
<tor1.eo> (0, '/home/jake\n', '')
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/jake/.ansible/tmp `"&& mkdir "` echo /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006 `" && echo ansible-tmp-1604432948.8-12944-242882870780006="` echo /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006 `" ) && sleep 0'"'"''
<tor1.eo> (0, 'ansible-tmp-1604432948.8-12944-242882870780006=/home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006\n', '')
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<tor1.eo> PUT /home/jake/.ansible/tmp/ansible-local-128893PB5Cf/tmpgIamRH TO /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006/AnsiballZ_command.py
<tor1.eo> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 '[tor1.eo]'
<tor1.eo> (0, 'sftp> put /home/jake/.ansible/tmp/ansible-local-128893PB5Cf/tmpgIamRH /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006/AnsiballZ_command.py\n', '')
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'chmod u+x /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006/ /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006/AnsiballZ_command.py && sleep 0'"'"''
<tor1.eo> (0, '', '')
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 -tt tor1.eo '/bin/sh -c '"'"'/usr/local/bin/python /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006/AnsiballZ_command.py && sleep 0'"'"''
<tor1.eo> (1, '\r\n{"msg": "non-zero return code", "cmd": ["grep", "-qE", "-e", "^nameserver 8\\\\.8\\\\.8\\\\.8[[:space:]]*$", "-e", "^nameserver 8\\\\.8\\\\.4\\\\.4[[:space:]]*$", "-e", "^nameserver 2001:4860:4860:.*:8888[[:space:]]*$", "-e", "^nameserver 2001:4860:4860:.*:8844[[:space:]]*$", "-e", "^nameserver 208\\\\.67\\\\.222\\\\.123[[:space:]]*$", "-e", "^nameserver 208\\\\.67\\\\.220\\\\.123[[:space:]]*$", "-e", "^nameserver 208\\\\.67\\\\.222\\\\.222[[:space:]]*$", "-e", "^nameserver 208\\\\.67\\\\.220\\\\.220[[:space:]]*$", "-e", "^nameserver 2620:119:35:.*:35[[:space:]]*$", "-e", "^nameserver 2620:119:53:.*:53[[:space:]]*$", "-e", "^nameserver 1\\\\.1\\\\.1\\\\.1[[:space:]]*$", "-e", "^nameserver 1\\\\.0\\\\.0\\\\.1[[:space:]]*$", "-e", "^nameserver 2606:4700:4700:.*:1111[[:space:]]*$", "-e", "^nameserver 2606:4700:4700:.*:1001[[:space:]]*$", "-e", "^nameserver 9\\\\.9\\\\.9\\\\.9[[:space:]]*$", "-e", "^nameserver 9\\\\.9\\\\.9\\\\.10[[:space:]]*$", "-e", "^nameserver 149\\\\.112\\\\.112\\\\.10[[:space:]]*$", "-e", "^nameserver 149\\\\.112\\\\.112\\\\.112[[:space:]]*$", "-e", "^nameserver 2620:fe:.*:fe[[:space:]]*$", "-e", "^nameserver 2620:fe:.*:9[[:space:]]*$", "-e", "^nameserver 2620:fe:.*:10[[:space:]]*$", "-e", "^nameserver 2620:fe:.*:fe:10[[:space:]]*$", "-e", "^nameserver 4\\\\.2\\\\.2\\\\.[1-6][[:space:]]*$", "/etc/resolv.conf"], "stdout": "", "stderr": "", "rc": -12, "start": "2020-11-03 11:49:10.301961", "end": "2020-11-03 11:49:10.756640", "delta": "0:00:00.454679", "changed": true, "failed": true, "invocation": {"module_args": {"_raw_params": "grep -qE -e \\"^nameserver 8\\\\.8\\\\.8\\\\.8[[:space:]]*$\\" -e \\"^nameserver 8\\\\.8\\\\.4\\\\.4[[:space:]]*$\\" -e \\"^nameserver 2001:4860:4860:.*:8888[[:space:]]*$\\" -e \\"^nameserver 2001:4860:4860:.*:8844[[:space:]]*$\\" -e \\"^nameserver 208\\\\.67\\\\.222\\\\.123[[:space:]]*$\\" -e \\"^nameserver 208\\\\.67\\\\.220\\\\.123[[:space:]]*$\\" -e \\"^nameserver 208\\\\.67\\\\.222\\\\.222[[:space:]]*$\\" -e \\"^nameserver 208\\\\.67\\\\.220\\\\.220[[:space:]]*$\\" -e \\"^nameserver 2620:119:35:.*:35[[:space:]]*$\\" -e \\"^nameserver 2620:119:53:.*:53[[:space:]]*$\\" -e \\"^nameserver 1\\\\.1\\\\.1\\\\.1[[:space:]]*$\\" -e \\"^nameserver 1\\\\.0\\\\.0\\\\.1[[:space:]]*$\\" -e \\"^nameserver 2606:4700:4700:.*:1111[[:space:]]*$\\" -e \\"^nameserver 2606:4700:4700:.*:1001[[:space:]]*$\\" -e \\"^nameserver 9\\\\.9\\\\.9\\\\.9[[:space:]]*$\\" -e \\"^nameserver 9\\\\.9\\\\.9\\\\.10[[:space:]]*$\\" -e \\"^nameserver 149\\\\.112\\\\.112\\\\.10[[:space:]]*$\\" -e \\"^nameserver 149\\\\.112\\\\.112\\\\.112[[:space:]]*$\\" -e \\"^nameserver 2620:fe:.*:fe[[:space:]]*$\\" -e \\"^nameserver 2620:fe:.*:9[[:space:]]*$\\" -e \\"^nameserver 2620:fe:.*:10[[:space:]]*$\\" -e \\"^nameserver 2620:fe:.*:fe:10[[:space:]]*$\\" -e \\"^nameserver 4\\\\.2\\\\.2\\\\.[1-6][[:space:]]*$\\"  /etc/resolv.conf", "warn": true, "_uses_shell": false, "stdin_add_newline": true, "strip_empty_ends": true, "argv": null, "chdir": null, "executable": null, "creates": null, "removes": null, "stdin": null}}}\r\n', 'Shared connection to tor1.eo closed.\r\n')
<tor1.eo> Failed to connect to the host via ssh: Shared connection to tor1.eo closed.
<tor1.eo> ESTABLISH SSH CONNECTION FOR USER: None
<tor1.eo> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jake/.ansible/cp/0ecae0b2d4 tor1.eo '/bin/sh -c '"'"'rm -f -r /home/jake/.ansible/tmp/ansible-tmp-1604432948.8-12944-242882870780006/ > /dev/null 2>&1 && sleep 0'"'"''
<tor1.eo> (0, '', '')
ok: [tor1.eo] => {
    "changed": false,
    "cmd": [
        "grep",
        "-qE",
        "-e",
        "^nameserver 8\\.8\\.8\\.8[[:space:]]*$",
        "-e",
        "^nameserver 8\\.8\\.4\\.4[[:space:]]*$",
        "-e",
        "^nameserver 2001:4860:4860:.*:8888[[:space:]]*$",
        "-e",
        "^nameserver 2001:4860:4860:.*:8844[[:space:]]*$",
        "-e",
        "^nameserver 208\\.67\\.222\\.123[[:space:]]*$",
        "-e",
        "^nameserver 208\\.67\\.220\\.123[[:space:]]*$",
        "-e",
        "^nameserver 208\\.67\\.222\\.222[[:space:]]*$",
        "-e",
        "^nameserver 208\\.67\\.220\\.220[[:space:]]*$",
        "-e",
        "^nameserver 2620:119:35:.*:35[[:space:]]*$",
        "-e",
        "^nameserver 2620:119:53:.*:53[[:space:]]*$",
        "-e",
        "^nameserver 1\\.1\\.1\\.1[[:space:]]*$",
        "-e",
        "^nameserver 1\\.0\\.0\\.1[[:space:]]*$",
        "-e",
        "^nameserver 2606:4700:4700:.*:1111[[:space:]]*$",
        "-e",
        "^nameserver 2606:4700:4700:.*:1001[[:space:]]*$",
        "-e",
        "^nameserver 9\\.9\\.9\\.9[[:space:]]*$",
        "-e",
        "^nameserver 9\\.9\\.9\\.10[[:space:]]*$",
        "-e",
        "^nameserver 149\\.112\\.112\\.10[[:space:]]*$",
        "-e",
        "^nameserver 149\\.112\\.112\\.112[[:space:]]*$",
        "-e",
        "^nameserver 2620:fe:.*:fe[[:space:]]*$",
        "-e",
        "^nameserver 2620:fe:.*:9[[:space:]]*$",
        "-e",
        "^nameserver 2620:fe:.*:10[[:space:]]*$",
        "-e",
        "^nameserver 2620:fe:.*:fe:10[[:space:]]*$",
        "-e",
        "^nameserver 4\\.2\\.2\\.[1-6][[:space:]]*$",
        "/etc/resolv.conf"
    ],
    "delta": "0:00:00.454679",
    "end": "2020-11-03 11:49:10.756640",
    "failed_when_result": false,
    "invocation": {
        "module_args": {
            "_raw_params": "grep -qE -e \"^nameserver 8\\.8\\.8\\.8[[:space:]]*$\" -e \"^nameserver 8\\.8\\.4\\.4[[:space:]]*$\" -e \"^nameserver 2001:4860:4860:.*:8888[[:space:]]*$\" -e \"^nameserver 2001:4860:4860:.*:8844[[:space:]]*$\" -e \"^nameserver 208\\.67\\.222\\.123[[:space:]]*$\" -e \"^nameserver 208\\.67\\.220\\.123[[:space:]]*$\" -e \"^nameserver 208\\.67\\.222\\.222[[:space:]]*$\" -e \"^nameserver 208\\.67\\.220\\.220[[:space:]]*$\" -e \"^nameserver 2620:119:35:.*:35[[:space:]]*$\" -e \"^nameserver 2620:119:53:.*:53[[:space:]]*$\" -e \"^nameserver 1\\.1\\.1\\.1[[:space:]]*$\" -e \"^nameserver 1\\.0\\.0\\.1[[:space:]]*$\" -e \"^nameserver 2606:4700:4700:.*:1111[[:space:]]*$\" -e \"^nameserver 2606:4700:4700:.*:1001[[:space:]]*$\" -e \"^nameserver 9\\.9\\.9\\.9[[:space:]]*$\" -e \"^nameserver 9\\.9\\.9\\.10[[:space:]]*$\" -e \"^nameserver 149\\.112\\.112\\.10[[:space:]]*$\" -e \"^nameserver 149\\.112\\.112\\.112[[:space:]]*$\" -e \"^nameserver 2620:fe:.*:fe[[:space:]]*$\" -e \"^nameserver 2620:fe:.*:9[[:space:]]*$\" -e \"^nameserver 2620:fe:.*:10[[:space:]]*$\" -e \"^nameserver 2620:fe:.*:fe:10[[:space:]]*$\" -e \"^nameserver 4\\.2\\.2\\.[1-6][[:space:]]*$\"  /etc/resolv.conf",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "warn": true
        }
    },
    "msg": "non-zero return code",
    "rc": -12,
    "start": "2020-11-03 11:49:10.301961",
    "stderr": "",
    "stderr_lines": [],
    "stdout": "",
    "stdout_lines": []
}

TASK [nusenu.relayor : Set OS specific variables] *****************************************************************************************************************************
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/main.yml:63
ok: [tor1.eo] => {
    "ansible_facts": {
        "tor_ConfDir": "/usr/local/etc/tor/enabled",
        "tor_DataDir": "/var/db/tor-instances"
    },
    "ansible_included_var_files": [
        "/home/jake/.ansible/roles/nusenu.relayor/vars/os_FreeBSD.yml"
    ],
    "changed": false
}

TASK [nusenu.relayor : Use a single private IPv4 address if we have no public IPv4 address] ***********************************************************************************
task path: /home/jake/.ansible/roles/nusenu.relayor/tasks/ip-list.yml:3
fatal: [tor1.eo]: FAILED! => {
    "msg": "The conditional check 'tor_v4ips == []' failed. The error was: An unhandled exception occurred while templating '{{ tor_available_public_ipv4s[0:tor_maxPublicIPs] }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Unexpected templating type error occurred on ({{ tor_available_public_ipv4s[0:tor_maxPublicIPs] }}): 'bool' object has no attribute '__getitem__'\n\nThe error appears to be in '/home/jake/.ansible/roles/nusenu.relayor/tasks/ip-list.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Use a single private IPv4 address if we have no public IPv4 address\n  ^ here\n"
}

PLAY RECAP ********************************************************************************************************************************************************************
tor1.eo                    : ok=7    changed=0    unreachable=0    failed=1    skipped=1    rescued=0    ignored=0
nusenu commented 4 years ago

TASK [nusenu.relayor : Use a single private IPv4 address if we have no public IPv4 address] *** fatal: [tor1.eo]: FAILED! => {"msg": "The conditional check 'tor_v4ips == []' failed. The error was: An unhandled exception occurred while templating '{{ tor_available_public_ipv4s[0:tor_maxPublicIPs] }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Unexpected templating type error occurred on ({{ tor_available_public_ipv4s[0:tor_maxPublicIPs] }}): 'bool' object has no attribute 'getitem'\n\nThe error appears to be in '/home/jake/.ansible/roles/nusenu.relayor/tasks/ip-list.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Use a single private IPv4 address if we have no public IPv4 address\n ^ here\n"}

A similar config works against ubuntu fine, so I assume its an issue specific to *BSD

What python version does HBSD have vs. Ubuntu?

Version information (please include the following information):

  • ansible version ansible 2.9.15
  • ansible-relayor version (please try to reproduce the bug against the git master branch) nusenu.relayor (v20.1.0)

I suspect this is related to

80

https://github.com/ansible/ansible/issues/14829 https://github.com/ansible-collections/ansible.netcommon/issues/136

Does your ansible/netcommon version contain the fix? Can you reproduce the issue with ansible 2.9.13?

jakevis commented 4 years ago

Hi @nusenu HBSD is running Python 3.7.9 Ubuntu is running Python 3.8.5

I have v1.4.1 of netcommon installed as well (installed from galaxy - but no idea how it should be integrated/used).

I never noticed an issue with 2.9.13; but I also cant get back to it either to verify (its not in the PPA anymore). jumping forward to ansible 2.10.3 results in a slightly different set of warnings - same gist:

TASK [nusenu.relayor : Use a single private IPv4 address if we have no public IPv4 address] ***********************************************************************************
[WARNING]: packaging Python module unavailable; unable to validate collection Ansible version requirements
fatal: [tor1.eo]: FAILED! => {"msg": "The conditional check 'tor_v4ips == []' failed. The error was: An unhandled exception occurred while templating '{{ tor_available_public_ipv4s[0:tor_maxPublicIPs] }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while templating '{{ ansible_all_ipv4_addresses| ipv4('address') | ipv4('public') }}'. Error was a <class 'ansible.errors.AnsibleFilterError'>, original message: The ipv4 filter requires python's netaddr be installed on the ansible controller\n\nThe error appears to be in '/home/jake/.ansible/roles/nusenu.relayor/tasks/ip-list.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Use a single private IPv4 address if we have no public IPv4 address\n  ^ here\n"}
jakevis commented 4 years ago

So a little bit more testing overnight - I was wrong about it still working on Ubuntu, I dont know what I was seeing... but yes, its failing there as well.

I rolled all the way back to https://github.com/ansible/ansible/archive/v2.9.12.tar.gz using pip install methods and its still broken there unfortunately as well. Ill revert the version of ansible and relayor now and see what it does..

jakevis commented 4 years ago

Now at: 2.9.1 nusenu.relayor, v19.1.7

And still getting the same set of errors.. so I guess this is outside of the realm of ansible (since I know we had it working here). Ill start walking python back here now...

jakevis commented 4 years ago

ansible-playbook 2.9.15 python version = 2.7.17 (default, Sep 30 2020, 13:38:04) [GCC 7.5.0] nusenu.relayor, v20.1.0

This combo works - so I am unblocked ;) Happy for this to be closed - or I can pull other logs et al if it would help ID what the issue was when using python 3.7/3.8

nusenu commented 4 years ago

Jake Vis:

ansible-playbook 2.9.15 python version = 2.7.17 (default, Sep 30 2020, 13:38:04) [GCC 7.5.0]

are you referring to the python version on the control machine or the relay?

Thanks, for your help debugging this.

nusenu commented 4 years ago

original message: The ipv4 filter requires python's netaddr be installed on the ansible controller

looks like you are missing one of the requirements: python netaddr

nusenu commented 4 years ago

Since I was unable to reproduce your issue I assume the following happened:

If you can confirm that assumption, we can close this issue.

jakevis commented 4 years ago

Since I was unable to reproduce your issue I assume the following happened:

  • You used to have python 2.x on your control machine with netaddr installed.
  • you upgraded to python 3.x (which didn't have netaddr installed)
  • the missing netaddr package caused the error

If you can confirm that assumption, we can close this issue.

Yeah let me close this out; now we are up and working again ill spend some time and upgrade + test in a more controlled manor. I know I stuffed up there in the first rebuild and didnt install netaddr; when I did, it started working on ubuntu with python 3.x (3.x on both the control machine, and tor node), but it didnt work for the HBSD boxes (also on 3.x). Dropping everything back to 2.X fixed everything... but its obviously not where we want to be.

Thanks again for your help and your tools @nusenu .