search

samdoran / ansible-collection-macos

Ansible collection of roles and modules for use on macOS
8 stars 2 forks source link

[BUG] Xcode CLI tools installation is getting skipped on fresh macOS 13.0 (stable) installs #9

Closed webknjaz closed 1 year ago

webknjaz commented 1 year ago

For some reason, the check for the presence of Xcode CLI tools is misbehaving, causing the installation to be skipped, followed by a failing check for the python3 install that fails and opens a window asking for the Xcode CLI tools install. Logging this here for now. It appears that the SSH output somehow leaks into the task results, causing this. The string that the check is expecting is present in the stdout value, though, not stderr.

$ ansible-playbook guest-bootstrap-python.yml -vvv \
    --ask-pass \
    --ask-become-pass \
    --limit ansible-core-ci-template-macos-13.0-experiment-7
ansible-playbook 2.10.17
  config file = [...]/ansible.cfg
  configured module search path = ['~/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = ~/.pyenv/versions/venv-py3.8.6/lib/python3.8/site-packages/ansible
  executable location = ~/.pyenv/versions/venv-py3.8.6/bin/ansible-playbook
  python version = 3.8.6 (default, Oct 26 2020, 13:09:51) [GCC 10.2.0]
Using [...]/ansible.cfg as config file
SSH password: 
BECOME password[defaults to SSH password]: 
host_list declined parsing [...]/inventory.yml as it did not pass its verify_file() method
script declined parsing [...]/inventory.yml as it did not pass its verify_file() method
Parsed [...]/inventory.yml inventory source with yaml plugin
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: guest-bootstrap-python.yml *************************************************************************************************************************
1 plays in guest-bootstrap-python.yml

PLAY [Pre-provision a fresh macOS 12.3+ install with Python 3] ***********************************************************************************************
META: ran handlers

TASK [samdoran.macos.command_line_tools : Check for Xcode Command Line Tools] ********************************************************************************
task path: ~/.ansible/collections/ansible_collections/samdoran/macos/roles/command_line_tools/tasks/bootstrap.yml:1
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J administrator@bastion.hostname.com:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e -tt ansible-core-ci-template-macos-13.0-experiment-7 'sudo -H -S  -p "[sudo via ansible, key=opiqtcjruxfbodaghsusojogpwexnrwy] password:" -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-opiqtcjruxfbodaghsusojogpwexnrwy ; stat /Library/Developer/CommandLineTools'"'"''
Escalation succeeded
<ansible-core-ci-template-macos-13.0-experiment-7> (1, b'\r\nstat: /Library/Developer/CommandLineTools: stat: No such file or directory\r\n', b'Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n')
<ansible-core-ci-template-macos-13.0-experiment-7> Failed to connect to the host via ssh: Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.
ok: [ansible-core-ci-template-macos-13.0-experiment-7] => {
    "changed": false,
    "failed_when_result": false,
    "msg": "non-zero return code",
    "rc": 1,
    "stderr": "Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n",
    "stderr_lines": [
        "Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed."
    ],
    "stdout": "\r\nstat: /Library/Developer/CommandLineTools: stat: No such file or directory\r\n",
    "stdout_lines": [
        "",
        "stat: /Library/Developer/CommandLineTools: stat: No such file or directory"
    ]
}

TASK [samdoran.macos.command_line_tools : Create hidden install file] ****************************************************************************************
task path: ~/.ansible/collections/ansible_collections/samdoran/macos/roles/command_line_tools/tasks/bootstrap.yml:14
skipping: [ansible-core-ci-template-macos-13.0-experiment-7] => {
    "changed": false,
    "skip_reason": "Conditional result was False"
}

[...]

TASK [samdoran.macos.command_line_tools : Test default Python] ***********************************************************************************************
task path: ~/.ansible/collections/ansible_collections/samdoran/macos/roles/command_line_tools/tasks/bootstrap.yml:47
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J administrator@bastion.hostname.com:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'echo ~administrator && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'/Users/administrator\n', b'')
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J administrator@bastion.hostname.com:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /Users/administrator/.ansible/tmp `"&& mkdir "` echo /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972 `" && echo ansible-tmp-1667402978.8919308-1397010-168807032675972="` echo /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972 `" ) && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'ansible-tmp-1667402978.8919308-1397010-168807032675972=/Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972\n', b'')
Using module file ~/.pyenv/versions/venv-py3.8.6/lib/python3.8/site-packages/ansible/modules/setup.py
<ansible-core-ci-template-macos-13.0-experiment-7> PUT ~/.ansible/tmp/ansible-local-13960056h9ow_yc/tmpdp3hnodm TO /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 scp -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J administrator@bastion.hostname.com:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ~/.ansible/tmp/ansible-local-13960056h9ow_yc/tmpdp3hnodm '[ansible-core-ci-template-macos-13.0-experiment-7]:/Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py'
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'', b'')
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J administrator@bastion.hostname.com:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'chmod u+x /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/ /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'', b'')
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J administrator@bastion.hostname.com:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e -tt ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'sudo -H -S  -p "[sudo via ansible, key=wvcyutmdohfracealzyzdgggsucdqdlh] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-wvcyutmdohfracealzyzdgggsucdqdlh ; /usr/bin/python3 /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<ansible-core-ci-template-macos-13.0-experiment-7> (1, b"\r\nxcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.\r\n", b'Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n')
<ansible-core-ci-template-macos-13.0-experiment-7> Failed to connect to the host via ssh: Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J administrator@bastion.hostname.com:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'rm -f -r /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/ > /dev/null 2>&1 && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'', b'')
fatal: [ansible-core-ci-template-macos-13.0-experiment-7]: FAILED! => {
    "ansible_facts": {},
    "changed": false,
    "failed_modules": {
        "ansible.legacy.setup": {
            "failed": true,
            "module_stderr": "Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n",
            "module_stdout": "\r\nxcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.\r\n",
            "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
            "rc": 1
        }
    },
    "msg": "The following modules failed to execute: ansible.legacy.setup\n"
}

PLAY RECAP ***************************************************************************************************************************************************
ansible-core-ci-template-macos-13.0-experiment-7 : ok=1    changed=0    unreachable=0    failed=1    skipped=4    rescued=0    ignored=0
webknjaz commented 1 year ago

@samdoran looks like your implementation diverged from my initial suggestion at https://github.com/samdoran/ansible-collection-macos/pull/3#issuecomment-1176695266 and this specific check might've never worked. In my implementation, I was relying on the return code, and you've gone for checking the stderr. But when using the raw module, it seems like everything returned by the command being executed ends up in stdout, not stderr. And stderr contains whatever the underlying SSH client logs. I'll let @mattclay correct me, but it appears like this check never worked as intended in the first place.

mattclay commented 1 year ago

Relying on the output of raw to be in only one of stderr or stdout is fragile, as it can change depending on the connection plugin and options used. I'd either use the return code or combine the output from stderr and stdout and check that instead.

samdoran commented 1 year ago

Relying on the output of raw to be in only one of stderr or stdout is fragile, as it can change depending on the connection plugin and options used

Oh, that's a good point. I just made a minor change the bootstrap playbook and was confused about the stderr vs stdout behavior.

samdoran commented 1 year ago

I forget why exactly I was checking for specific strings. I think checking the return code is probably the best solution.

webknjaz commented 1 year ago

@samdoran will you be releasing these recent changes soon? I don't know when #8 will be ready, so no need to wait for it.

samdoran commented 1 year ago

I can make a release whenever. I wasn't sure if we wanted to wait for other changes.

webknjaz commented 1 year ago

I suppose making another release when that PR is in is also an option. I'm working on multiple things which is why the timing is unpredictable. But it seems like a good idea to release bug fixes right away if the process isn't too involved. If it's a burden, I can build an automation that would allow you to have a single-click workflow for making new releases, if you want.

samdoran commented 1 year ago

Version 2.2.3 is published.