search

nodejs / build

Better build and test infra for Node.
502 stars 164 forks source link

/usr/local/bin/tap2junit: /usr/local/opt/python/bin/python3.7: bad interpreter: No such file or directory #2512

Closed richardlau closed 3 years ago

richardlau commented 3 years ago

It looks like the Orka 10.14 macOS hosts can't run tap2junit: e.g. https://ci.nodejs.org/job/node-test-commit-osx/38260/nodes=osx1014/console

08:19:36 + tap2junit -i test.tap -o test.xml
08:19:36 /var/folders/kn/6gbh6l053wl4qw5s1b1nrm5r0000gp/T/jenkins3572654198302378329.sh: /usr/local/bin/tap2junit: /usr/local/opt/python/bin/python3.7: bad interpreter: No such file or directory
08:19:36

If I log into any of the three Orka macOS 10.14 hosts it looks like the tap2junit script is referencing a non-existent Python in its #! line:

test-orka-macos10:~ administrator$ cat /usr/local/bin/tap2junit
#!/usr/local/opt/python/bin/python3.7
# -*- coding: utf-8 -*-
import re
import sys
from tap2junit.__main__ import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
test-orka-macos10:~ administrator$ ls -l /usr/local/opt/python/bin/
total 32
lrwxr-xr-x  1 administrator  staff    52 Oct  5 08:29 2to3 -> ../Frameworks/Python.framework/Versions/3.9/bin/2to3
lrwxr-xr-x  1 administrator  staff    56 Oct  5 08:29 2to3-3.9 -> ../Frameworks/Python.framework/Versions/3.9/bin/2to3-3.9
-rwxr-xr-x  1 administrator  staff  1011 Dec 11 08:15 easy_install-3.9
lrwxr-xr-x  1 administrator  staff    53 Oct  5 08:29 idle3 -> ../Frameworks/Python.framework/Versions/3.9/bin/idle3
lrwxr-xr-x  1 administrator  staff    55 Oct  5 08:29 idle3.9 -> ../Frameworks/Python.framework/Versions/3.9/bin/idle3.9
-rwxr-xr-x  1 administrator  staff   966 Dec 11 08:15 pip3
-rwxr-xr-x  1 administrator  staff   970 Dec 11 08:15 pip3.9
lrwxr-xr-x  1 administrator  staff    54 Oct  5 08:29 pydoc3 -> ../Frameworks/Python.framework/Versions/3.9/bin/pydoc3
lrwxr-xr-x  1 administrator  staff    56 Oct  5 08:29 pydoc3.9 -> ../Frameworks/Python.framework/Versions/3.9/bin/pydoc3.9
lrwxr-xr-x  1 administrator  staff    55 Oct  5 08:29 python3 -> ../Frameworks/Python.framework/Versions/3.9/bin/python3
lrwxr-xr-x  1 administrator  staff    62 Oct  5 08:29 python3-config -> ../Frameworks/Python.framework/Versions/3.9/bin/python3-config
lrwxr-xr-x  1 administrator  staff    57 Oct  5 08:29 python3.9 -> ../Frameworks/Python.framework/Versions/3.9/bin/python3.9
lrwxr-xr-x  1 administrator  staff    64 Oct  5 08:29 python3.9-config -> ../Frameworks/Python.framework/Versions/3.9/bin/python3.9-config
-rwxr-xr-x  1 administrator  staff   974 Dec 11 08:15 wheel3
test-orka-macos10:~ administrator$
richardlau commented 3 years ago

FWIW for comparison on a working Orka 10.15 host, test-orka-macos10.15-x64-1:

administrator@test-orka-macos10 ~ % cat /usr/local/bin/tap2junit
#!/Library/Developer/CommandLineTools/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys

from tap2junit.__main__ import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
administrator@test-orka-macos10 ~ % ls -al /Library/Developer/CommandLineTools/usr/bin/python*
lrwxr-xr-x  1 root  wheel  67 Dec 14 07:52 /Library/Developer/CommandLineTools/usr/bin/python3 -> ../../Library/Frameworks/Python3.framework/Versions/3.7/bin/python3
lrwxr-xr-x  1 root  wheel  74 Dec 14 07:52 /Library/Developer/CommandLineTools/usr/bin/python3-config -> ../../Library/Frameworks/Python3.framework/Versions/3.7/bin/python3-config
lrwxr-xr-x  1 root  wheel  69 Dec 14 07:52 /Library/Developer/CommandLineTools/usr/bin/python3.7 -> ../../Library/Frameworks/Python3.framework/Versions/3.7/bin/python3.7
lrwxr-xr-x  1 root  wheel  76 Dec 14 07:52 /Library/Developer/CommandLineTools/usr/bin/python3.7-config -> ../../Library/Frameworks/Python3.framework/Versions/3.7/bin/python3.7-config
lrwxr-xr-x  1 root  wheel  70 Dec 14 07:52 /Library/Developer/CommandLineTools/usr/bin/python3.7m -> ../../Library/Frameworks/Python3.framework/Versions/3.7/bin/python3.7m
lrwxr-xr-x  1 root  wheel  77 Dec 14 07:52 /Library/Developer/CommandLineTools/usr/bin/python3.7m-config -> ../../Library/Frameworks/Python3.framework/Versions/3.7/bin/python3.7m-config
administrator@test-orka-macos10 ~ %

I think the #! line (and the tap2junit binary itself) is coming from the package installer (pip) rather than the tap2junit code (https://github.com/nodejs/tap2junit) itself.

AshCripps commented 3 years ago

So I assume this just requires a reinstall of tap2junit?

richardlau commented 3 years ago

So I assume this just requires a reinstall of tap2junit?

🤞.

Just tried to run the ansible playbook against one of the hosts but got an error complaining about Python.

$ ansible-playbook -l test-orka-macos10.14-x64-1 playbooks/jenkins/worker/create.yml
[DEPRECATION WARNING]: DEFAULT_SQUASH_ACTIONS option, Loop squashing is deprecated and this configuration will no longer be used , use a list directly with
 the module argument instead. This feature will be removed in version 2.11. Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
[DEPRECATION WARNING]: The use of 'static' has been deprecated. Use 'import_tasks' for static inclusion, or 'include_tasks' for dynamic inclusion. This
feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

PLAY [test,release,infra-softlayer-ubuntu1404-x64-2,!*-win*] ***********************************************************************************************

TASK [check if secret is properly set] *********************************************************************************************************************
ok: [test-orka-macos10.14-x64-1]

TASK [run os-specific bootstrap] ***************************************************************************************************************************

TASK [run raspberry pi bootstrap] **************************************************************************************************************************
skipping: [test-orka-macos10.14-x64-1]

TASK [package-upgrade : include package manager tasks] *****************************************************************************************************
included: /home/rlau/sandbox/github/build/ansible/roles/package-upgrade/tasks/partials/brew.yml for test-orka-macos10.14-x64-1

TASK [package-upgrade : Check for xcode-tools] *************************************************************************************************************
changed: [test-orka-macos10.14-x64-1]

TASK [package-upgrade : Install xcode-tools] ***************************************************************************************************************
skipping: [test-orka-macos10.14-x64-1]

TASK [package-upgrade : Check if Homebrew is already installed] ********************************************************************************************
fatal: [test-orka-macos10.14-x64-1]: FAILED! => {"changed": false, "module_stderr": "Shared connection to 199.7.167.99 closed.\r\n", "module_stdout": "/bin/sh: /usr/bin/python3: No such file or directory\r\n", "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error", "rc": 127}

PLAY RECAP *************************************************************************************************************************************************
test-orka-macos10.14-x64-1 : ok=3    changed=1    unreachable=0    failed=1    skipped=3    rescued=0    ignored=0

$
AshCripps commented 3 years ago

https://github.com/nodejs/build/blob/48a9ac25198a26af4546b93dfbb42b6feb988e39/ansible/ansible.cfg#L57-L59

is set to usr/bin/python3 but the 10.14 macs seem to be:

test-orka-macos10:~ administrator$ which python3
/usr/local/bin/python3
richardlau commented 3 years ago

Temporarily commented out the entry for ansible_python_interpreter in ansible.cfg for macOS and set it in the inventory.xml for the host I'm running against (test-orka-macos10.14-x64-1) and the playbook gets a bit further and then errors on:

TASK [package-upgrade : Upgrade installed packages] ********************************************************************************************************
task path: /home/rlau/sandbox/github/build/ansible/roles/package-upgrade/tasks/partials/brew.yml:26
fatal: [test-orka-macos10.14-x64-1]: FAILED! => {"changed": false, "msg": "Updating Homebrew...\nError: Could not remove python keg! Do so manually:\n  sudo rm -rf /usr/local/Cellar/python/3.7.7"}
richardlau commented 3 years ago

Rerunning got past that failure (in "Upgrade installed packages") and I'm now stuck at

TASK [package-upgrade : Update Casks] **********************************************************************************************************************
task path: /home/rlau/sandbox/github/build/ansible/roles/package-upgrade/tasks/partials/brew.yml:41
fatal: [test-orka-macos10.14-x64-1]: FAILED! => {"changed": false, "msg": "Error: Calling brew cask upgrade is disabled! Use brew upgrade [--cask] instead."}

This looks to have been a recent breakage caused by Homebrew. Refs: https://github.com/ansible-collections/community.general/issues/1524 Refs: https://github.com/Homebrew/discussions/discussions/340#discussioncomment-232364

richardlau commented 3 years ago

Apparently the latest ansible 2.10 contains a fix, but for other reasons the most recent versions of ansible in Fedora 33 (where I've been running the script) is 2.9.16. I'll try to set up a container and install ansible via pip.