Closed alexreg closed 2 years ago
Hi @alexreg, thanks for reporting that.
Unfortunately this error seems to be macOS specific, so I will not be able to help you debug it directly (since I don't have access to a macOS environment).
Something that might give us a hint of what is happening is the line foopackage 0.0.0 is already the active version in easy-install.pth
. Have you tried to run this in an isolated virtual environment?
It also says Installing foo script to /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin
, Have you checked if the foo
script exist in that folder? If that is the case the problem is likely to be related to a misalignment between the homebrew version of distutils/sysconfig and the expectations in setuptools.
If that is not the case what is the folder you expect the script to be installed to?
Another question (just to verify if the problem is fixed in newer versions of setuptools): does the problem still persists if you: update setuptools to the latest version?
Hi @abravalheri. Oddly enough, I'm getting setuptools reporting version 62.1.0
now. Perhaps it upgraded in the background in the meanwhile.
Indeed, the foo
script exists in /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin
. I suppose the problem is that this is the wrong place?
Perhaps the problem is that this is the wrong location? Or something goes awry with the interaction with pip
? The way I first encountered this was that pip install
works but pip install -e
does not.
🤔 that is going to be tricky to track down :P, I will see if I can have a look on this weekend.
While pip install .
does its own thing, pip install -e.
delegates to python setup.py develop
, which is also different from python setup.py install
. So there are 3 installation methods.
The fact that python setup.py install
works is promising... Now what we need to do is scrutinizing the code to see how they come up with 2 different locations for the installation directory.
I just hope that the fact that python setup.py install
is not working by accident, because Homebrew does seem to customize Python's directories in https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/python@3.9.rb#L375.
@alexreg could you show us what are the values you are getting in your machine for the following?
sys.prefix
sys.exec_prefix
sys.base_prefix
sys.base_exec_prefix
The output of `python -m sysconfig` (before variables) (or the values of `sysconfig.get_paths()`)
distutils.sysconfig.get_config_vars()
@abravalheri Thanks for looking into this.
Actually, I should clarify: it seems like python setup.py install
and python setup.py develop
both work in the sense they write the console script to /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin
. However, pip install
writes it to /opt/homebrew/bin
, which is really where it should go, whereas (I just checked), pip install -e
writes it to /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin
.
Here's the output you asked for:
Thank you very much @alexreg for the information.
Actually, I should clarify: it seems like
python setup.py install
andpython setup.py develop
both work in the sense they write the console script to/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin
.
Ok, thanks for checking that.
This makes things more complicated, since it is not a simple matter of aligning install
and develop
. Based on the extra info that you provided, it would seem that Homebrew actually wants setuptools
to install files into /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin
(as indicated by the output of python3 -m sysconfig
under Paths > scripts
). This is also compatible with the values of sys.*prefix
.
It might be the case there are multiple sources of conflicting information. So we have to find out what pip
is doing here to understand the problem.
The error with distutils.sysconfig
is probably because it needs an extra import distutils.sysconfig
.
Ah yes, of course. Here's the output of that too.
>>> print(distutils.sysconfig.get_config_vars())
{'prefix': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', 'exec_prefix': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', 'py_version': '3.9.12', 'py_version_short': '3.9', 'py_version_nodot': '39', 'installed_base': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', 'base': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', 'installed_platbase': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', 'platbase': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', 'projectbase': '/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/bin', 'platlibdir': 'lib', 'abiflags': '', 'ABIFLAGS': '', 'AC_APPLE_UNIVERSAL_BUILD': 0, 'AIX_BUILDDATE': 0, 'AIX_GENUINE_CPLUSPLUS': 0, 'ALT_SOABI': 0, 'ANDROID_API_LEVEL': 0, 'AR': '/usr/bin/xcrun ar', 'ARFLAGS': 'rcs', 'BASECFLAGS': '-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic', 'BASECPPFLAGS': '', 'BASEMODLIBS': '', 'BINDIR': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin', 'BINLIBDEST': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9', 'BLDLIBRARY': '', 'BLDSHARED': 'clang -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'BUILDEXE': '.exe', 'BUILDPYTHON': 'python.exe', 'BUILD_GNU_TYPE': 'aarch64-apple-darwin21.3.0', 'BYTESTR_DEPS': '\\', 'CC': 'clang', 'CCSHARED': '', 'CFLAGS': '-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'CFLAGSFORSHARED': '', 'CFLAGS_ALIASING': '-fno-strict-aliasing', 'CONFIGFILES': 'configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in', 'CONFIGURE_CFLAGS': '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'CONFIGURE_CFLAGS_NODIST': '-I/opt/homebrew/include -flto -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden', 'CONFIGURE_CPPFLAGS': '-I/opt/homebrew/include', 'CONFIGURE_LDFLAGS': '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'CONFIGURE_LDFLAGS_NODIST': '-L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -flto -Wl,-export_dynamic -g', 'CONFIG_ARGS': "'--prefix=/opt/homebrew/opt/python@3.9' '--enable-ipv6' '--datarootdir=/opt/homebrew/opt/python@3.9/share' '--datadir=/opt/homebrew/opt/python@3.9/share' '--without-ensurepip' '--enable-loadable-sqlite-extensions' '--with-openssl=/opt/homebrew/opt/openssl@1.1' '--with-dbmliborder=gdbm:ndbm' '--enable-optimizations' '--with-lto' '--with-system-expat' '--with-system-ffi' '--with-system-libmpdec' '--enable-framework=/opt/homebrew/opt/python@3.9/Frameworks' '--with-dtrace' 'MACOSX_DEPLOYMENT_TARGET=12' 'CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk' 'CFLAGS_NODIST=-I/opt/homebrew/include' 'LDFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk' 'LDFLAGS_NODIST=-L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib' 'CPPFLAGS=-I/opt/homebrew/include' 'CC=clang' 'PKG_CONFIG_PATH=/opt/homebrew/opt/openssl@1.1/lib/pkgconfig:/opt/homebrew/opt/readline/lib/pkgconfig:/opt/homebrew/opt/sqlite/lib/pkgconfig:/opt/homebrew/opt/xz/lib/pkgconfig' 'PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/opt/homebrew/Library/Homebrew/os/mac/pkgconfig/12'", 'CONFINCLUDEDIR': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include', 'CONFINCLUDEPY': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9', 'COREPYTHONPATH': '', 'COVERAGE_INFO': '/private/tmp/pythonA3.9-20220326-49966-1n3ovp7/Python-3.9.12/coverage.info', 'COVERAGE_REPORT': '/private/tmp/pythonA3.9-20220326-49966-1n3ovp7/Python-3.9.12/lcov-report', 'COVERAGE_REPORT_OPTIONS': '--no-branch-coverage --title "CPython lcov report"', 'CPPFLAGS': '-I. -I./Include -I/opt/homebrew/include', 'CXX': 'clang++', 'DESTDIRS': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9 /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9 /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', 'DESTLIB': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9', 'DESTPATH': '', 'DESTSHARED': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', 'DFLAGS': '', 'DIRMODE': 755, 'DIST': 'README.rst ChangeLog configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in Include Lib Misc Ext-dummy', 'DISTDIRS': 'Include Lib Misc Ext-dummy', 'DISTFILES': 'README.rst ChangeLog configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in', 'DLINCLDIR': '.', 'DLLLIBRARY': '', 'DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754': 0, 'DOUBLE_IS_BIG_ENDIAN_IEEE754': 0, 'DOUBLE_IS_LITTLE_ENDIAN_IEEE754': 1, 'DTRACE': '/usr/sbin/dtrace', 'DTRACE_DEPS': '\\', 'DTRACE_HEADERS': 'Include/pydtrace_probes.h', 'DTRACE_OBJS': '', 'DYNLOADFILE': 'dynload_shlib.o', 'ENABLE_IPV6': 1, 'ENSUREPIP': 'no', 'EXE': '', 'EXEMODE': 755, 'EXPORTSFROM': '', 'EXPORTSYMS': '', 'EXTRATESTOPTS': '', 'EXTRA_CFLAGS': '', 'EXT_SUFFIX': '.cpython-39-darwin.so', 'FILEMODE': 644, 'FLOAT_WORDS_BIGENDIAN': 0, 'FLOCK_NEEDS_LIBBSD': 0, 'GETPGRP_HAVE_ARG': 0, 'GITBRANCH': '', 'GITTAG': '', 'GITVERSION': '', 'GNULD': 'no', 'HAVE_ACCEPT4': 0, 'HAVE_ACOSH': 1, 'HAVE_ADDRINFO': 1, 'HAVE_ALARM': 1, 'HAVE_ALIGNED_REQUIRED': 0, 'HAVE_ALLOCA_H': 1, 'HAVE_ALTZONE': 0, 'HAVE_ASINH': 1, 'HAVE_ASM_TYPES_H': 0, 'HAVE_ATANH': 1, 'HAVE_BIND_TEXTDOMAIN_CODESET': 0, 'HAVE_BLUETOOTH_BLUETOOTH_H': 0, 'HAVE_BLUETOOTH_H': 0, 'HAVE_BROKEN_MBSTOWCS': 0, 'HAVE_BROKEN_NICE': 0, 'HAVE_BROKEN_PIPE_BUF': 0, 'HAVE_BROKEN_POLL': 0, 'HAVE_BROKEN_POSIX_SEMAPHORES': 0, 'HAVE_BROKEN_PTHREAD_SIGMASK': 0, 'HAVE_BROKEN_SEM_GETVALUE': 1, 'HAVE_BROKEN_UNSETENV': 0, 'HAVE_BUILTIN_ATOMIC': 1, 'HAVE_CHFLAGS': 1, 'HAVE_CHOWN': 1, 'HAVE_CHROOT': 1, 'HAVE_CLOCK': 1, 'HAVE_CLOCK_GETRES': 1, 'HAVE_CLOCK_GETTIME': 1, 'HAVE_CLOCK_SETTIME': 1, 'HAVE_COMPUTED_GOTOS': 1, 'HAVE_CONFSTR': 1, 'HAVE_CONIO_H': 0, 'HAVE_COPYSIGN': 1, 'HAVE_COPY_FILE_RANGE': 0, 'HAVE_CRYPT_H': 0, 'HAVE_CRYPT_R': 0, 'HAVE_CTERMID': 1, 'HAVE_CTERMID_R': 1, 'HAVE_CURSES_FILTER': 1, 'HAVE_CURSES_H': 1, 'HAVE_CURSES_HAS_KEY': 1, 'HAVE_CURSES_IMMEDOK': 1, 'HAVE_CURSES_IS_PAD': 0, 'HAVE_CURSES_IS_TERM_RESIZED': 1, 'HAVE_CURSES_RESIZETERM': 1, 'HAVE_CURSES_RESIZE_TERM': 1, 'HAVE_CURSES_SYNCOK': 1, 'HAVE_CURSES_TYPEAHEAD': 1, 'HAVE_CURSES_USE_ENV': 1, 'HAVE_CURSES_WCHGAT': 1, 'HAVE_DECL_ISFINITE': 1, 'HAVE_DECL_ISINF': 1, 'HAVE_DECL_ISNAN': 1, 'HAVE_DECL_RTLD_DEEPBIND': 0, 'HAVE_DECL_RTLD_GLOBAL': 1, 'HAVE_DECL_RTLD_LAZY': 1, 'HAVE_DECL_RTLD_LOCAL': 1, 'HAVE_DECL_RTLD_MEMBER': 0, 'HAVE_DECL_RTLD_NODELETE': 1, 'HAVE_DECL_RTLD_NOLOAD': 1, 'HAVE_DECL_RTLD_NOW': 1, 'HAVE_DECL_TZNAME': 0, 'HAVE_DEVICE_MACROS': 1, 'HAVE_DEV_PTC': 0, 'HAVE_DEV_PTMX': 1, 'HAVE_DIRECT_H': 0, 'HAVE_DIRENT_D_TYPE': 1, 'HAVE_DIRENT_H': 1, 'HAVE_DIRFD': 1, 'HAVE_DLFCN_H': 1, 'HAVE_DLOPEN': 1, 'HAVE_DUP2': 1, 'HAVE_DUP3': 0, 'HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH': 1, 'HAVE_DYNAMIC_LOADING': 1, 'HAVE_ENDIAN_H': 0, 'HAVE_EPOLL': 0, 'HAVE_EPOLL_CREATE1': 0, 'HAVE_ERF': 1, 'HAVE_ERFC': 1, 'HAVE_ERRNO_H': 1, 'HAVE_EXECV': 1, 'HAVE_EXPLICIT_BZERO': 0, 'HAVE_EXPLICIT_MEMSET': 0, 'HAVE_EXPM1': 1, 'HAVE_FACCESSAT': 1, 'HAVE_FCHDIR': 1, 'HAVE_FCHMOD': 1, 'HAVE_FCHMODAT': 1, 'HAVE_FCHOWN': 1, 'HAVE_FCHOWNAT': 1, 'HAVE_FCNTL_H': 1, 'HAVE_FDATASYNC': 0, 'HAVE_FDOPENDIR': 1, 'HAVE_FDWALK': 0, 'HAVE_FEXECVE': 0, 'HAVE_FINITE': 0, 'HAVE_FLOCK': 1, 'HAVE_FORK': 1, 'HAVE_FORKPTY': 1, 'HAVE_FPATHCONF': 1, 'HAVE_FSEEK64': 0, 'HAVE_FSEEKO': 1, 'HAVE_FSTATAT': 1, 'HAVE_FSTATVFS': 1, 'HAVE_FSYNC': 1, 'HAVE_FTELL64': 0, 'HAVE_FTELLO': 1, 'HAVE_FTIME': 1, 'HAVE_FTRUNCATE': 1, 'HAVE_FUTIMENS': 1, 'HAVE_FUTIMES': 1, 'HAVE_FUTIMESAT': 0, 'HAVE_GAI_STRERROR': 1, 'HAVE_GAMMA': 0, 'HAVE_GCC_ASM_FOR_MC68881': 0, 'HAVE_GCC_ASM_FOR_X64': 0, 'HAVE_GCC_ASM_FOR_X87': 0, 'HAVE_GCC_UINT128_T': 1, 'HAVE_GETADDRINFO': 1, 'HAVE_GETC_UNLOCKED': 1, 'HAVE_GETENTROPY': 1, 'HAVE_GETGRGID_R': 1, 'HAVE_GETGRNAM_R': 1, 'HAVE_GETGROUPLIST': 1, 'HAVE_GETGROUPS': 1, 'HAVE_GETHOSTBYNAME': 1, 'HAVE_GETHOSTBYNAME_R': 0, 'HAVE_GETHOSTBYNAME_R_3_ARG': 0, 'HAVE_GETHOSTBYNAME_R_5_ARG': 0, 'HAVE_GETHOSTBYNAME_R_6_ARG': 0, 'HAVE_GETITIMER': 1, 'HAVE_GETLOADAVG': 1, 'HAVE_GETLOGIN': 1, 'HAVE_GETNAMEINFO': 1, 'HAVE_GETPAGESIZE': 1, 'HAVE_GETPEERNAME': 1, 'HAVE_GETPGID': 1, 'HAVE_GETPGRP': 1, 'HAVE_GETPID': 1, 'HAVE_GETPRIORITY': 1, 'HAVE_GETPWENT': 1, 'HAVE_GETPWNAM_R': 1, 'HAVE_GETPWUID_R': 1, 'HAVE_GETRANDOM': 0, 'HAVE_GETRANDOM_SYSCALL': 0, 'HAVE_GETRESGID': 0, 'HAVE_GETRESUID': 0, 'HAVE_GETSID': 1, 'HAVE_GETSPENT': 0, 'HAVE_GETSPNAM': 0, 'HAVE_GETWD': 1, 'HAVE_GLIBC_MEMMOVE_BUG': 0, 'HAVE_GRP_H': 1, 'HAVE_HSTRERROR': 1, 'HAVE_HTOLE64': 0, 'HAVE_HYPOT': 1, 'HAVE_IEEEFP_H': 0, 'HAVE_IF_NAMEINDEX': 1, 'HAVE_INET_ATON': 1, 'HAVE_INET_PTON': 1, 'HAVE_INITGROUPS': 1, 'HAVE_INTTYPES_H': 1, 'HAVE_IO_H': 0, 'HAVE_IPA_PURE_CONST_BUG': 0, 'HAVE_KILL': 1, 'HAVE_KILLPG': 1, 'HAVE_KQUEUE': 1, 'HAVE_LANGINFO_H': 1, 'HAVE_LARGEFILE_SUPPORT': 0, 'HAVE_LCHFLAGS': 1, 'HAVE_LCHMOD': 1, 'HAVE_LCHOWN': 1, 'HAVE_LGAMMA': 1, 'HAVE_LIBDL': 1, 'HAVE_LIBDLD': 0, 'HAVE_LIBIEEE': 0, 'HAVE_LIBINTL_H': 0, 'HAVE_LIBREADLINE': 1, 'HAVE_LIBRESOLV': 0, 'HAVE_LIBSENDFILE': 0, 'HAVE_LIBUTIL_H': 0, 'HAVE_LIBUUID': 0, 'HAVE_LINK': 1, 'HAVE_LINKAT': 1, 'HAVE_LINUX_AUXVEC_H': 0, 'HAVE_LINUX_CAN_BCM_H': 0, 'HAVE_LINUX_CAN_H': 0, 'HAVE_LINUX_CAN_J1939_H': 0, 'HAVE_LINUX_CAN_RAW_FD_FRAMES': 0, 'HAVE_LINUX_CAN_RAW_H': 0, 'HAVE_LINUX_CAN_RAW_JOIN_FILTERS': 0, 'HAVE_LINUX_MEMFD_H': 0, 'HAVE_LINUX_NETLINK_H': 0, 'HAVE_LINUX_QRTR_H': 0, 'HAVE_LINUX_RANDOM_H': 0, 'HAVE_LINUX_TIPC_H': 0, 'HAVE_LINUX_VM_SOCKETS_H': 0, 'HAVE_LINUX_WAIT_H': 0, 'HAVE_LOCKF': 1, 'HAVE_LOG1P': 1, 'HAVE_LOG2': 1, 'HAVE_LONG_DOUBLE': 1, 'HAVE_LSTAT': 1, 'HAVE_LUTIMES': 1, 'HAVE_MADVISE': 1, 'HAVE_MAKEDEV': 1, 'HAVE_MBRTOWC': 1, 'HAVE_MEMFD_CREATE': 0, 'HAVE_MEMORY_H': 1, 'HAVE_MEMRCHR': 0, 'HAVE_MKDIRAT': 1, 'HAVE_MKFIFO': 1, 'HAVE_MKFIFOAT': 0, 'HAVE_MKNOD': 1, 'HAVE_MKNODAT': 0, 'HAVE_MKTIME': 1, 'HAVE_MMAP': 1, 'HAVE_MREMAP': 0, 'HAVE_NCURSES_H': 1, 'HAVE_NDIR_H': 0, 'HAVE_NETPACKET_PACKET_H': 0, 'HAVE_NET_IF_H': 1, 'HAVE_NICE': 1, 'HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION': 0, 'HAVE_OPENAT': 1, 'HAVE_OPENPTY': 1, 'HAVE_PATHCONF': 1, 'HAVE_PAUSE': 1, 'HAVE_PIPE2': 0, 'HAVE_PLOCK': 0, 'HAVE_POLL': 1, 'HAVE_POLL_H': 1, 'HAVE_POSIX_FADVISE': 0, 'HAVE_POSIX_FALLOCATE': 0, 'HAVE_POSIX_SPAWN': 1, 'HAVE_POSIX_SPAWNP': 1, 'HAVE_PREAD': 1, 'HAVE_PREADV': 1, 'HAVE_PREADV2': 0, 'HAVE_PRLIMIT': 0, 'HAVE_PROCESS_H': 0, 'HAVE_PROTOTYPES': 1, 'HAVE_PTHREAD_CONDATTR_SETCLOCK': 0, 'HAVE_PTHREAD_DESTRUCTOR': 0, 'HAVE_PTHREAD_GETCPUCLOCKID': 0, 'HAVE_PTHREAD_H': 1, 'HAVE_PTHREAD_INIT': 0, 'HAVE_PTHREAD_KILL': 1, 'HAVE_PTHREAD_SIGMASK': 1, 'HAVE_PTY_H': 0, 'HAVE_PWRITE': 1, 'HAVE_PWRITEV': 1, 'HAVE_PWRITEV2': 0, 'HAVE_READLINK': 1, 'HAVE_READLINKAT': 1, 'HAVE_READV': 1, 'HAVE_REALPATH': 1, 'HAVE_RENAMEAT': 1, 'HAVE_RL_APPEND_HISTORY': 1, 'HAVE_RL_CATCH_SIGNAL': 1, 'HAVE_RL_COMPLETION_APPEND_CHARACTER': 1, 'HAVE_RL_COMPLETION_DISPLAY_MATCHES_HOOK': 1, 'HAVE_RL_COMPLETION_MATCHES': 1, 'HAVE_RL_COMPLETION_SUPPRESS_APPEND': 1, 'HAVE_RL_PRE_INPUT_HOOK': 1, 'HAVE_RL_RESIZE_TERMINAL': 1, 'HAVE_ROUND': 1, 'HAVE_RTPSPAWN': 0, 'HAVE_SCHED_GET_PRIORITY_MAX': 1, 'HAVE_SCHED_H': 1, 'HAVE_SCHED_RR_GET_INTERVAL': 0, 'HAVE_SCHED_SETAFFINITY': 0, 'HAVE_SCHED_SETPARAM': 0, 'HAVE_SCHED_SETSCHEDULER': 0, 'HAVE_SEM_CLOCKWAIT': 0, 'HAVE_SEM_GETVALUE': 1, 'HAVE_SEM_OPEN': 1, 'HAVE_SEM_TIMEDWAIT': 0, 'HAVE_SEM_UNLINK': 1, 'HAVE_SENDFILE': 1, 'HAVE_SETEGID': 1, 'HAVE_SETEUID': 1, 'HAVE_SETGID': 1, 'HAVE_SETGROUPS': 1, 'HAVE_SETHOSTNAME': 1, 'HAVE_SETITIMER': 1, 'HAVE_SETLOCALE': 1, 'HAVE_SETPGID': 1, 'HAVE_SETPGRP': 1, 'HAVE_SETPRIORITY': 1, 'HAVE_SETREGID': 1, 'HAVE_SETRESGID': 0, 'HAVE_SETRESUID': 0, 'HAVE_SETREUID': 1, 'HAVE_SETSID': 1, 'HAVE_SETUID': 1, 'HAVE_SETVBUF': 1, 'HAVE_SHADOW_H': 0, 'HAVE_SHM_OPEN': 1, 'HAVE_SHM_UNLINK': 1, 'HAVE_SIGACTION': 1, 'HAVE_SIGALTSTACK': 1, 'HAVE_SIGFILLSET': 1, 'HAVE_SIGINFO_T_SI_BAND': 1, 'HAVE_SIGINTERRUPT': 1, 'HAVE_SIGNAL_H': 1, 'HAVE_SIGPENDING': 1, 'HAVE_SIGRELSE': 1, 'HAVE_SIGTIMEDWAIT': 0, 'HAVE_SIGWAIT': 1, 'HAVE_SIGWAITINFO': 0, 'HAVE_SNPRINTF': 1, 'HAVE_SOCKADDR_ALG': 0, 'HAVE_SOCKADDR_SA_LEN': 1, 'HAVE_SOCKADDR_STORAGE': 1, 'HAVE_SOCKETPAIR': 1, 'HAVE_SPAWN_H': 1, 'HAVE_SSIZE_T': 1, 'HAVE_STATVFS': 1, 'HAVE_STAT_TV_NSEC': 0, 'HAVE_STAT_TV_NSEC2': 1, 'HAVE_STDARG_PROTOTYPES': 1, 'HAVE_STDINT_H': 1, 'HAVE_STDLIB_H': 1, 'HAVE_STD_ATOMIC': 1, 'HAVE_STRDUP': 1, 'HAVE_STRFTIME': 1, 'HAVE_STRINGS_H': 1, 'HAVE_STRING_H': 1, 'HAVE_STRLCPY': 1, 'HAVE_STROPTS_H': 0, 'HAVE_STRSIGNAL': 1, 'HAVE_STRUCT_PASSWD_PW_GECOS': 1, 'HAVE_STRUCT_PASSWD_PW_PASSWD': 1, 'HAVE_STRUCT_STAT_ST_BIRTHTIME': 1, 'HAVE_STRUCT_STAT_ST_BLKSIZE': 1, 'HAVE_STRUCT_STAT_ST_BLOCKS': 1, 'HAVE_STRUCT_STAT_ST_FLAGS': 1, 'HAVE_STRUCT_STAT_ST_GEN': 1, 'HAVE_STRUCT_STAT_ST_RDEV': 1, 'HAVE_STRUCT_TM_TM_ZONE': 1, 'HAVE_SYMLINK': 1, 'HAVE_SYMLINKAT': 1, 'HAVE_SYNC': 1, 'HAVE_SYSCONF': 1, 'HAVE_SYSEXITS_H': 1, 'HAVE_SYS_AUDIOIO_H': 0, 'HAVE_SYS_AUXV_H': 0, 'HAVE_SYS_BSDTTY_H': 0, 'HAVE_SYS_DEVPOLL_H': 0, 'HAVE_SYS_DIR_H': 0, 'HAVE_SYS_ENDIAN_H': 0, 'HAVE_SYS_EPOLL_H': 0, 'HAVE_SYS_EVENT_H': 1, 'HAVE_SYS_FILE_H': 1, 'HAVE_SYS_IOCTL_H': 1, 'HAVE_SYS_KERN_CONTROL_H': 1, 'HAVE_SYS_LOADAVG_H': 0, 'HAVE_SYS_LOCK_H': 1, 'HAVE_SYS_MEMFD_H': 0, 'HAVE_SYS_MKDEV_H': 0, 'HAVE_SYS_MMAN_H': 1, 'HAVE_SYS_MODEM_H': 0, 'HAVE_SYS_NDIR_H': 0, 'HAVE_SYS_PARAM_H': 1, 'HAVE_SYS_POLL_H': 1, 'HAVE_SYS_RANDOM_H': 1, 'HAVE_SYS_RESOURCE_H': 1, 'HAVE_SYS_SELECT_H': 1, 'HAVE_SYS_SENDFILE_H': 0, 'HAVE_SYS_SOCKET_H': 1, 'HAVE_SYS_STATVFS_H': 1, 'HAVE_SYS_STAT_H': 1, 'HAVE_SYS_SYSCALL_H': 1, 'HAVE_SYS_SYSMACROS_H': 0, 'HAVE_SYS_SYS_DOMAIN_H': 1, 'HAVE_SYS_TERMIO_H': 0, 'HAVE_SYS_TIMES_H': 1, 'HAVE_SYS_TIME_H': 1, 'HAVE_SYS_TYPES_H': 1, 'HAVE_SYS_UIO_H': 1, 'HAVE_SYS_UN_H': 1, 'HAVE_SYS_UTSNAME_H': 1, 'HAVE_SYS_WAIT_H': 1, 'HAVE_SYS_XATTR_H': 1, 'HAVE_TCGETPGRP': 1, 'HAVE_TCSETPGRP': 1, 'HAVE_TEMPNAM': 1, 'HAVE_TERMIOS_H': 1, 'HAVE_TERM_H': 1, 'HAVE_TGAMMA': 1, 'HAVE_TIMEGM': 1, 'HAVE_TIMES': 1, 'HAVE_TMPFILE': 1, 'HAVE_TMPNAM': 1, 'HAVE_TMPNAM_R': 0, 'HAVE_TM_ZONE': 1, 'HAVE_TRUNCATE': 1, 'HAVE_TZNAME': 0, 'HAVE_UCS4_TCL': 0, 'HAVE_UNAME': 1, 'HAVE_UNISTD_H': 1, 'HAVE_UNLINKAT': 1, 'HAVE_USABLE_WCHAR_T': 0, 'HAVE_UTIL_H': 1, 'HAVE_UTIMENSAT': 1, 'HAVE_UTIMES': 1, 'HAVE_UTIME_H': 1, 'HAVE_UUID_CREATE': 0, 'HAVE_UUID_ENC_BE': 0, 'HAVE_UUID_GENERATE_TIME_SAFE': 0, 'HAVE_UUID_H': 0, 'HAVE_UUID_UUID_H': 1, 'HAVE_WAIT3': 1, 'HAVE_WAIT4': 1, 'HAVE_WAITID': 1, 'HAVE_WAITPID': 1, 'HAVE_WCHAR_H': 1, 'HAVE_WCSCOLL': 1, 'HAVE_WCSFTIME': 1, 'HAVE_WCSXFRM': 1, 'HAVE_WMEMCMP': 1, 'HAVE_WORKING_TZSET': 1, 'HAVE_WRITEV': 1, 'HAVE_X509_VERIFY_PARAM_SET1_HOST': 1, 'HAVE_ZLIB_COPY': 1, 'HAVE__GETPTY': 0, 'HOST_GNU_TYPE': 'aarch64-apple-darwin21.3.0', 'INCLDIRSTOMAKE': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9 /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9', 'INCLUDEDIR': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include', 'INCLUDEPY': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9', 'INSTALL': '/usr/bin/install -c', 'INSTALL_DATA': '/usr/bin/install -c -m 644', 'INSTALL_PROGRAM': '/usr/bin/install -c', 'INSTALL_SCRIPT': '/usr/bin/install -c', 'INSTALL_SHARED': '/usr/bin/install -c -m 755', 'INSTSONAME': 'Python.framework/Versions/3.9/Python', 'IO_H': 'Modules/_io/_iomodule.h', 'IO_OBJS': '\\', 'LDCXXSHARED': 'clang++ -bundle -undefined dynamic_lookup', 'LDFLAGS': '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'LDLIBRARY': 'Python.framework/Versions/3.9/Python', 'LDLIBRARYDIR': '', 'LDSHARED': 'clang -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'LDVERSION': '3.9', 'LIBC': '', 'LIBDEST': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9', 'LIBDIR': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib', 'LIBFFI_INCLUDEDIR': '/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/ffi', 'LIBM': '', 'LIBOBJDIR': 'Python/', 'LIBOBJS': '', 'LIBPC': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/pkgconfig', 'LIBPL': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin', 'LIBPYTHON': '', 'LIBRARY': 'libpython3.9.a', 'LIBRARY_OBJS': '\\', 'LIBRARY_OBJS_OMIT_FROZEN': '\\', 'LIBS': '-ldl -framework CoreFoundation', 'LIBSUBDIRS': 'tkinter tkinter/test tkinter/test/test_tkinter \\', 'LINKCC': 'clang', 'LINKFORSHARED': '-Wl,-stack_size,1000000 -framework CoreFoundation /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/Python', 'LIPO_32BIT_FLAGS': '', 'LIPO_INTEL64_FLAGS': '', 'LLVM_PROF_ERR': 'no', 'LLVM_PROF_FILE': 'LLVM_PROFILE_FILE="code-%p.profclangr"', 'LLVM_PROF_MERGER': '/usr/bin/xcrun llvm-profdata merge -output=code.profclangd *.profclangr', 'LN': 'ln', 'LOCALMODLIBS': '', 'MACHDEP': 'darwin', 'MACHDEP_OBJS': '', 'MACHDESTLIB': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9', 'MACOSX_DEPLOYMENT_TARGET': '12', 'MAINCC': 'clang', 'MAJOR_IN_MKDEV': 0, 'MAJOR_IN_SYSMACROS': 0, 'MAKESETUP': './Modules/makesetup', 'MANDIR': '/opt/homebrew/opt/python@3.9/share/man', 'MKDIR_P': './install-sh -c -d', 'MODBUILT_NAMES': 'posix errno pwd _sre _codecs _weakref _functools _operator _collections _abc itertools atexit _signal _stat time _thread _locale _io faulthandler _tracemalloc _peg_parser _symtable xxsubtype', 'MODDISABLED_NAMES': '', 'MODLIBS': '', 'MODOBJS': 'Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/_operator.o Modules/_collectionsmodule.o Modules/_abc.o Modules/itertoolsmodule.o Modules/atexitmodule.o Modules/signalmodule.o Modules/_stat.o Modules/timemodule.o Modules/_threadmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/_peg_parser.o Modules/symtablemodule.o Modules/xxsubtype.o', 'MODULE_OBJS': '\\', 'MULTIARCH': 'darwin', 'MULTIARCH_CPPFLAGS': '-DMULTIARCH=\\"darwin\\"', 'MVWDELCH_IS_EXPRESSION': 1, 'NO_AS_NEEDED': '', 'OBJECT_OBJS': '\\', 'OPENSSL_INCLUDES': '-I/opt/homebrew/opt/openssl@1.1/include', 'OPENSSL_LDFLAGS': '-L/opt/homebrew/opt/openssl@1.1/lib', 'OPENSSL_LIBS': '-lssl -lcrypto', 'OPT': '-DNDEBUG -g -fwrapv -O3 -Wall', 'OTHER_LIBTOOL_OPT': '', 'PACKAGE_BUGREPORT': 0, 'PACKAGE_NAME': 0, 'PACKAGE_STRING': 0, 'PACKAGE_TARNAME': 0, 'PACKAGE_URL': 0, 'PACKAGE_VERSION': 0, 'PARSER_HEADERS': '\\', 'PARSER_OBJS': '\\ \\ Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o', 'PEGEN_HEADERS': '\\', 'PEGEN_OBJS': '\\', 'PGO_PROF_GEN_FLAG': '-fprofile-instr-generate', 'PGO_PROF_USE_FLAG': '-fprofile-instr-use=code.profclangd', 'PLATLIBDIR': 'lib', 'POBJS': '\\', 'POSIX_SEMAPHORES_NOT_ENABLED': 0, 'PROFILE_TASK': '-m test --pgo', 'PTHREAD_KEY_T_IS_COMPATIBLE_WITH_INT': 0, 'PTHREAD_SYSTEM_SCHED_SUPPORTED': 1, 'PURIFY': '', 'PY3LIBRARY': '', 'PYLONG_BITS_IN_DIGIT': 0, 'PYTHON': 'python', 'PYTHONFRAMEWORK': 'Python', 'PYTHONFRAMEWORKDIR': 'Python.framework', 'PYTHONFRAMEWORKINSTALLDIR': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework', 'PYTHONFRAMEWORKPREFIX': '/opt/homebrew/opt/python@3.9/Frameworks', 'PYTHONPATH': '', 'PYTHON_FOR_BUILD': './python.exe -E', 'PYTHON_HEADERS': '\\', 'PYTHON_OBJS': '\\', 'PY_BUILTIN_HASHLIB_HASHES': '"md5,sha1,sha256,sha512,sha3,blake2"', 'PY_BUILTIN_MODULE_CFLAGS': '-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/include -flto -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -fprofile-instr-use=code.profclangd -I./Include/internal -I. -I./Include -I/opt/homebrew/include -DPy_BUILD_CORE_BUILTIN', 'PY_CFLAGS': '-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'PY_CFLAGS_NODIST': '-I/opt/homebrew/include -flto -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -fprofile-instr-use=code.profclangd -I./Include/internal', 'PY_COERCE_C_LOCALE': 1, 'PY_CORE_CFLAGS': '-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/include -flto -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -fprofile-instr-use=code.profclangd -I./Include/internal -I. -I./Include -I/opt/homebrew/include -DPy_BUILD_CORE', 'PY_CORE_LDFLAGS': '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -flto -Wl,-export_dynamic -g', 'PY_CPPFLAGS': '-I. -I./Include -I/opt/homebrew/include', 'PY_FORMAT_SIZE_T': '"z"', 'PY_LDFLAGS': '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk', 'PY_LDFLAGS_NODIST': '-L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -flto -Wl,-export_dynamic -g', 'PY_SSL_DEFAULT_CIPHERS': 1, 'PY_SSL_DEFAULT_CIPHER_STRING': 0, 'PY_STDMODULE_CFLAGS': '-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/include -flto -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -fprofile-instr-use=code.profclangd -I./Include/internal -I. -I./Include -I/opt/homebrew/include', 'Py_DEBUG': 0, 'Py_ENABLE_SHARED': 0, 'Py_HASH_ALGORITHM': 0, 'Py_TRACE_REFS': 0, 'QUICKTESTOPTS': '-x test_subprocess test_io test_lib2to3 \\', 'READELF': ':', 'RESSRCDIR': 'Mac/Resources/framework', 'RETSIGTYPE': 'void', 'RUNSHARED': 'DYLD_FRAMEWORK_PATH=/private/tmp/pythonA3.9-20220326-49966-1n3ovp7/Python-3.9.12', 'SCRIPTDIR': '/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib', 'SETPGRP_HAVE_ARG': 0, 'SHELL': '/bin/sh', 'SHLIBS': '-ldl -framework CoreFoundation', 'SHLIB_SUFFIX': '.so', 'SHM_NEEDS_LIBRT': 0, 'SIGNED_RIGHT_SHIFT_ZERO_FILLS': 0, 'SITEPATH': '', 'SIZEOF_DOUBLE': 8, 'SIZEOF_FLOAT': 4, 'SIZEOF_FPOS_T': 8, 'SIZEOF_INT': 4, 'SIZEOF_LONG': 8, 'SIZEOF_LONG_DOUBLE': 8, 'SIZEOF_LONG_LONG': 8, 'SIZEOF_OFF_T': 8, 'SIZEOF_PID_T': 4, 'SIZEOF_PTHREAD_KEY_T': 8, 'SIZEOF_PTHREAD_T': 8, 'SIZEOF_SHORT': 2, 'SIZEOF_SIZE_T': 8, 'SIZEOF_TIME_T': 8, 'SIZEOF_UINTPTR_T': 8, 'SIZEOF_VOID_P': 8, 'SIZEOF_WCHAR_T': 4, 'SIZEOF__BOOL': 1, 'SOABI': 'cpython-39-darwin', 'SRCDIRS': 'Parser Parser/pegen Objects Python Modules Modules/_io Programs', 'SRC_GDB_HOOKS': './Tools/gdb/libpython.py', 'STDC_HEADERS': 1, 'STRICT_SYSV_CURSES': "/* Don't use ncurses extensions */", 'STRIPFLAG': '-s', 'SUBDIRS': '', 'SUBDIRSTOO': 'Include Lib Misc', 'SYSLIBS': '', 'SYS_SELECT_WITH_SYS_TIME': 1, 'TCLTK_INCLUDES': '', 'TCLTK_LIBS': '', 'TESTOPTS': '', 'TESTPATH': '', 'TESTPYTHON': 'DYLD_FRAMEWORK_PATH=/private/tmp/pythonA3.9-20220326-49966-1n3ovp7/Python-3.9.12 ./python.exe', 'TESTPYTHONOPTS': '', 'TESTRUNNER': 'DYLD_FRAMEWORK_PATH=/private/tmp/pythonA3.9-20220326-49966-1n3ovp7/Python-3.9.12 ./python.exe ./Tools/scripts/run_tests.py', 'TESTTIMEOUT': 1200, 'TIMEMODULE_LIB': 0, 'TIME_WITH_SYS_TIME': 1, 'TM_IN_SYS_TIME': 0, 'TZPATH': '/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo', 'UNICODE_DEPS': '\\', 'UNIVERSALSDK': '', 'UPDATE_FILE': 'python3 ./Tools/scripts/update_file.py', 'USE_COMPUTED_GOTOS': 0, 'VERSION': '3.9', 'WINDOW_HAS_FLAGS': 1, 'WITH_DECIMAL_CONTEXTVAR': 1, 'WITH_DOC_STRINGS': 1, 'WITH_DTRACE': 1, 'WITH_DYLD': 1, 'WITH_LIBINTL': 0, 'WITH_NEXT_FRAMEWORK': 1, 'WITH_PYMALLOC': 1, 'WITH_VALGRIND': 0, 'X87_DOUBLE_ROUNDING': 0, 'XMLLIBSUBDIRS': 'xml xml/dom xml/etree xml/parsers xml/sax', 'abs_builddir': '/private/tmp/pythonA3.9-20220326-49966-1n3ovp7/Python-3.9.12', 'abs_srcdir': '/private/tmp/pythonA3.9-20220326-49966-1n3ovp7/Python-3.9.12', 'datarootdir': '/opt/homebrew/opt/python@3.9/share', 'srcdir': '/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin', 'SO': '.cpython-39-darwin.so', 'userbase': '/Users/alex/Library/Python/3.9'}
It might be the case there are multiple sources of conflicting information. So we have to find out what pip is doing here to understand the problem.
Yeah, this sounds pretty likely.
It looks like pip install
(which doesn't use setuptools, I'm told) puts the console script in /opt/homebrew/bin
, whereas pip install -e
(which uses setuptools) puts it in /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/bin
, just like how python setup.py install
and python setup.py develop
do. I think that ideally, the script would always get installed in /opt/homebrew/bin
, since that's in the user's PATH, as macOS Homebrew intends.
I suspect this is the relevant bit of the Homebrew formula (python@3.9
). I'm not sure if the issue is on the setuptools end, or if perhaps something needs to be done to configure setuptools in this site-customisation code?
Thank you very much @alexreg, what I think might be happening here is the following:
In python@3.9
, Homebrew does re-define *prefix
and the "scripts" install scheme to point to /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9
, however, Homebrew also overwrite the default value for python setup.py install --prefix
to /usr/local/bin
via the distutils.cfg
file.
Since setuptools tries to overwrite distutils
with setuptools/_distutils
, the distutils/distutils.cfg
file can no longer be found.
(Please have in mind that I don't have a macOS computer to test this hypothesis, so I might be wrong.)
@jaraco, maybe we can solve this by adding yet another search path to distutils.dist.find_config_files
, something like the following (draft, not tested):
sys_file = os.path.join(sys_dir, "distutils.cfg")
if os.path.isfile(sys_file):
files.append(sys_file)
+ elif sys.platform == "darwin" and getattr(sys, "_framework", None):
+ # Here we try to restrict the changes to Homebrew,
+ # but we might want to always fallback regardless of the system
+ # (`sys._framework` seems to exists according to CPython's sysconfg).
+ alt_file = _get_sitepackages_config("distutils.cfg")
+ if alt_file is not None:
+ files.append(alt_file)
# ...
+ def _get_sitepackages_config(filename):
+ import site
+
+ for site_path in site.getsitepackages():
+ candidate = os.path.join(site_path, "distutils", filename)
+ if os.path.isfile(candidate):
+ return candidate
+ return None
@abravalheri Seems like you're spot on. To test, I copied /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/distutils.cfg
to /opt/homebrew//lib/python3.9/site-packages/setuptools/_distutils/distutils.cfg
, and then ran pip install -e .
on a project. This works as expect, and installs my project's console scripts into /opt/homebrew/bin
.
Here's the contents of the file, for record.
[install]
prefix=/opt/homebrew
[build_ext]
include_dirs=/opt/homebrew/include:/opt/homebrew/opt/openssl@1.1/include:/opt/homebrew/opt/sqlite/include
library_dirs=/opt/homebrew/lib:/opt/homebrew/opt/openssl@1.1/lib:/opt/homebrew/opt/sqlite/lib
Note that /opt/homebrew
is the Homebrew prefix on M1 Macs, and the analogue to the traditional /usr/local
.
Thank you very much for the feedback and tests @alexreg, let's see what Jason suggests as the path forward for this issue. We might need to change pypa/distutils
.
Thanks alexreg for the report and abravalheri for the detailed investigation.
I'm surprised I haven't encountered this issue, because I develop on an M1 mac using Python installed using Homebrew and use editable installs (pip -e
/setup.py develop
) as a standard part of my workflow. Oh, I guess the difference is that I almost exclusively use virtualenvs for those editable installs.
It's true that historically, there was no such thing as a standard install of distutils. Seemingly each distribution and platform variant had their own customization of distutils, and it seems homebrew was no exception. In this case, I'm pleased to see that Homebrew didn't monkeypatch distutils but only supplied a distutils.cfg file. Still, as you can see, now that distutils is migrating from stdlib to Setuptools, that hook no longer holds.
The good news is that the latest distutils does provide a supported mechanism for overriding install locations, and it's a mechanism shared by other projects, including pip, and that's by overriding sysconfig._INSTALL_SCHEMES
.
And on Python 3.10, I see the correct value:
>>> import sysconfig
>>> sysconfig.get_preferred_scheme('prefix')
'osx_framework_library'
>>> sysconfig.get_path('scripts')
'/opt/homebrew/bin'
So the next question - why isn't distutils using that value?
This issue is certainly rooted in distutils, so moving it there.
I see now that the scheme is essentially irrelevant, because it's the prefix
/ install_base
that's being customized by the pydistutils.cfg
file.
distutils main $ py -c "import distutils.command.install as inst; print(inst._load_scheme('posix_prefix')['scripts'])"
{base}/bin
Trying to figure out how sysconfig
resolves scripts to /opt/homebrew
, I took a look at the base
config var, but that's not where it happens:
distutils main $ py -c "import sysconfig; print(sysconfig.get_config_var('base')); print(sysconfig.get_path('scripts'))"
/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10
/opt/homebrew/bin
Hi @jaraco, I remember seeing that different Python versions distributed by homebrew would use different techniques to customize the installation scheme.
The customization for Python3.10 is likely to use a different approach than the ones for 3.9, 3.8, 3.7.
These links might be relevant:
Oh! Good tip. So maybe this issue doesn't affect Python 3.10?
I think maybe that's true:
distutils main $ py -c "import distutils.command.install as inst; print(inst._load_scheme(inst._resolve_scheme('posix_prefix'))['scripts'])"
/opt/homebrew/bin
distutils main $ py -3.9 -c "import distutils.command.install as inst; print(inst._load_scheme(inst._resolve_scheme('posix_prefix'))['scripts'])"
{base}/bin
Yeah, I think it does not affect 3.10. However it will still affect older versions (with EOS programmed to October 2025)...
I don't know if the Homebrew team is planning to keep supporting these versions and if they plan to change the customization mechanism for future patch releases of the "bottles"...
In that case, Python 3.9 and earlier will probably not get the install schemes treatment, so distutils is left without the patched context left by Homebrew. I'm now starting to agree with abravalheri's recommendation to honor pydistutils.cfg if found in the system site packages. In addition to limiting it to framework builds, I'd also want to limit it to Python 3.9 and earlier.
Another option could be simply to declare this use case as unsupported. It seems to be pretty rare. I believe there's a workaround where if you run with SETUPTOOLS_USE_DISTUTILS=stdlib
set in the environment, that should cause the old distutils to be invoked at the stdlib location, and thus honor the pydistutils.cfg
. e.g.:
$ env SETUPTOOLS_USE_DISTUTILS=stdlib pip install -e .
@alexreg Could you try that and confirm if the scripts install as expected?
@jaraco Yep, that works too.
I should note, however, that subsequently doing pip uninstall [package]
does not remove the console script, only the package itself. This was likewise the case with the other approach (symlinking distutils.cfg
).
Yep, that works too.
That's great news. Now the follow-up question - will this workaround be sufficient to get by until Python 3.9 is sunset or do you need a more robust solution?
Also, I should note that setuptools is working on a PEP 660 based solution that should obviate the need for pip to call setup.py develop and should address the issue, even for older Pythons. It just needs to land.
I should note, however, that subsequently doing
pip uninstall [package]
does not remove the console script, only the package itself. This was likewise the case with the other approach (symlinkingdistutils.cfg
).
I suspect that's the case irrespective of python versions or setuptools versions. I'm pretty sure pip relies on setuptools' setup.py develop to uninstall and it doesn't remove any scripts. That's a bug that's not going to get fixed until Setuptools adopts a PEP 660 solution.
@jaraco Good to know about the PEP 660 plans.
The problem right now is that Homebrew Python is stuck on 3.9, and will be so for the foreseeable future due to some weird policies. So, if a fix in setuptools is possible, that would be much appreciated.
In #155, I've drafted a compatibility shim that should enable installing resources to the homebrew prefix. I've tested the functionality superficially, but haven't tested it in a real-world environment. Here's how you can test it in your environment:
setuptools._distutils
with this copy of distutils.$ wget -O - https://github.com/pypa/distutils/archive/refs/heads/refactor/142-allow-pathlike.zip | bsdtar -x -C $(py -3.9 -c "import distutils, os; print(os.path.dirname(distutils.__file__))") --strip-components 2 --exclude docs/
Note the command above requires wget
, libarchive
, and python-launcher
(all available in homebrew). Note also that it applies the code directly to your environment, so use at your own risk. If you'd rather take it safer, just download the code yourself and copy over the files to your site-packages/setuptools/_distutils directory manually.
@jaraco Thank you, that's great. A brief test suggests your solution works for me too. (As before, pip uninstall
does not work, but that's a small matter in comparison. Shouldn't be a problem to wait for an implementation of PEP 660.)
Thanks Alexander for reviewing. I'll merge it and get it out to Setuptools and we can iterate if needed.
Just to clarify: patching sysconfig
for <= 3.9 on the Homebrew side was (and still is) considered as an option. There is however one big breakage (which we accepted as a "breaking change" for 3.10): --prefix
no longer works properly for site-packages etc. This is because that works by swapping base
and base
is not used in the modified sysconfig
for some items. We did make quite a bit of use of --prefix
in dependents, so some migration work would be required to add --install-lib
etc (which was easier to cover in a 3.9 -> 3.10 transition).
This is a limitation in sysconfig
. Fedora also share the same issue: https://bugzilla.redhat.com/show_bug.cgi?id=2026979. There's a few areas of sysconfig
that really haven't matured enough yet, e.g. https://github.com/python/cpython/issues/88142.
The reason --prefix
works in pip
is because we (ab)use this hack: https://github.com/pypa/pip/blob/ae324d17033253626725330c6635c0089927fbf5/src/pip/_internal/locations/_sysconfig.py#L157-L161. This hack wasn't actually Homebrew-specific. It was specific to Apple's Python and we just decided to name our scheme the same to make use of that.
setuptools version
60.10.0
Python version
3.9.12
OS
macOS 12.3.1
Additional environment information
Uses
python
formula from macOS Homebrew.Description
Installing a package with console scripts defined using
python setup.py develop
does not actually install the console scripts. (It works withpython setup.py install
, however.)This issue was previously discussed on the pypa/pip repo.
Expected behavior
Setuptools should install console script in the appropriate place.
How to Reproduce
setup.py
foopackage/__init__.py
python setup.py develop
.Output