sebastian-nomi
commented
10 months ago Upon delving into the codebase, I realized that: Invoke relies on the Popen class from the subprocess module to run the commands. When utilizing M1 Macs, certain information stored within the platform module doesn't align with the machine's actual specifications. Instead, it appears to mirror the details of the machine that was employed to compile the Python binary itself. While I haven't delved into the specifics, I think that Python employs the variables from the platform module within the subprocess module. This behavior potentially explains why processes and subprocesses generated by subprocess.run and Popen default to an architecture of i386 rather than arm64. For additional insights, you can refer to this discussion on Stack Overflow: Link.
When using with the subprocess module on M1 Macs, a crucial step involves explicitly specifying the architecture before invoking the desired command. Further details about this can be found in another Stack Overflow post: Link.
I opted to compile the invoke tool locally. I implemented a somewhat hacky modification to the start method within the local class located in the runners module. Subsequently, I compiled the binary in my local environment. This modification fixed my problem and I think something similar can be implemented in the actual module if this is too hacky for your guys. ideally, it would be nice to have the option to specify if we want to specify the architecture from the collection class or in the config file let user decide if he wants to use default or not.
Here's the specific modification I made:
`def start(self, command: str, shell: str, env: Dict[str, Any], set_arch:str = None) -> None: if self.using_pty:
if set_arch:
_command = ['arch', f'-{set_arch}', shell, "-c", command]
else:
_command = [shell, "-c", command]
if pty is None: # Encountered ImportError
err = "You indicated pty=True, but your platform doesn't support the 'pty' module!" # noqa
sys.exit(err)
cols, rows = pty_size()
self.pid, self.parent_fd = pty.fork()
# If we're the child process, load up the actual command in a
# shell, just as subprocess does; this replaces our process - whose
# pipes are all hooked up to the PTY - with the "real" one.
if self.pid == 0:
# TODO: both pty.spawn() and pexpect.spawn() do a lot of
# setup/teardown involving tty.setraw, getrlimit, signal.
# Ostensibly we'll want some of that eventually, but if
# possible write tests - integration-level if necessary -
# before adding it!
#
# Set pty window size based on what our own controlling
# terminal's window size appears to be.
# TODO: make subroutine?
winsize = struct.pack("HHHH", rows, cols, 0, 0)
fcntl.ioctl(sys.stdout.fileno(), termios.TIOCSWINSZ, winsize)
# Use execve for bare-minimum "exec w/ variable # args + env"
# behavior. No need for the 'p' (use PATH to find executable)
# for now.
# NOTE: stdlib subprocess (actually its posix flavor, which is
# written in C) uses either execve or execv, depending.
os.execve(shell, _command, env)
else:
if set_arch:
command = f'arch -{set_arch} ' + command
self.process = Popen(
command,
shell=True,
executable=shell,
env=env,
stdout=PIPE,
stderr=PIPE,
stdin=PIPE,
)`
I utilize Bazel and Go among other tools in my workflow. While attempting to execute Bazel through PyInvoke, I've encountered persistent build failures. It seems that the architecture utilized by c.run is i386, rather than arm64 which I require. Do you have any suggestions for overcoming this challenge? Additionally, I'm curious if there are any upcoming plans to introduce support for arm64? Your insights would be greatly appreciated.