gregneagle / relocatable-python

A tool for building standalone relocatable Python.framework bundles
Apache License 2.0
156 stars 42 forks source link

Add support to upgrade pip and install wheel prior to creating the bundle #6

Closed erikng closed 3 years ago

erikng commented 3 years ago

This adds support for the following:

  1. Upgrade pip prior to using it (this could fix issues with bugs in installations, like pyobjc)
  2. Installs wheel prior to installing the external modules/requirements file, so that we can speed up the process of making the relocatable python but also knowing that the developer of these modules has created the wheels themselves.
gregneagle commented 3 years ago

It's unclear to me what happens if the wheels are not universal. Will pip be smart and ignore the wheel and compile source as universal? I'd like faster builds, too, but I'd rather have reliably universal builds and if that means doing it slower, oh well...

gregneagle commented 3 years ago

I've never understood why, when this script installs pip, it sometimes results in a pip that has available updates. Why doesn't it just install pip-latest?

erikng commented 3 years ago

It's unclear to me what happens if the wheels are not universal. Will pip be smart and ignore the wheel and compile source as universal?

My understanding is yes, but without a M1 to test this, it's really hard to say. But for instance it compiled earlier prior to Ronald releasing the non-universal wheels for 7.0.1. He said he still has to figure out how to build/upload universal2 wheels.

re: https://github.com/ronaldoussoren/pyobjc/issues/333

erikng commented 3 years ago

I've never understood why, when this script installs pip, it sometimes results in a pip that has available updates. Why doesn't it just install pip-latest?

My guess is ensure pip just checks that it's installed and working, not that it's updated. pip is included in the 3.9.1 RC1 package, but it too is out of date.

gregneagle commented 3 years ago

"He said he still has to figure out how to build/upload universal2 wheels." which is why I'm worried we'll get x86_64 only code in PyObjC libs if we use the wheels.

gregneagle commented 3 years ago

I don't see why we need an M1 to test this. We can look at the .so and .dylibs and see if they are universal.

gregneagle commented 3 years ago

It does look like you can use ensurepip to upgrade pip if needed, so rather than calling ensurepip, the using pip to upgrade pip, maybe just get the latest pip the first time?

Help on package ensurepip:

NAME
    ensurepip

FILE
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ensurepip/__init__.py

MODULE DOCS
    https://docs.python.org/library/ensurepip

PACKAGE CONTENTS
    __main__
    _uninstall

FUNCTIONS
    bootstrap(root=None, upgrade=False, user=False, altinstall=False, default_pip=True, verbosity=0)
        Bootstrap pip into the current Python installation (or the given root
        directory).

        Note that calling this function will alter both sys.path and os.environ.

    version()
        Returns a string specifying the bundled version of pip.

DATA
    __all__ = ['version', 'bootstrap']

Note the upgrade to bootstrap

erikng commented 3 years ago

I have your answer regarding wheel. If it needs a universal2 one and it doesn't exist, it will create from source

Collecting pyobjc
  Downloading pyobjc-7.0.1-py3-none-any.whl (3.1 kB)
Collecting pyobjc-framework-InstallerPlugins==7.0.1
  Downloading pyobjc_framework_InstallerPlugins-7.0.1-py2.py3-none-any.whl (4.3 kB)
Collecting pyobjc-framework-Intents==7.0.1; platform_release >= "16.0"
  Downloading pyobjc-framework-Intents-7.0.1.tar.gz (107 kB)
     |████████████████████████████████| 107 kB 4.5 MB/s 

  Building wheel for pyobjc-framework-Intents (setup.py) ... done
  Created wheel for pyobjc-framework-Intents: filename=pyobjc_framework_Intents-7.0.1-cp36-abi3-macosx_11_0_universal2.whl size=23752 sha256=e2bf45f38a8f46dc8b94419b7bda50847a43cb73f4f85671e1aaca909ac09a50
  Stored in directory: /private/tmp/pip-ephem-wheel-cache-wz4c9vdl/wheels/42/48/d8/ba51cfc860c4a62838320e84f2605dea9f6ea62b76a24630ab
erikng commented 3 years ago

Also, surprisingly, it looks like the bootstrap function doesn't get the latest version!

relocatable-python % ./Python.framework/Versions/3.9/bin/python3.9 -s -m ensurepip --upgrade
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /var/folders/lz/b1h_3hks69b99smnyyrf46lc0000gn/T/tmp1veyannf
Processing /private/var/folders/lz/b1h_3hks69b99smnyyrf46lc0000gn/T/tmp1veyannf/setuptools-49.2.1-py3-none-any.whl
Processing /private/var/folders/lz/b1h_3hks69b99smnyyrf46lc0000gn/T/tmp1veyannf/pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
  WARNING: The script easy_install-3.9 is installed in '/Users/erikg/Library/Python/3.9/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts pip3 and pip3.9 are installed in '/Users/erikg/Library/Python/3.9/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-20.2.3 setuptools-49.2.1

relocatable-python % ./Python.framework/Versions/3.9/bin/python3.9 -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
  Using cached pip-20.3-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.2.3
    Uninstalling pip-20.2.3:
      Successfully uninstalled pip-20.2.3
  WARNING: The scripts pip, pip3 and pip3.9 are installed in '/Users/erikg/Library/Python/3.9/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-20.3