How to patch prior to installing Python 3.4.10?

[karthicraghupathi]

I'm looking to install Python 3.4.10 using asdf-python on macOS.

On running asdf install python 3.4.10, I get this error:

python-build 3.4.10 /Users/karthicr/.asdf/installs/python/3.4.10
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.4.10.tar.xz...
-> https://www.python.org/ftp/python/3.4.10/Python-3.4.10.tar.xz
Installing Python-3.4.10...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 10.15.7 using python-build 1.2.20-19-g8ac91b4f)

Inspect or clean up the working tree at /var/folders/dy/1vzm3lbx1673twz_pmd8vcnmr40220/T/python-build.20201007152143.45040
Results logged to /var/folders/dy/1vzm3lbx1673twz_pmd8vcnmr40220/T/python-build.20201007152143.45040.log

Last 10 log lines:
Python/random.c:89:19: note: did you mean 'py_getentropy'?
Python/random.c:81:1: note: 'py_getentropy' declared here
py_getentropy(unsigned char *buffer, Py_ssize_t size, int fatal)
^
Python/random.c:98:19: error: implicit declaration of function 'getentropy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
            res = getentropy(buffer, len);
                  ^
2 errors generated.
make: *** [Python/random.o] Error 1
make: *** Waiting for unfinished jobs....

On doing some research, I believe https://bugs.python.org/issue28676 is probably why the install is failing. However I see that patch has not made it into 3.4.x. I'm now trying to apply patch https://bugs.python.org/file45465/getentropy.patch prior to installing version 3.4.10.

I believe since asdf-python is a wrapper over pyenv, per instructions at https://github.com/pyenv/pyenv/tree/master/plugins/python-build#applying-patches-to-python-before-compiling, I ran asdf install python --patch 3.4.10 < <(curl -sSL https://bugs.python.org/file45465/getentropy.patch), however, I'm getting the following error:

python-build --patch /Users/karthicr/.asdf/installs/python/--patch
Usage: python-build [-kpv] <definition> <prefix>
       python-build --definitions
       python-build --version

  -k/--keep        Do not remove source tree after installation
  -p/--patch       Apply a patch from stdin before building
  -v/--verbose     Verbose mode: print compilation status to stdout
  -4/--ipv4        Resolve names to IPv4 addresses only
  -6/--ipv6        Resolve names to IPv6 addresses only
  --definitions    List all built-in definitions
  --version        Show version of python-build
  -g/--debug       Build a debug version

(23) Failed writing body

Can I get some pointers on how to use the --patch option when installing Python?

[donaldguy]

Its a dirty workaround, and maybe directly relevant or not?

but to install 3.8.2 (to match Ubuntu 20.04) on my Big Sur mac, for the time being my script that would call asdf install (notably without a version number, so implied current; but you can script around that?) instead calls

function do_asdf_install {
  plugin=$1

  # https://github.com/pyenv/pyenv/issues/1643
  if [[ "$plugin" = "python" && "$(uname -s)" = "Darwin" ]] && uname -r | grep -q '^20'; then
    cp $ASDF_DIR/plugins/python/bin/install  $ASDF_DIR/plugins/python/bin/install.orig

    # https://github.com/python/cpython/pull/21113
    perl -npe 's|\$\(python_build_path\)\K(.*)| --patch \1 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\\?full_index\\=1)|' \
      $ASDF_DIR/plugins/python/bin/install.orig > $ASDF_DIR/plugins/python/bin/install
  fi
  asdf install $plugin
  asdf reshim $plugin

  if test -f $ASDF_DIR/plugins/$plugin/bin/install.orig; then
    mv $ASDF_DIR/plugins/$plugin/bin/install.orig $ASDF_DIR/plugins/$plugin/bin/install;
  fi
}

instead

[donaldguy]

a more useful asdf-python fix if @danhper et al would like it) would be like ~roughly:

diff --git bin/install bin/install
index 4a93aab..a2d89cd 100755
--- bin/install
+++ bin/install
@@ -16,8 +16,14 @@ install_python() {
   fi
   install_or_update_python_build

-  echo "python-build $version $install_path"
-  $(python_build_path) "$version" "$install_path"
+  if [[ -n "${ASDF_PYTHON_PATCH_URL:-}" ]]; then 
+    echo "python-build --patch $version $install_path"
+    echo "with patch file from: $ASDF_PYTHON_PATCH_URL"
+    $(python_build_path) --patch "$version" "$install_path" < <(curl -sSL "$ASDF_PYTHON_PATCH_URL")
+  else
+    echo "python-build $version $install_path"
+    $(python_build_path) "$version" "$install_path"
+  fi
 }

 install_default_python_packages() {

then just e.g.

export ASDF_PYTHON_PATCH_URL="https://bugs.python.org/file45465/getentropy.patch"
asdf install python 3.4.10

EDIT: though doing literally that^ as OP doesn't seem to apply - looks like that patch is malformed?

my version

export ASDF_PYTHON_PATCH_URL="https://github.com/python/cpython/commit/8ea6353.patch?full_index=1"
asdf install python 3.8.2

works tho

[karthicraghupathi]

@donaldguy Thanks for the pointer. I'm going to give that a shot and will let you know how it goes.

[danhper]

a more useful asdf-python fix if @danhper et al would like it) would be like ~roughly:

diff --git bin/install bin/install
index 4a93aab..a2d89cd 100755
--- bin/install
+++ bin/install
@@ -16,8 +16,14 @@ install_python() {
   fi
   install_or_update_python_build

-  echo "python-build $version $install_path"
-  $(python_build_path) "$version" "$install_path"
+  if [[ -n "${ASDF_PYTHON_PATCH_URL:-}" ]]; then 
+    echo "python-build --patch $version $install_path"
+    echo "with patch file from: $ASDF_PYTHON_PATCH_URL"
+    $(python_build_path) --patch "$version" "$install_path" < <(curl -sSL "$ASDF_PYTHON_PATCH_URL")
+  else
+    echo "python-build $version $install_path"
+    $(python_build_path) "$version" "$install_path"
+  fi
 }

 install_default_python_packages() {

then just e.g.

export ASDF_PYTHON_PATCH_URL="https://bugs.python.org/file45465/getentropy.patch"
asdf install python 3.4.10

EDIT: though doing literally that^ as OP doesn't seem to apply - looks like that patch is malformed?

my version

export ASDF_PYTHON_PATCH_URL="https://github.com/python/cpython/commit/8ea6353.patch?full_index=1"
asdf install python 3.8.2

works tho

Thanks for sharing. I would be happy to merge if you could send a PR. Thanks!

[karthicraghupathi]

@donaldguy I applied your suggested fix and now I'm able to use the ASDF_PYTHON_PATCH_URL to apply patches when installing python with asdf-python.

And like you mentioned, the link to the getentropy patch in my issue description does not target Python 3.4.x on macOS. I was able to find the correct patch here: https://gist.github.com/orip/00142a498cf6d0c536b5d38771ad6249. This patch did fix the getentropy issue I was encountering earlier. However I now have a new problem but that is an entirely different issue altogether.

For all intents and purposes, your fix now provides users the ability to apply patch files while installing python versions. Thanks a bunch for looking into this and providing a fix.

I'm hoping you can send a PR so this fix can be incorporated into asdf-python as mentioned by @danhper. If you are unable to do so, I'm happy to send a PR crediting you for the fix.

Now I'm going to search the webs to see if I can find yet another patch that fixes this new error:

$ ASDF_PYTHON_PATCH_URL="https://gist.githubusercontent.com/orip/00142a498cf6d0c536b5d38771ad6249/raw/8cd96db639a723803fb072d0b15697f9c95a19e4/Python-3.4.3-macos.patch" asdf install python 3.4.10
python-build --patch 3.4.10 /Users/karthicr/.asdf/installs/python/3.4.10
with patch file from: https://gist.githubusercontent.com/orip/00142a498cf6d0c536b5d38771ad6249/raw/8cd96db639a723803fb072d0b15697f9c95a19e4/Python-3.4.3-macos.patch
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.4.10.tar.xz...
-> https://www.python.org/ftp/python/3.4.10/Python-3.4.10.tar.xz
Installing Python-3.4.10...
patching file Python/random.c
patch unexpectedly ends in middle of line
Hunk #1 succeeded at 3 with fuzz 1.
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.1 using python-build 1.2.21-6-g2bf6111f)

Inspect or clean up the working tree at /var/folders/yq/82xt_rsn3ds1473grw012dqxtx0bq9/T/python-build.20210102005545.6062
Results logged to /var/folders/yq/82xt_rsn3ds1473grw012dqxtx0bq9/T/python-build.20210102005545.6062.log

Last 10 log lines:
./Modules/posixmodule.c:8283:12: note: forward declaration of 'struct sf_hdtr'
    struct sf_hdtr sf;
           ^
./Modules/posixmodule.c:8339:11: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    ret = sendfile(in, out, offset, &sbytes, &sf, flags);
          ^
clang -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include   -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include   -Werror=declaration-after-statement   -I. -IInclude -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/karthicr/.asdf/installs/python/3.4.10/include  -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/karthicr/.asdf/installs/python/3.4.10/include   -DPy_BUILD_CORE  -c ./Modules/_weakref.c -o Modules/_weakref.o
6 warnings and 2 errors generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....

[karthicraghupathi]

Thanks @pierreneter for creating the PR. I will go ahead and close this ticket.