build/bin/sage-system-python: Try "python" first

[mkoeppe]

This is a workaround for a minor annoyance affecting sage developers who update packages on macOS.

python2 urllib works correctly with ssl but the python3 may not (as discussed in #29418). The correct fix is to update sage_bootstrap to initialize certificates, or to use a higher-level library for downloading. The workaround to use python2 gives us a bit of time... until Apple removes python2 from the OS.

CC: @jhpalmieri

Component: build

Author: Matthias Koeppe

Branch: 1d36a2e

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/29890

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,7 @@
-This has an advantage on macOS, where the python2 urllib works correctly with ssl but the python3 may not.
+This is a workaround for a minor annoyance affecting sage developers who update packages on macOS.
+
+python2 urllib works correctly with ssl but the python3 may not (as discussed in #29418). The correct fix is to update `sage_bootstrap` to initialize certificates, or to use a higher-level library for downloading. The workaround to use python2 gives us a bit of time... until Apple removes python2 from the OS.

+
+

[mkoeppe]

Branch: u/mkoeppe/build_bin_sage_system_pythontrypython__first

[mkoeppe]

Commit: 1d36a2e

[mkoeppe]

New commits:

1d36a2e

build/bin/sage-system-python: Try python first

[jhpalmieri]

Reviewer: John Palmieri

[jhpalmieri]

comment:5

Seems to work with a build from scratch and also building a bunch of optional packages. I didn't see any new breakages with those, just packages which were broken independently of this branch: boost, database_stein_watkins_mini, sage_numerical_backends_cplex, sage_numerical_backends_gurobi, texlive (although why anyone would want to install texlive via Sage is baffling to me).

I tested this on OS X using very few system packages, to try out installing as many of Sage's packages as possible (in case some package used Python 3 syntax in its installation process). I used the system's gfortran because of lack of patience, and I used the system's curl because Sage's build of R failed when Sage built its own curl. That is, I used this command:

./configure --with-system-boost_cropped=no --with-system-curl=yes --with-system-freetype=no --with-system-glpk=no --with-system-gmp=no --with-system-gsl=no --with-system-iconv=no --with-system-libffi=no --with-system-libgd=no --with-system-libpng=no --with-system-mpc=no --with-system-mpfi=no --with-system-mpfr=no --with-system-mpir=no --with-system-ncurses=no --with-system-ninja_build=no --with-system-ntl=no --with-system-openblas=no --with-system-pkgconf=no --with-system-ppl=no --with-system-python3=no --with-system-r=no --with-system-readline=no --with-system-sqlite=no --with-system-suitesparse=no --with-system-xz=no --with-system-yasm=no --with-system-zeromq=no --with-system-zlib=no

Was there some quicker way to tell ./configure to ignore system packages?

[jhpalmieri]

comment:6

database_stein_watkins_mini is taken care of at #29897.

[mkoeppe]

comment:7

Replying to @jhpalmieri:

I used this command:

./configure --with-system-boost_cropped=no --with-system-curl=yes --with-system-freetype=no --with-system-glpk=no --with-system-gmp=no --with-system-gsl=no --with-system-iconv=no --with-system-libffi=no --with-system-libgd=no --with-system-libpng=no --with-system-mpc=no --with-system-mpfi=no --with-system-mpfr=no --with-system-mpir=no --with-system-ncurses=no --with-system-ninja_build=no --with-system-ntl=no --with-system-openblas=no --with-system-pkgconf=no --with-system-ppl=no --with-system-python3=no --with-system-r=no --with-system-readline=no --with-system-sqlite=no --with-system-suitesparse=no --with-system-xz=no --with-system-yasm=no --with-system-zeromq=no --with-system-zlib=no

Was there some quicker way to tell ./configure to ignore system packages?

Unfortunately, no.

[mkoeppe]

comment:8

Replying to @jhpalmieri:

Seems to work with a build from scratch and also building a bunch of optional packages. I didn't see any new breakages with those, just packages which were broken independently of this branch: boost, database_stein_watkins_mini, sage_numerical_backends_cplex, sage_numerical_backends_gurobi, texlive (although why anyone would want to install texlive via Sage is baffling to me).

Probably we should add automatic testing for all optional packages. .github/workflows/tox-optional.yml only tests the following:

4ti2 pynormaliz qepcad lrslib latte_int topcom barvinok isl qhull sage_numerical_backends_coin primecount plantri polymake jupymake kenzo libsemigroups mcqd meataxe mpfrcx openssl p_group_cohomology rst2ipynb sirocco tdlib tides

(and their dependencies).

[jhpalmieri]

comment:9

The experimental packages are even worse: I tried most of them, and only one (compilerwrapper) succeeded.

[mkoeppe]

comment:10

Let's continue on #29900 (Meta-ticket: Fix optional and experimental packages for Sage 9.2)

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 1d36a2e to 1c78ade

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

1c78ade

build/bin/sage-system-python: Test that python has argparse

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 1c78ade to 1d36a2e

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

[mkoeppe]

comment:13

Sorry, pushed to the wrong ticket.

[vbraun]

Changed branch from u/mkoeppe/build_bin_sage_system_pythontrypython__first to 1d36a2e

[mkoeppe]

comment:15

Follow up in #29929

[mkoeppe]

Changed commit from 1d36a2e to none