search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.47k stars 486 forks source link

Meta-ticket: support macOS 11 Big Sur #30651

Closed slel closed 3 years ago

slel commented 4 years ago

This is to support macOS 11 Big Sur.

Related:

References:

Depends on #30494

CC: @dimpase @jhpalmieri @mkoeppe @slel @vbraun @williamstein

Component: porting

Keywords: macOS

Issue created by migration from https://trac.sagemath.org/ticket/30651

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -5,3 +5,8 @@
 Related:

 - #30494: Meta-ticket: Support Xcode 12
+
+References:
+- https://bugs.python.org/issue41116
+
+
wdjoyner commented 4 years ago
comment:3

I'm trying to compile sage-9.2.rc0 on big sur with xcode 12.0.1.

I ran sage -f openblas. Here's the tail of that:

(see the openblas attachment).

After trying lots of things such as

$ ./configure \
  --enable-universalsdk=$(xcodebuild -version -sdk macosx Path) \
  --with-universal-archs=intel-64
$ source /Users/wdj/sagefiles/sage-9.2.rc0/.homebrew-build-env
$ ./config.status --recheck && ./config.status
$ ./sage -f gfortran

the compilation of sage -f gfortran stopped with an error compiling R.

However, Sage did start and crash with this error:

wdj@jeeves sage-9.2.rc0 % ./sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.2.rc0, Release Date: 2020-10-05                 │
│ Using Python 3.8.5. Type "help()" for help.                        │
└────────────────────────────────────────────────────────────────────┘
---------------------------------------------------------------------------
ModuleNotFoundErrorPython 3.8.5:
/Users/wdj/sagefiles/sage-9.2.rc0/local/bin/python3
Thu Oct  8 13:27:16 2020
A problem occurred executing Python code.  Here is the sequence of function
calls leading up to the error, with the most recent (innermost) call last.

(for the rest, see the error-in-start attachment).

It looks like Sage's Python doesn't work, right? Wrong:

wdj@jeeves sage-9.2.rc0 % ./sage -python3
Python 3.8.5 (default, Oct  7 2020, 13:59:53)
[Clang 12.0.0 (clang-1200.0.32.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 2+3
5

In case it's any use, here is what homebrew claims it has installed:

wdj@jeeves sage-9.2.rc0 % brew list --versions
autoconf 2.69
automake 1.16.1_1
bdw-gc 8.0.4
boost 1.71.0
cmake 3.15.5
fontconfig 2.13.1
freetype 2.10.1
gcc 9.2.0_1
gd 2.2.5
gettext 0.20.1 0.21
gmp 6.2.0 6.1.2_2
gnutls 3.6.15
gpatch 2.7.6
guile 3.0.4
icu4c 64.2
isl 0.22.1 0.21
jpeg 9c
libde265 1.0.3
libdvdcss 1.4.2
libevent 2.1.12
libffi 3.3 3.2.1
libidn2 2.3.0 2.2.0_1
libmpc 1.1.0 1.2.0
libomp 9.0.0
libpng 1.6.37
libtasn1 4.14 4.16.0
libtiff 4.0.10_1
libtool 2.4.6_1 2.4.6_2
libunistring 0.9.10
libyaml 0.2.5
mpfi 1.5.3
mpfr 4.1.0 4.0.2
nettle 3.4.1 3.6
ntl 11.4.1
openssl 1.0.2t
openssl@1.1 1.1.1g 1.1.1d
p11-kit 0.23.21 0.23.18.1
pari 2.11.4
pkg-config 0.29.2_3 0.29.2
readline 8.0.4
ruby 2.7.1_2
sqlite 3.33.0
unbound 1.11.0
webp 1.0.3
xz 5.2.4
jhpalmieri commented 4 years ago
comment:4

I've actually gotten this to build for me without patching Sage in any way, just building 9.2.rc0. Homebrew setup:

% brew list
autoconf    gd           haskell-stack  libpng        ncurses      r
automake    gdbm         icu4c          libtasn1      nettle       re2c
bdw-gc      gettext      ilmbase        libtiff       ninja        readline
bison       ghc          intltool       libtool       openblas     sphinx-doc
boost       ghostscript  isl            libunistring  openexr      sqlite
bzip2       git          jansson        libxml2       openssl@1.1  suite-sparse
cmake       glib         jpeg           meson         p11-kit      tbb
curl        glpk         libatomic_ops  metis         pandoc       texinfo
doxygen     gmp          libevent       mpc           pcre         unbound
fontconfig  gnutls       libffi         mpfi          pcre2        webp
freetype    gpatch       libidn2        mpfr          pkg-config   xz
gcc         gsl          libmpc         mpir          ppl          yasm
gcc@9       guile        libmpdclient   nauty         python@3.8   zeromq

Then I used the command

$ ./configure \
  --with-system-readline=no \
  --with-system-zlib=no \
  --with-system-gfortran=no \
  --with-system-openblas=no \
  --with-system-bzip2=no

after which make worked but make ptestlong showed a bunch of failures, all like this:

Expected nothing
Got:
    ld: warning: dylib (/usr/local/lib/libgmp.dylib) was built for newer macOS version (10.15) than being linked (10.9)
    ld: warning: dylib (/usr/local/lib/libmpfr.dylib) was built for newer macOS version (10.15) than being linked (10.9)
    ld: warning: dylib (/usr/local/lib/libgmpxx.dylib) was built for newer macOS version (10.15) than being linked (10.9)

Maybe because MACOSX_DEPLOYMENT_TARGET is being set? I could try building Sage's mpfr and gmp. But in any case, I got Sage to build, unfortunately by not using a bunch of the system versions of things.

jhpalmieri commented 4 years ago
comment:5

Replying to @jhpalmieri:

Maybe because MACOSX_DEPLOYMENT_TARGET is being set? I could try building Sage's mpfr and gmp.

I could try that, but it won't work because our mpir is broken when using Xcode 12, with the usual error, and the latest release of mpir (git clone git://github.com/wbhart/mpir.git mpir) doesn't help.

dimpase commented 4 years ago
comment:6

mpir is basically not developed any more (for the time being at least), so we should not use it.

What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.

dimpase commented 4 years ago
comment:7

Replying to @wdjoyner:

I'm trying to compile sage-9.2.rc0 on big sur with xcode 12.0.1.

I ran sage -f openblas. Here's the tail of that:

[...]

David, could you please replace this 10-screen comment by an attachement?

wdjoyner commented 4 years ago

Attachment: openblas-tail.txt

openblas error

wdjoyner commented 4 years ago

Attachment: error-in-start.txt

error in start

jhpalmieri commented 4 years ago
comment:8

Replying to @dimpase:

mpir is basically not developed any more (for the time being at least), so we should not use it.

It's a standard package, so I'm confused about what the alternatives are.

Anyway, I hacked the spkg-install.in script for mpir, as in #30729, and I got it to build. That eliminated those doctest failures. I'm left with two:

sage -t --long --random-seed=0 src/sage/misc/gperftools.py
**********************************************************************
File "src/sage/misc/gperftools.py", line 118, in sage.misc.gperftools.Profiler._libc
Failed example:
    Profiler()._libc()
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.gperftools.Profiler._libc[1]>", line 1, in <module>
        Profiler()._libc()
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/misc/gperftools.py", line 129, in _libc
        raise ImportError('failed to open libc')
    ImportError: failed to open libc

and

sage -t --long --random-seed=0 src/sage/tests/parigp.py
    Timed out

What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.

I haven't changed Sage, so it's whatever gets set in sage-env: 10.9, I think.

slel commented 4 years ago
comment:9

Replying to @jhpalmieri:

Replying to @dimpase:

mpir is basically not developed any more (for the time being at least), so we should not use it.

It's a standard package, so I'm confused about what the alternatives are.

One can choose between gmp and mpir. Both are standard, but only one is needed.

Use the configure flag --with-mp=gmp or with-mp=mpir to choose.

Likewise, one can choose between ATLAS and OpenBLAS, and both are standard.

Use --with-blas=openblas or --with-blas=atlas for that.

In that case #30350 proposes to remove ATLAS.

jhpalmieri commented 4 years ago
comment:10

Sorry, I led us on a bit of a wild goose chase for the past few comments. Back in comment:4, I said the right thing:

I could try building Sage's mpfr and gmp.

But then (maybe I was tired or distracted), I started building Sage's mpir (and also Sage's gmp, which was appropriate). There is no reason to care about mpir: just use --with-system-gmp=no and Sage will build its own gmp and mpfr.

mkoeppe commented 4 years ago
comment:11

Replying to @jhpalmieri:

Replying to @dimpase:

What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.

I haven't changed Sage, so it's whatever gets set in sage-env: 10.9, I think.

Just a quick remark - MACOSX_DEPLOYMENT_TARGET only gets set in builds configured --without-system-python. (See #30724, which makes this code in sage-env cleaner.)

jhpalmieri commented 4 years ago
comment:12

Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own, so MACOSX_DEPLOYMENT_TARGET is relevant. (Also, Sage's Python can't find various system libraries to build the corresponding modules, which is why I'm passing the flags --with-system-zlib=no --with-system-bzip2=no to configure.)

mkoeppe commented 4 years ago
comment:13

Replying to @jhpalmieri:

Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own

Is this because of dependencies of the python3 package? In this case, #30559 may be relevant

wdjoyner commented 4 years ago
comment:14

Replying to @jhpalmieri:

I've actually gotten this to build for me without patching Sage in any way, just building 9.2.rc0.

...

Then I used the command

./configure --with-system-readline=no --with-system-zlib=no --with-system-gfortran=no --with-system-openblas=no --with-system-bzip2=no

make worked.

This configure command followed by make gave me a functioning sage, so thank you!

R didn't compile but I don't use it.

jhpalmieri commented 4 years ago
comment:15

Replying to @mkoeppe:

Replying to @jhpalmieri:

Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own

Is this because of dependencies of the python3 package?

Yes, sqlite is not successfully detected. I'll try turning that dependency off.

jhpalmieri commented 4 years ago
comment:16

Sage built using the system Python after modifying the dependencies to avoid sqlite. I used

./configure --with-system-gfortran=no --with-system-openblas=no --with-system-gmp=no

Unfortunately the documentation won't build and many doctests fail , in both cases with a bus error. For example:

File "src/sage/algebras/quatalg/quaternion_algebra.py", line 107, in sage.algebras.quatalg.quaternion_algebra.QuaternionAlgebraFactory
Failed example:
    QuaternionAlgebra(sqrt(-1), sqrt(-3))
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.algebras.quatalg.quaternion_algebra.QuaternionAlgebraFactory[4]>", line 1, in <module>
        QuaternionAlgebra(sqrt(-Integer(1)), sqrt(-Integer(3)))
      File "sage/structure/factory.pyx", line 367, in sage.structure.factory.UniqueFactory.__call__ (build/cythonized/sage/structure/factory.c:2181)
        key, kwds = self.create_key_and_extra_args(*args, **kwds)
      File "sage/structure/factory.pyx", line 471, in sage.structure.factory.UniqueFactory.create_key_and_extra_args (build/cythonized/sage/structure/factory.c:3402)
        return self.create_key(*args, **kwds), {}
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/algebras/quatalg/quaternion_algebra.py", line 246, in create_key
        if a == 0 or b == 0:
      File "sage/symbolic/expression.pyx", line 3182, in sage.symbolic.expression.Expression.__nonzero__ (build/cythonized/sage/symbolic/expression.cpp:20543)
        res = self.test_relation()
      File "sage/symbolic/expression.pyx", line 3296, in sage.symbolic.expression.Expression.test_relation (build/cythonized/sage/symbolic/expression.cpp:21404)
        if self.lhs().is_algebraic() and self.rhs().is_algebraic():
      File "sage/symbolic/expression.pyx", line 2257, in sage.symbolic.expression.Expression.is_algebraic (build/cythonized/sage/symbolic/expression.cpp:15979)
        ex = sage.rings.all.QQbar(self)
      File "sage/structure/parent.pyx", line 900, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9336)
        return mor._call_(x)
      File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4515)
        return C._element_constructor(x)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/rings/qqbar.py", line 1496, in _element_constructor_
        return x._algebraic_(QQbar)
      File "sage/symbolic/expression.pyx", line 1754, in sage.symbolic.expression.Expression._algebraic_ (build/cythonized/sage/symbolic/expression.cpp:12658)
        return algebraic(self, field)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 1227, in algebraic
        return AlgebraicConverter(field)(ex)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 218, in __call__
        return self.arithmetic(ex, operator)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 1061, in arithmetic
        return self.field(base**expt)
      File "sage/structure/element.pyx", line 2057, in sage.structure.element.Element.__pow__ (build/cythonized/sage/structure/element.c:14456)
        return coercion_model.bin_op(left, right, pow)
      File "sage/structure/coerce.pyx", line 1196, in sage.structure.coerce.CoercionModel.bin_op (build/cythonized/sage/structure/coerce.c:10205)
        return (<Action>action)._act_(y, x)
      File "sage/categories/action.pyx", line 173, in sage.categories.action.Action._act_ (build/cythonized/sage/categories/action.c:3900)
        cpdef _act_(self, g, x):
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/rings/qqbar.py", line 6054, in _act_
        target_arg.sin() * target_abs)
      File "sage/rings/real_mpfi.pyx", line 4557, in sage.rings.real_mpfi.RealIntervalFieldElement.sin (build/cythonized/sage/rings/real_mpfi.c:26988)
        sig_on()
    cysignals.signals.SignalError: Bus error
dimpase commented 4 years ago
comment:17

Replying to @jhpalmieri:

Sorry, I led us on a bit of a wild goose chase for the past few comments. Back in comment:4, I said the right thing:

I could try building Sage's mpfr and gmp.

But then (maybe I was tired or distracted), I started building Sage's mpir (and also Sage's gmp, which was appropriate). There is no reason to care about mpir: just use --with-system-gmp=no and Sage will build its own gmp and mpfr.

They cannot be, and are not, "installed together" - mpir uses the "gmp compatibility" mode where it installs gmp-compatible libs etc. You need to run ./configure with--with-mp=gmpto select gmp as the package to use formp, (= multiple precision), as the default formpismpir`.

Default mp should be changed to gmp, I think, as mpir is getting bitrotten more and more. See #30752 (where a lie about mpir/gmp told by ./configure -h is uncovered, as well)

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,6 +1,8 @@
 This is to support macOS 11 Big Sur.

-
+- #30752 gmp instead of mpir
+- #30929 GH Actions: Update macos test environments
+- #30745 homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local

 Related:

@@ -8,5 +10,7 @@

 References:
 - https://bugs.python.org/issue41116
+- https://groups.google.com/g/sage-devel/c/5lBTr4gZuic/m/RkisHYLWCgAJ

+
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -3,6 +3,10 @@
 - #30752 gmp instead of mpir
 - #30929 GH Actions: Update macos test environments
 - #30745 homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local
+- #31050 upgrade wheel
+- `MACOSX_DEPLOYMENT_TARGET` problems - https://groups.google.com/g/sage-devel/c/Sj3YBVJoHQM/m/Zt-Ih_hJAwAJ
+- #30589 Upgrade Python to 3.9.1
+

 Related:

@@ -12,5 +16,3 @@
 - https://bugs.python.org/issue41116
 - https://groups.google.com/g/sage-devel/c/5lBTr4gZuic/m/RkisHYLWCgAJ

-
-
jhpalmieri commented 3 years ago
comment:22

I am having problems building scipy. This is after installing #31050, #30589, upgrades to setuptools and setuptools_scm, plus patches (related to parsing the OS X version number) to cffi and numpy. Then scipy fails with

  /usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so
  gfortran: warning: couldn't understand version 11
  ld: library not found for -lgcc_s.10.4
  collect2: error: ld returned 1 exit status
  error: Command "/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so" failed with exit status 1
  Building wheel for scipy (PEP 517): finished with status 'error'
  ERROR: Failed building wheel for scipy
isuruf commented 3 years ago
comment:23

Try doing export MACOSX_DEPLOYMENT_TARGET=10.16 to fix the issue until gfortran understands 11

isuruf commented 3 years ago
comment:24

Or you can set SYSTEM_VERSION_COMPAT=1 which would make the system return 10.16 automatically

jhpalmieri commented 3 years ago
comment:25

I don't know how to set these variables in such a way to get Sage to build. (This is with the changes listed in comment:22.)

isuruf commented 3 years ago
comment:26

If I set them globally, then pillow fails: it can't find zlib.

What's the error?

jhpalmieri commented 3 years ago
comment:27

Now I'm seeing a different error when building pillow, not about zlib:

clang: error: the clang compiler does not support '-march=native'

Details: I did

% export MACOSX_DEPLOYMENT_TARGET=10.16
% make distclean; ./configure; make pillow

Same error whether I use vanilla Sage 9.3.beta5, or that + #30589, or that + #30589 + new versions of setuptools and setuptools_scm. Pillow builds successfully if I then do

% unset MACOSX_DEPLOYMENT_TARGET
% make distclean; ./configure; make pillow
isuruf commented 3 years ago
comment:28

logs please

jhpalmieri commented 3 years ago
comment:29

If I instead do export SYSTEM_VERSION_COMPAT=1, the Pillow build gets farther but ends with

ERROR: Pillow-8.0.1-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
jhpalmieri commented 3 years ago

vanilla 9.3.beta5

jhpalmieri commented 3 years ago

Attachment: pillow-8.0.1-VANILLA.log

built using #30589

jhpalmieri commented 3 years ago

Attachment: pillow-8.0.1-30589.log

built with SYSTEM_VERSION_COMPAT=1

jhpalmieri commented 3 years ago

Attachment: pillow-8.0.1-SYSTEM_VERSION_COMPAT=1.log

top-level config.log

jhpalmieri commented 3 years ago
comment:30

Attachment: config.log

Here are various logs. Let me know if you want others.

isuruf commented 3 years ago
comment:31

Try patching pip at https://github.com/pypa/pip/blob/master/src/pip/_vendor/packaging/tags.py#L438 by doing,

if version == (10, 16):
    version = (11, 0)
jhpalmieri commented 3 years ago
comment:32

That patch has no obvious effect when setting MACOSX_DEPLOYMENT_TARGET or SYSTEM_VERSION_COMPAT or both: I see the same errors.

isuruf commented 3 years ago
comment:33

I edited the patch. Can you retry?

jhpalmieri commented 3 years ago
comment:34

I did the same edit independently. With SYSTEM_VERSION_COMPAT=1, Pillow now builds! With MACOSX_DEPLOYMENT_TARGET=10.16, it does not: same error as before.

jhpalmieri commented 3 years ago
comment:35

And in case it matters, this is all with 9.3.beta5 + #30589 + updated setuptools and setuptools_scm.

jhpalmieri commented 3 years ago
comment:36

After running make pillow, I tried make numpy and got an unexpected error:

Package 'numpy' is currently not installed
Uninstalling 'numpy' with legacy uninstaller
Traceback (most recent call last):
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/var/tmp/sage/build/numpy-1.19.4/src/../lapack_conf.py", line 7, in <module>
    pc_blas   = pkgconfig.parse('cblas blas')
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 248, in parse
    _raise_if_not_exists(package)
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 103, in _raise_if_not_exists
    raise PackageNotFoundError(package)
pkgconfig.pkgconfig.PackageNotFoundError: cblas not found

Missing dependency? After running just make, it has now reached numpy and gotten past this problem.

jhpalmieri commented 3 years ago
comment:37

scipy failed, unfortunately:

  /usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so
  gfortran: warning: couldn't understand version 11
  ld: library not found for -lgcc_s.10.4
  collect2: error: ld returned 1 exit status
  error: Command "/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so" failed with exit status 1
  Building wheel for scipy (PEP 517): finished with status 'error'

numpy had a warning in its log, but it built successfully:

building library "npymath" sources
gfortran: warning: couldn't understand version 11
gfortran: warning: couldn't understand version 11
gfortran: warning: couldn't understand version 11
gfortran: warning: couldn't understand version 11
gfortran: warning: couldn't understand version 11
gfortran: warning: couldn't understand version 11
gfortran: warning: couldn't understand version 11
gfortran: warning: couldn't understand version 11
jhpalmieri commented 3 years ago

Attachment: scipy-1.5.4.log

isuruf commented 3 years ago
comment:38

Is this with SYSTEM_COMPAT_VERSION=1 or MACOSX_DEPLOYMENT_TARGET=10.16. Are these set globally or in the spkg-install.in of scipy?

zlscherr commented 3 years ago
comment:39

I know that Homebrew fixes scipy by setting

MACOSX_DEPLOYMENT_TARGET=11.0

If the OS is big sur.

jhpalmieri commented 3 years ago
comment:40

This is with SYSTEM_COMPAT_VERSION=1 set globally.

zlscherr commented 3 years ago
comment:41

to elaborate on my previous comment, if I run

make scipy

then it fails. However, if I run

MACOSX_DEPLOYMENT_TARGET=11.0 make scipy

then it succeeds with no issues. I don't know too much about makefiles but a simple fix could be to set MACOSX_DEPLOYMENT_TARGET if the operating system is Big Sur and then unset it after scipy installs.

jhpalmieri commented 3 years ago
comment:42

I can get Sage to build on OS X 11.1 with the following changes:

return tuple(map(int, get_config_var("MACOSX_DEPLOYMENT_TARGET").split('.')))

The problem is that get_config_var(...) returns the int 11 rather than the string "11", so I patched it to do str(get_config_var(...)). Similar with numpy.

I did not set any global environment variables.

There were some doctest failures, I think all due to gfortran. For example:

File "src/sage/misc/inline_fortran.py", line 114, in sage.misc.inline_fortran._import_module_from_path_impl
Failed example:
    fortran(code, globals())
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/misc/inline_fortran.py", line 170, in eval
        out = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
      File "/usr/local/Cellar/python@3.9/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 420, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      File "/usr/local/Cellar/python@3.9/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 524, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/bin/python3', '-c', 'import numpy.f2py; numpy.f2py.main()', '-c', '-m', 'fortran_module', 'fortran_module.f', '--quiet', '--f77exec=sage-inline-fortran', '--f90exec=sage-inline-fortran']' returned non-zero exit status 1.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.inline_fortran._import_module_from_path_impl[1]>", line 1, in <module>
        fortran(code, globals())
      File "sage/misc/lazy_import.pyx", line 360, in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:4032)
        return self.get_object()(*args, **kwds)
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/misc/inline_fortran.py", line 76, in __call__
        return self.eval(*args, **kwds)
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/misc/inline_fortran.py", line 172, in eval
        raise RuntimeError(
    RuntimeError: failed to compile Fortran code:
    b'gfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\ngfortran: warning: couldn\'t understand version 11\nerror: Command "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/bin/sage-inline-fortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib /var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/src.macosx-11-x86_64-3.9/fortran_modulemodule.o /var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/src.macosx-11-x86_64-3.9/fortranobject.o /var/folders/z6/yjw_7s357yx3_mhh81__lplc0000gn/T/tmpr718avny/fortran_module.o -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -lgfortran -o ./fortran_module.cpython-39-darwin.so" failed with exit status 1\n'
**********************************************************************
File "src/sage/misc/inline_fortran.py", line 117, in sage.misc.inline_fortran._import_module_from_path_impl
Failed example:
    fib(a, 10)
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.inline_fortran._import_module_from_path_impl[4]>", line 1, in <module>
        fib(a, Integer(10))
    NameError: name 'fib' is not defined
**********************************************************************
File "src/sage/misc/inline_fortran.py", line 118, in sage.misc.inline_fortran._import_module_from_path_impl
Failed example:
    a
Expected:
    array([  0.,   1.,   1.,   2.,   3.,   5.,   8.,  13.,  21.,  34.])
Got:
    array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
**********************************************************************
1 item had failures:
   3 of  10 in sage.misc.inline_fortran._import_module_from_path_impl
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -5,8 +5,8 @@
 - #30745 homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local
 - #31050 upgrade wheel
 - `MACOSX_DEPLOYMENT_TARGET` problems - https://groups.google.com/g/sage-devel/c/Sj3YBVJoHQM/m/Zt-Ih_hJAwAJ
+- #31128 MR50: Upgrade cffi to 1.14.4
 - #30589 Upgrade Python to 3.9.1
-

 Related:
mkoeppe commented 3 years ago
comment:44

Replying to @jhpalmieri:

  • upgrade setuptools and setuptools_scm (not sure if this is necessary)

Could you open a ticket for this?

jhpalmieri commented 3 years ago
comment:45

Replying to @mkoeppe:

Replying to @jhpalmieri:

  • upgrade setuptools and setuptools_scm (not sure if this is necessary)

Could you open a ticket for this?

Done at #31134.

jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -6,6 +6,7 @@
 - #31050 upgrade wheel
 - `MACOSX_DEPLOYMENT_TARGET` problems - https://groups.google.com/g/sage-devel/c/Sj3YBVJoHQM/m/Zt-Ih_hJAwAJ
 - #31128 MR50: Upgrade cffi to 1.14.4
+- #31134: upgrade setuptools and setuptools_scm
 - #30589 Upgrade Python to 3.9.1

 Related: