search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.32k stars 453 forks source link

Meta-ticket: Support Xcode 12 #30494

Closed mkoeppe closed 1 year ago

mkoeppe commented 4 years ago

Xcode 12 was publicly released on 2020-09-16.

Sage does not build from source using Xcode 12.2 beta, released 2020.09.17, clang version 12.0.0 (clang-1200.0.32.4)

Failing packages (blockers):

Failing packages with replacements available via homebrew:

Failing optional packages:

Failing experimental packages:

Tickets:

Follow-up:

CC: @jhpalmieri @dimpase @slel @vbraun

Component: porting

Keywords: macOS, Xcode

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/30494

jhpalmieri commented 3 years ago
comment:45

openblas: same kind of issue

lapacke_cggsvp_work.c:52:9: error: implicit declaration of function 'cggsvp_' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        LAPACK_cggsvp( &jobu, &jobv, &jobq, &m, &p, &n, a, &lda, b, &ldb, &tola,
        ^
jhpalmieri commented 3 years ago
comment:46

And readline, too:

../terminal.c:242:7: error: implicit declaration of function 'ioctl' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  if (ioctl (tty, TIOCGWINSZ, &window_size) == 0)
      ^
jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -11,6 +11,7 @@
 - ecl #30594
 - scipy
 - openblas
+- readline

 Tickets: 
 - #30487 - GH Actions: Test also with Xcode 12 beta
jhpalmieri commented 3 years ago
comment:47

Upgrading to readline 8.0 worked for me. Is there a ticket for that?

jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -11,7 +11,7 @@
 - ecl #30594
 - scipy
 - openblas
-- readline
+- readline #30603

 Tickets: 
 - #30487 - GH Actions: Test also with Xcode 12 beta
jhpalmieri commented 3 years ago
comment:48

I opened #30603 for readline.

mkoeppe commented 3 years ago
comment:49

openblas - see https://github.com/xianyi/OpenBLAS/issues/2805

mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -1,3 +1,5 @@
+Xcode 12 was publicly released on 2020-09-16. 
+
 Sage does not build from source using Xcode 12.2 beta, released 2020.09.17, clang version 12.0.0 (clang-1200.0.32.4)

 - https://groups.google.com/d/msg/sage-support/ZLgu-1pi6nc/W7Hqe7AgAgAJ
dimpase commented 3 years ago
comment:51

I have a patch for scipy that makes it build - posting on #30604 in a moment.

dimpase commented 3 years ago

Description changed:

--- 
+++ 
@@ -11,7 +11,7 @@
 - symmetrica
 - rubiks (see also #28022)
 - ecl #30594
-- scipy
+- scipy #30604
 - openblas
 - readline #30603
dimpase commented 3 years ago
comment:52

Replying to @mkoeppe:

openblas - see https://github.com/xianyi/OpenBLAS/issues/2805

I am using openblas from Homebrew, it works with XCode 12 (on macOS 10), no?

jhpalmieri commented 3 years ago
comment:53

Replying to @mkoeppe:

openblas - see https://github.com/xianyi/OpenBLAS/issues/2805

With that patch I don't get the "implicit declaration" errors, but instead I get errors like

lapacke_dggsvp_work.c:49:71: error: too many arguments to function call, expected 21, have 24
                       &tolb, k, l, u, &ldu, v, &ldv, q, &ldq, iwork, tau, work,
                                                                      ^~~~~~~~~~
jhpalmieri commented 3 years ago
comment:54

I tried upgrading to OpenBLAS 0.3.10, but I get the same problem.

jhpalmieri commented 3 years ago
comment:55

Replying to @dimpase:

Replying to @mkoeppe:

openblas - see https://github.com/xianyi/OpenBLAS/issues/2805

I am using openblas from Homebrew, it works with XCode 12 (on macOS 10), no?

Yes, that works fine for me. Same for readline, but both of those were reported as failures on sage-release.

mkoeppe commented 3 years ago
comment:56

Perhaps the Xcode 12 tickets for packages for which we can use working homebrew packages should be considered "critical", not "blockers" - but they need to be fixed too.

jhpalmieri commented 3 years ago
comment:57

Sure, sounds good.

jhpalmieri commented 3 years ago
comment:58

I think we're just down to symmetrica now, combined with a reasonable set of homebrew packages. I don't know enough (read: any) C to be able to do anything meaningful with symmetrica, though.

dimpase commented 3 years ago
comment:59

I'll fix symmetrica, no worries

dimpase commented 3 years ago

Description changed:

--- 
+++ 
@@ -8,7 +8,7 @@
 Failing packages:
 - gf2x #30593
 - ecm #30600
-- symmetrica
+- symmetrica #30608
 - rubiks (see also #28022)
 - ecl #30594
 - scipy #30604
jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -12,7 +12,7 @@
 - rubiks (see also #28022)
 - ecl #30594
 - scipy #30604
-- openblas
+- openblas #30610
 - readline #30603

 Tickets:
jhpalmieri commented 3 years ago
comment:61

I opened #30610 for OpenBLAS, not that I know how to fix it.

jhpalmieri commented 3 years ago
comment:62

mpir has a similar issue: its configure script fails because of the same sort of error. Again, the homebrew package works just fine.

jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -14,6 +14,7 @@
 - scipy #30604
 - openblas #30610
 - readline #30603
+- mpir

 Tickets: 
 - #30487 - GH Actions: Test also with Xcode 12 beta
jhpalmieri commented 3 years ago
comment:63

python3 builds with Xcode 12 on OS X 10.15.6, which is nice. It failed the last time I tried it with OS X 11, but maybe that's just a MACOSX_DEPLOYMENT_TARGET misconfiguration. It can be dealt with later, in any case.

jhpalmieri commented 3 years ago
comment:64

gsl fails. Its config.log says

configure:3504: gcc -g -O2   -L/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/new-xcode/sage-9.2.beta12/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/new-xcode/sage-9.2.beta12/local/lib  conftest.c -lopenblas -lm >&5
ld: library not found for -lopenblas
clang: error: linker command failed with exit code 1 (use -v to see invocation)
configure:3508: $? = 1
configure:3546: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "gsl"
| #define PACKAGE_TARNAME "gsl"
| #define PACKAGE_VERSION "2.6"
| #define PACKAGE_STRING "gsl 2.6"
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| #define PACKAGE "gsl"
| #define VERSION "2.6"
| #define RELEASED /**/
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:3551: error: in `/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/new-xcode/sage-9.2.beta12/local/var/tmp/sage/build/gsl-2.6/src':
configure:3553: error: C compiler cannot create executables
See `config.log' for more details

(This is the only config.log file around, so I'm not sure what it's talking about.) Maybe the homebrew installations of openblas and gsl are confusing it. To be honest, I'm not sure that Sage's gsl built with the homebrew openblas before.

jhpalmieri commented 3 years ago
comment:65

r also fails, with some familiar errors and an unexpected one:

conftest.c:250:11: warning: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
    char *ver = BZ2_bzlibVersion();
          ^     ~~~~~~~~~~~~~~~~~~
conftest.c:251:5: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
    exit(strcmp(ver, "1.0.6") < 0);
    ^
conftest.c:251:5: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
conftest.c:251:10: error: implicitly declaring library function 'strcmp' with type 'int (const char *, const char *)' [-Werror,-Wimplicit-function-declaration]
    exit(strcmp(ver, "1.0.6") < 0);
         ^
conftest.c:251:10: note: include the header <string.h> or explicitly provide a declaration for 'strcmp'

and then

configure:42275: result: no
configure:42281: checking whether bzip2 support suffices
configure:42288: error: bzip2 library and headers are required

(This is using the system bzip2 installation.)

jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -15,6 +15,8 @@
 - openblas #30610
 - readline #30603
 - mpir
+- gsl (?)
+- r

 Tickets: 
 - #30487 - GH Actions: Test also with Xcode 12 beta
jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -5,19 +5,24 @@
 - https://groups.google.com/d/msg/sage-support/ZLgu-1pi6nc/W7Hqe7AgAgAJ
 - https://github.com/mkoeppe/sage/runs/1059961873

-Failing packages:
+Failing packages (blockers):
 - gf2x #30593
 - ecm #30600
 - symmetrica #30608
-- rubiks (see also #28022)
 - ecl #30594
 - scipy #30604
+
+Failing packages with replacements available via homebrew:
 - openblas #30610
 - readline #30603
 - mpir
 - gsl (?)
 - r

+Failing optional packages:
+- rubiks (see also #28022)
+
+
 Tickets: 
 - #30487 - GH Actions: Test also with Xcode 12 beta
 - #27754 - Upgrade: Python 3.8.5
dimpase commented 3 years ago

Description changed:

--- 
+++ 
@@ -8,7 +8,7 @@
 Failing packages (blockers):
 - gf2x #30593
 - ecm #30600
-- symmetrica #30608
+- symmetrica #30608 or #29061
 - ecl #30594
 - scipy #30604
jhpalmieri commented 3 years ago
comment:69

There is a problem with another package, but I can't figure out which. If I run

$ ./configure --with-system-boost_cropped=no --with-system-curl=yes --with-system-freetype=no --with-system-glpk=no --with-system-gmp=no --with-system-libffi=no --with-system-libgd=no --with-system-libpng=no --with-system-mpc=no --with-system-mpfi=no --with-system-mpfr=no --with-system-ninja_build=no --with-system-ntl=no --with-system-pkgconf=no --with-system-ppl=no --with-system-python3=no --with-system-sqlite=no --with-system-suitesparse=no --with-system-yasm=no --with-system-zeromq=no --with-system-zlib

then Sage builds, but the docs fail with this:

Warning: Missing title for sage.combinat.posets.mobile
[combinat ] building [inventory]: targets for 378 source files that are out of date
[combinat ] updating environment: [new config] 378 added, 0 changed, 0 removed
------------------------------------------------------------------------
0   signals.cpython-38-darwin.so        0x000000011139862a print_backtrace + 58
1   signals.cpython-38-darwin.so        0x000000011139c277 sigdie + 39
2   signals.cpython-38-darwin.so        0x000000011139c1f0 sigdie_for_sig + 256
3   libsystem_platform.dylib            0x00007fff6a9965fd _sigtramp + 29
4   ???                                 0xf687400000000001 0x0 + 17764237623930388481
5   libsystem_c.dylib                   0x00007fff6a86c808 abort + 120
6   libsystem_malloc.dylib              0x00007fff6a96250b has_default_zone0 + 0
7   libsystem_malloc.dylib              0x00007fff6a96540f malloc_report + 151
8   memory.cpython-38-darwin.so         0x000000033a29b144 __pyx_f_4sage_3ext_6memory_sage_sig_free + 20
9   polyhedron.cpython-38-darwin.so     0x000000034423eb71 _ZN23Parma_Polyhedra_Library10PolyhedronD2Ev + 49
10  polyhedron.cpython-38-darwin.so     0x000000034423d742 _ZL47__pyx_tp_dealloc_3ppl_10polyhedron_C_PolyhedronP7_object + 82
11  libpython3.8.dylib                  0x000000010f4fe76e dict_dealloc + 510
12  libpython3.8.dylib                  0x000000010f52f83c subtype_clear + 252
13  libpython3.8.dylib                  0x000000010f611c6b collect + 3563
14  libpython3.8.dylib                  0x000000010f612135 _PyObject_GC_Alloc + 389
15  libpython3.8.dylib                  0x000000010f6121a5 _PyObject_GC_New + 21
16  libpython3.8.dylib                  0x000000010f5ff9ab PyTraceBack_Here + 155
17  category_object.cpython-38-darwin.s 0x000000011246d177 __Pyx_AddTraceback + 839
18  category_object.cpython-38-darwin.s 0x0000000112472795 __pyx_f_4sage_9structure_15category_object_14CategoryObject_getattr_from_category + 453
19  category_object.cpython-38-darwin.s 0x0000000112472f90 __pyx_tp_getattro_4sage_9structure_15category_object_CategoryObject + 64
20  cachefunc.cpython-38-darwin.so      0x0000000112272703 __pyx_pw_4sage_4misc_9cachefunc_24CachedMethodCallerNoArgs_1__init__ + 2275
21  libpython3.8.dylib                  0x000000010f520774 type_call + 292
22  cachefunc.cpython-38-darwin.so      0x0000000112250d71 __Pyx_PyObject_Call + 97
23  cachefunc.cpython-38-darwin.so      0x000000011226784b __pyx_tp_descr_get_4sage_4misc_9cachefunc_CachedMethod + 1579
24  libpython3.8.dylib                  0x000000010f50e527 _PyObject_GenericGetAttrWithDict + 487
25  category_object.cpython-38-darwin.s 0x0000000112472f62 __pyx_tp_getattro_4sage_9structure_15category_object_CategoryObject + 18
26  libpython3.8.dylib                  0x000000010f50e77c _PyObject_GetMethod + 268
27  libpython3.8.dylib                  0x000000010f5a7e91 _PyEval_EvalFrameDefault + 29489
28  libpython3.8.dylib                  0x000000010f4cbe2d function_code_fastcall + 237
29  libpython3.8.dylib                  0x000000010f4cb2aa _PyObject_FastCallDict + 218
30  libpython3.8.dylib                  0x000000010f4cc993 _PyObject_Call_Prepend + 131
31  libpython3.8.dylib                  0x000000010f525354 slot_tp_init + 180
32  libpython3.8.dylib                  0x000000010f520774 type_call + 292
33  libpython3.8.dylib                  0x000000010f4cb476 _PyObject_MakeTpCall + 374
34  libpython3.8.dylib                  0x000000010f5aba0c call_function + 652
35  libpython3.8.dylib                  0x000000010f5a806a _PyEval_EvalFrameDefault + 29962
...
[snip]
...
457 libpython3.8.dylib                  0x000000010f5ac967 _PyEval_EvalCodeWithName + 3287
458 libpython3.8.dylib                  0x000000010f4cbfad _PyFunction_Vectorcall + 253
459 libpython3.8.dylib                  0x000000010f4cb7c4 PyVectorcall_Call + 100
460 libpython3.8.dylib                  0x000000010f6107d3 pymain_run_module + 179
461 libpython3.8.dylib                  0x000000010f60fec8 Py_RunMain + 1416
462 libpython3.8.dylib                  0x000000010f6106b3 pymain_main + 403
463 libpython3.8.dylib                  0x000000010f61070b Py_BytesMain + 43
464 libdyld.dylib                       0x00007fff6a79dcc9 start + 1
465 ???                                 0x0000000000000006 0x0 + 6
------------------------------------------------------------------------
Unhandled SIGABRT: An abort() occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------

I've tried configuring with a subset of those packages specified, but I haven't been able to narrow it down.

mkoeppe commented 3 years ago
comment:70

There's some ppl stuff in the backtrace, so that would be my first guess

jhpalmieri commented 3 years ago
comment:71

I saw that, too. I thought I'd checked it, but I'll try again. (Taking a working build and then doing ./sage -f ppl resulted in a successful build, but I'll try from scratch.)

jhpalmieri commented 3 years ago
comment:72

After running

./configure --with-system-glpk=no --with-system-gmp=no --with-system-ppl=no

sagelib fails to build, and there are lots of messages like this:

ld: illegal thread local variable reference to regular symbol __ZN3NTL20ZZXFac_InitNumPrimesE for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'g++' failed with exit status 1
mkoeppe commented 3 years ago
comment:73

This might be unrelated breakage coming from homebrew's ntl/flint packages - tracked at #29339/#27764

mkoeppe commented 3 years ago
comment:74

Best to remove these from the homebrew installation. In some situations these packages seem to leak into Sage even with --with-system...=no, as reported recently on sage-devel

jhpalmieri commented 3 years ago
comment:75

Okay, after removing homebrew's ntl, I see the failure in comment:69 with ./configure --with-system-glpk=no --with-system-gmp=no --with-system-ppl=no. For what it's worth, if I do make distclean, followed by the above configure command, followed by export SAGE_CHECK=yes and make ppl, then ppl passes its test suite.

jhpalmieri commented 3 years ago
comment:76

glpk pass its test suite, and gmp shows one failure:

../../test-driver: line 107: 32897 Abort trap: 6           "$@" > $log_file 2>&1
FAIL: t-sqrlo
mkoeppe commented 3 years ago

Reviewer: https://github.com/mkoeppe/sage/actions/runs/264223897

mkoeppe commented 3 years ago

Changed reviewer from https://github.com/mkoeppe/sage/actions/runs/264223897 to https://github.com/mkoeppe/sage/actions/runs/264540655

dimpase commented 3 years ago
comment:79

Replying to @jhpalmieri:

glpk pass its test suite, and gmp shows one failure:

../../test-driver: line 107: 32897 Abort trap: 6           "$@" > $log_file 2>&1
FAIL: t-sqrlo

Did you try with Sage's GMP 6.1.2? It's old. I tried building and testing GMP 6.2.0, all its tests pass. For the record, it's on kabylake-apple-darwin19.6.0, clang version 1200.0.32.4

This is the same GMP version as on Homebrew, so it should be OK to use the latter.

dimpase commented 3 years ago

Description changed:

--- 
+++ 
@@ -15,6 +15,7 @@
 Failing packages with replacements available via homebrew:
 - openblas #30610
 - readline #30603
+- gmp (#30625)
 - mpir
 - gsl (?)
 - r
dimpase commented 3 years ago
comment:81

I've opened #30625 to update GMP

jhpalmieri commented 3 years ago
comment:82

Replying to @dimpase:

Replying to @jhpalmieri:

glpk pass its test suite, and gmp shows one failure:

../../test-driver: line 107: 32897 Abort trap: 6           "$@" > $log_file 2>&1
FAIL: t-sqrlo

Did you try with Sage's GMP 6.1.2?

Yes: using Sage's gmp, glpk, and ppl leads to failures when building the documentation (and also doctest failures along the same lines): see comment:69. I was trying to see if the packages passed their test suites after seeming to install correctly.

dimpase commented 3 years ago
comment:83

I also see an error in a scipy selftest, related to Cython.

dimpase commented 3 years ago
comment:84

R does not build, as its bzlib2/bzip2 check fails. Indeed, the corr. C test has undeclared external functions

configure:42230: checking if bzip2 version >= 1.0.6
configure:42258: clang -o conftest  -g -O2  -fPIC -fPIC   -L/Users/dima/software/sagetrac-mirror/local/lib/R/lib -Wl,-rpath,/Users/dima/software/sagetrac-mirror/local/lib/R/lib -L/Users/dima/software/sage
trac-mirror/local/lib -Wl,-rpath,/Users/dima/software/sagetrac-mirror/local/lib  conftest.c -lbz2 -lz -licucore -ldl -lm  -liconv >&5
conftest.c:251:11: warning: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
    char *ver = BZ2_bzlibVersion();
          ^     ~~~~~~~~~~~~~~~~~~
conftest.c:252:5: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
    exit(strcmp(ver, "1.0.6") < 0);
    ^
conftest.c:252:5: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
conftest.c:252:10: error: implicitly declaring library function 'strcmp' with type 'int (const char *, const char *)' [-Werror,-Wimplicit-function-declaration]
    exit(strcmp(ver, "1.0.6") < 0);
         ^
conftest.c:252:10: note: include the header <string.h> or explicitly provide a declaration for 'strcmp'
1 warning and 2 errors generated.
dimpase commented 3 years ago
comment:85

gsl did build well with the patch for openblas from #30610

dimpase commented 3 years ago
comment:86

see #30631 for an R fix

dimpase commented 3 years ago

Description changed:

--- 
+++ 
@@ -15,10 +15,10 @@
 Failing packages with replacements available via homebrew:
 - openblas #30610
 - readline #30603
-- gmp (#30625)
+- gmp #30625
 - mpir
 - gsl (?)
-- r
+- r #30631

 Failing optional packages:
 - rubiks (see also #28022)
dimpase commented 3 years ago
comment:87

giac builds, but hangs on startup. And some giac-related tests time out too.

dimpase commented 3 years ago
comment:88

With beta13, #29061, #28623, #30486 (i.e. arb 2.18) and flint(2.6.1)+ntl+gmp+mpfr+mpc+mpfi+readline(and deps)+boost from Homebrew - but openblas, gsl, R built by Sage (with the updates), all the long tests pass. So this seems to be sorted, more or less.