local-conda-forge-ubuntu-standard: cvxopt build fails

[mkoeppe]

https://github.com/mkoeppe/sage/runs/1201631449

  building 'umfpack' extension
  /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-standard/conda/bin/x86_64-conda-linux-gnu-cc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-standard/conda/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-standard/conda/include -fPIC -I/usr/include -I/home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-standard/local/include -I/home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-standard/conda/include/python3.8 -c src/C/umfpack.c -o build/temp.linux-x86_64-3.8/src/C/umfpack.o
  In file included from /home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-standard/conda/include/python3.8/Python.h:11:0,
                   from src/C/cvxopt.h:22,
                   from src/C/umfpack.c:22:
  /usr/include/limits.h:26:10: fatal error: bits/libc-header-start.h: No such file or directory
   #include <bits/libc-header-start.h>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command '/home/runner/work/sage/sage/.tox/local-conda-forge-ubuntu-standard/conda/bin/x86_64-conda-linux-gnu-cc' failed with exit status 1
  Building wheel for cvxopt (setup.py): finished with status 'error'
  ERROR: Failed building wheel for cvxopt
  Running setup.py clean for cvxopt

We adjust cvxopt's install script, last changed in #31584

CC: @dimpase @isuruf

Component: packages: standard

Author: Matthias Koeppe

Branch/Commit: u/mkoeppe/local_conda_forge_ubuntu_standard__cvxopt_build_fails @ b2a8d0a

Reviewer: https://github.com/mkoeppe/sage/runs/7666787900

Issue created by migration from https://trac.sagemath.org/ticket/30710

[isuruf]

comment:2

Can you print the values of, https://github.com/sagemath/sage-prod/blob/master/build/pkgs/cvxopt/spkg-install.in#L20-L33

[isuruf]

comment:4

Issue is that somewhere, some code is adding /usr/include to include directories and that conflicts with conda's sysroot.

[dimpase]

comment:5

Replying to @isuruf:

Issue is that somewhere, some code is adding /usr/include to include directories and that conflicts with conda's sysroot.

isn't this done by cvxopt's setup.py ? Probably it needs a patch to special-case conda. (Sage does a bit of patching of this file.)

[mkoeppe]

comment:6

We can try if upgrading cvxopt (#31467) helps

[mkoeppe]

comment:7

Moving to 9.4, as 9.3 has been released.

[fchapoton]

comment:9

bump to 9.6

[mkoeppe]

comment:11

Same after upgrade to cvxopt 1.3.0 in #34150 (https://github.com/mkoeppe/sage/runs/7646124113)

[isuruf]

comment:12

The lines at https://github.com/sagemath/sage-prod/blob/master/build/pkgs/cvxopt/spkg-install.in#L68-L71 should be changed to

export CVXOPT_SUITESPARSE_LIB_DIR="${SAGE_LOCAL}/lib"
export CVXOPT_SUITESPARSE_INC_DIR="${SAGE_LOCAL}/include"

even when suitesparse is coming from the system, so that setup.py doesn't add its default /usr/include.

[mkoeppe]

comment:13

Yes, good idea.

In #31584 comment:10, I wrote:

We do not need to do this when we use system suitesparse, as we do not want to claim authority that our configure script's suitesparse-finding technique is better than the one in cvxopt's install script.

But as it turns out, our technique (= just relying on standard compiler/linker paths) is better.

[mkoeppe]

Branch: u/mkoeppe/local_conda_forge_ubuntu_standard__cvxopt_build_fails

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Commit: 5d3902e

[mkoeppe]

New commits:

5d3902e

build/pkgs/cvxopt/spkg-install.in: Unconditionally set CVXOPT_SUITESPARSE_LIB_DIR, CVXOPT_SUITESPARSE_INC_DIR

[isuruf]

comment:16

Why export CVXOPT_SUITESPARSE_LIB_DIR="${SAGE_LOCAL}" instead of export CVXOPT_SUITESPARSE_LIB_DIR="${SAGE_LOCAL}/lib"?

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 5d3902e to b2a8d0a

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

b2a8d0a

build/pkgs/cvxopt/spkg-install.in: Fixup

[mkoeppe]

comment:18

Thanks. This made no sense. I've fixed the line below for GLPK, which made the same error. In the end it does not matter what we put there -- this directory is in the library search path anyway.

[isuruf]

comment:19

Looks good to me. Can you run the github actions for this?

[mkoeppe]

Reviewer: https://github.com/mkoeppe/sage/runs/7666787900

[mkoeppe]

comment:21

This fixed it but it looks like it broke other platforms, including ubuntu-jammy

  gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib -g -O2 build/temp.linux-x86_64-cpython-310/src/C/lapack.o -L/usr/lib/x86_64-linux-gnu/openblas-pthread/ -L/usr/lib/x86_64-linux-gnu -lopenblas -lopenblas -o build/lib.linux-x86_64-cpython-310/cvxopt/lapack.cpython-310-x86_64-linux-gnu.so
  building 'umfpack' extension
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -O2 -fPIC -I/sage/local/include -I/sage/local/var/lib/sage/venv-python3.10/include -I/usr/include/python3.10 -c src/C/umfpack.c -o build/temp.linux-x86_64-cpython-310/src/C/umfpack.o
  src/C/umfpack.c:23:10: fatal error: umfpack.h: No such file or directory
     23 | #include "umfpack.h"
        |          ^~~~~~~~~~~
  compilation terminated.

https://github.com/mkoeppe/sage/runs/7682737008?check_suite_focus=true

[mkoeppe]

Description changed:

--- 
+++ 
@@ -16,4 +16,4 @@
   Running setup.py clean for cvxopt

- +We adjust cvxopt's install script, last changed in #31584

[mkoeppe]

comment:24

Related: #31905