build fails due to missing noexcepts - Cython 3!

[dimpase]

see https://groups.google.com/g/sage-support/c/K4v7jBCAOfs/m/C4NdP6UIBgAJ and https://github.com/sagemath/sage-numerical-backends-coin/issues/9

[dimpase]

I don't have gurobi, so this needs testing

[ypfmde]

I'm not sure how to test it, probably I should read the developer guide :-). The only place where I find the file gurobi_backend.pyx is in the directory SAGE_ROOT/local/var/lib/sage/venv-python3.11/var/tmp/sage/build/sage_numerical_backends_gurobi-9.3.1/src/sage_numerical_backends_gurobi. If I replace it with the provided version and do a make build afterwards, building fails and gurobi_backend.pyx is back to the old version (no idea where it comes from?!).

[dimpase]

1. create sage_numerical_backends_gurobi-...tar.gz (by doing the right thing with code here) and place it in Sage's upstream/ directory, so that it can be picked up
2. In Sage's dir, run make sage_numerical_backends_gurobi to install in in Sage
3. test

[dimpase]

I can try to create a release here, so that the resulting tarball can be placed in Sage's upstream/. Will it help?

[ypfmde]

Hmm, I tried that, but probably wrong: I unpacked sage_numerical_backends_gurobi-...tar.gz, replaced gurobi_backend.pyx with the modified file, then packed and gzipped the directory back to sage_numerical_backends_gurobi-...tar.gz. However, after (an unsuccessful) make sage_numerical_backends_gurobi, the upstream/sage_numerical_backends_gurobi-...tar.gz file is back to the original one ...

[dimpase]

@ypfmde - installing from the checked out git tree should work. (it works for coin). That is, assuming sage in your PATH:

git clone git@github.com:sagemath/sage-numerical-backends-gurobi.git
cd sage-numerical-backends-gurobi
sage -python -m pip install .

assuming this succeeds, you can test etc. (sorry, had to edit out nonsense)

[ypfmde]

It works! (Tested for Sage version 10.2.rc2)

[dimpase]

did you try running tests?

[ypfmde]

Running ./sage -t build/pkgs/sage_numerical_backends_gurobi inside SAGE_ROOT results in "All tests passed!". I hope that's what has to be done. (Trying sage setup.py test inside sage-numerical-backends-gurobi after the git cloning and installation results in an error though. sage points to the correct Sage installation.)

[dimpase]

(Trying sage setup.py test inside sage-numerical-backends-gurobi after the git cloning and installation results in an error though. sage points to the correct Sage installation.)

The latter is probably fixed by 88bd06603a83065f7bea0fe97cc0417a8bb89fb2 Please git pull and try again.

Also, please try sage setup.py check_sage_testsuite

[ypfmde]

Still does not work (could also be caused by some messy setup of my machines):

Using gurobi_include_directories=['/home/mueller/local/gurobi/linux64/include'], libraries=['gurobi100'], library_dirs=['/home/mueller/local/gurobi/linux64/lib'] Checking whether HAVE_SAGE_CPYTHON_STRING... Checking whether HAVE_ADD_COL_UNTYPED_ARGS... Using compile_time_env: {'HAVE_SAGE_CPYTHON_STRING': True, 'HAVE_ADD_COL_UNTYPED_ARGS': True} running test WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox. /home/mueller/local/devel/sage/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/setuptools/command/test.py:193: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !!

    ********************************************************************************
    Requirements should be satisfied by a PEP 517 installer.
    If you are using pip, you can try `pip install --use-pep517`.
    ********************************************************************************

!! ir_d = dist.fetch_build_eggs(dist.install_requires) /home/mueller/local/devel/sage/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/setuptools/command/test.py:194: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !!

    ********************************************************************************
    Requirements should be satisfied by a PEP 517 installer.
    If you are using pip, you can try `pip install --use-pep517`.
    ********************************************************************************

!! tr_d = dist.fetch_build_eggs(dist.tests_require or []) /home/mueller/local/devel/sage/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/setuptools/command/test.py:195: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !!

    ********************************************************************************
    Requirements should be satisfied by a PEP 517 installer.
    If you are using pip, you can try `pip install --use-pep517`.
    ********************************************************************************

!! er_d = dist.fetch_build_eggs( running egg_info writing sage_numerical_backends_gurobi.egg-info/PKG-INFO writing dependency_links to sage_numerical_backends_gurobi.egg-info/dependency_links.txt writing requirements to sage_numerical_backends_gurobi.egg-info/requires.txt writing top-level names to sage_numerical_backends_gurobi.egg-info/top_level.txt dependency /home/mueller/local/devel/sage/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/cysignals/macros.h won't be automatically included in the manifest: the path must be relative dependency /home/mueller/local/devel/sage/local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/cysignals/struct_signals.h won't be automatically included in the manifest: the path must be relative dependency /home/mueller/local/devel/sage/src/sage/cpython/string_impl.h won't be automatically included in the manifest: the path must be relative dependency /home/mueller/local/gurobi/linux64/include/gurobi_c.h won't be automatically included in the manifest: the path must be relative dependency /usr/include/python3.11/Python.h won't be automatically included in the manifest: the path must be relative reading manifest template 'MANIFEST.in' warning: no previously-included files found matching '.c' warning: no previously-included files found matching '.cpp' warning: no previously-included files found matching '/.cpp' adding license file 'LICENSE' writing manifest file 'sage_numerical_backends_gurobi.egg-info/SOURCES.txt' running build_ext Traceback (most recent call last): File "/home/mueller/sage-numerical-backends-gurobi/check_license.py", line 4, in from sage.numerical.mip import MIPSolverException File "sage/numerical/mip.pyx", line 238, in init sage.numerical.mip File "sage/rings/integer_ring.pyx", line 1, in init sage.rings.integer_ring File "sage/rings/integer.pyx", line 1, in init sage.rings.integer File "sage/rings/rational.pyx", line 77, in init sage.rings.rational ImportError: cannot import name ZZ check_license.py returned error 256

[dimpase]

Running ./sage -t build/pkgs/sage_numerical_backends_gurobi inside SAGE_ROOT results in "All tests passed!". I hope that's what has to be done. (Trying sage setup.py test inside sage-numerical-backends-gurobi after the git cloning and installation results in an error though. sage points to the correct Sage installation.)

Well, no. After installing the backend with sage --python -m pip install . in the same directory you can run

sage -t sage_numerical_backends_gurobi/

This should return

...
Doctesting 3 files.
sage -t --warn-long 53.9 --random-seed=1897633607503523310248208778802730065 sage_numerical_backends_gurobi/gurobi_backend.pxd
    [0 tests, 0.00 s]
sage -t --warn-long 53.9 --random-seed=1897633607503523310248208778802730065 sage_numerical_backends_gurobi/gurobi_backend.pyx
    [216 tests, 0.22 s]
sage -t --warn-long 53.9 --random-seed=1897633607503523310248208778802730065 sage_numerical_backends_gurobi/__init__.py
    [0 tests, 0.00 s]
----------------------------------------------------------------------
All tests passed!
...

Anyhow, I went to getting myself a Gurobi licence and trying this; it works (I just pushed a commit to fix one doctest, but it's minor)

[dimpase]

You can also test installed Gurobi backend in SAGE_ROOT, by running

./sage -tp src/sage/numerical/

which works for me. What doesn't quite work is the only other place that tests with Gurobi (if available):

sage -t --warn-long 53.9 --random-seed=148439558831923248819587417992715184495 src/sage/combinat/matrices/dancing_links.pyx
**********************************************************************
File "src/sage/combinat/matrices/dancing_links.pyx", line 1038, in sage.combinat.matrices.dancing_links.dancing_linksWrapper.to_milp
Failed example:
    p.show()                                                              # needs sage.numerical.mip
Expected:
    Maximization:
    <BLANKLINE>
    <BLANKLINE>
    Constraints:...
      one 1 in 0-th column: 1.0 <= x_0 + x_1 <= 1.0
      one 1 in 1-th column: 1.0 <= x_0 + x_2 <= 1.0
      one 1 in 2-th column: 1.0 <= x_0 + x_1 <= 1.0
      one 1 in 3-th column: 1.0 <= x_3 <= 1.0
    Variables:
      x_0 is a boolean variable (min=0.0, max=1.0)
      x_1 is a boolean variable (min=0.0, max=1.0)
      x_2 is a boolean variable (min=0.0, max=1.0)
      x_3 is a boolean variable (min=0.0, max=1.0)
Got:
    Maximization:
    <BLANKLINE>
    <BLANKLINE>
    Constraints:
      one 1 in 0-th column: 1.0 <= x_ + x_ <= 1.0
      one 1 in 1-th column: 1.0 <= x_ + x_ <= 1.0
      one 1 in 2-th column: 1.0 <= x_ + x_ <= 1.0
      one 1 in 3-th column: 1.0 <= x_ <= 1.0
    Variables:
      x_ = x_0 is a boolean variable (min=0.0, max=1.0)
      x_ = x_1 is a boolean variable (min=0.0, max=1.0)
      x_ = x_2 is a boolean variable (min=0.0, max=1.0)
      x_ = x_3 is a boolean variable (min=0.0, max=1.0)
**********************************************************************
1 item had failures:
   1 of   8 in sage.combinat.matrices.dancing_links.dancing_linksWrapper.to_milp
    [244 tests, 1 failure, 4.10 s]
----------------------------------------------------------------------
sage -t --warn-long 53.9 --random-seed=148439558831923248819587417992715184495 src/sage/combinat/matrices/dancing_links.pyx  # 1 doctest failed

So something is weird with printing constraints (no index!):

...
one 1 in 3-th column: 1.0 <= x_ <= 1.0
...

[dimpase]

OK, at least this is sorted.

[ypfmde]

Yes, the test sage -t sage_numerical_backends_gurobi/ worked fine. Your other test ./sage -tp src/sage/numerical/ inside SAGE_ROOT passes everything for me. But I have the slightly older gurobi version 10.0.0.

[dimpase]

Yes, the test sage -t sage_numerical_backends_gurobi/ worked fine. Your other test ./sage -tp src/sage/numerical/ inside SAGE_ROOT passes everything for me. But I have the slightly older gurobi version 10.0.0.

the "other test" is sage -t src/sage/combinat/matrices/dancing_links.pyx. Does it pass for you?

[ypfmde]

No, it doesn't:

Running doctests with ID 2023-11-15-23-41-24-722a6cae.
Git branch: develop
Git ref: 10.2.rc3
Running with SAGE_LOCAL='/home/mueller/local/devel/sage/local' and SAGE_VENV='/home/mueller/local/devel/sage/local/var/lib/sage/venv-python3.11'
Using --optional=arch,bliss,cbc,csdp,dsdp,gap_packages,libsemigroups,mcqd,msolve,papilo,pip,sage,sage_spkg,scip,scip_sdp,soplex
Features to be detected: 4ti2,benzene,bliss,buckygen,conway_polynomials,csdp,cvxopt,cvxopt,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_cubic_hecke,database_jones_numfield,database_knotinfo,dvipng,fpylll,gap_package_atlasrep,gap_package_design,gap_package_grape,gap_package_guava,gap_package_hap,gap_package_polycyclic,gap_package_qpa,gap_package_quagroup,gfan,graphviz,imagemagick,ipython,jupymake,kenzo,latte_int,lrcalc_python,lrslib,mcqd,meataxe,mpmath,msolve,nauty,networkx,numpy,palp,pandoc,pdf2svg,pdftocairo,pexpect,phitigra,pillow,plantri,polytopes_db,polytopes_db_4d,pplpy,primecountpy,ptyprocess,pynormaliz,pyparsing,python_igraph,requests,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.ecl,sage.libs.flint,sage.libs.gap,sage.libs.linbox,sage.libs.m4ri,sage.libs.ntl,sage.libs.pari,sage.libs.singular,sage.misc.cython,sage.modular,sage.modules,sage.numerical.mip,sage.plot,sage.rings.complex_double,sage.rings.finite_rings,sage.rings.function_field,sage.rings.number_field,sage.rings.padics,sage.rings.polynomial.pbori,sage.rings.real_double,sage.rings.real_mpfr,sage.sat,sage.schemes,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,scipy,singular,sphinx,sympy,tdlib
Doctesting 1 file.
sage -t --random-seed=226504368971247626389933238662858079630 src/sage/combinat/matrices/dancing_links.pyx
**********************************************************************
File "src/sage/combinat/matrices/dancing_links.pyx", line 1038, in sage.combinat.matrices.dancing_links.dancing_linksWrapper.to_milp
Failed example:
    p.show()                                                              # needs sage.numerical.mip
Expected:
    Maximization:
    <BLANKLINE>
    <BLANKLINE>
    Constraints:...
      one 1 in 0-th column: 1.0 <= x_0 + x_1 <= 1.0
      one 1 in 1-th column: 1.0 <= x_0 + x_2 <= 1.0
      one 1 in 2-th column: 1.0 <= x_0 + x_1 <= 1.0
      one 1 in 3-th column: 1.0 <= x_3 <= 1.0
    Variables:
      x_0 is a boolean variable (min=0.0, max=1.0)
      x_1 is a boolean variable (min=0.0, max=1.0)
      x_2 is a boolean variable (min=0.0, max=1.0)
      x_3 is a boolean variable (min=0.0, max=1.0)
Got:
    Maximization:
    <BLANKLINE>
    <BLANKLINE>
    Constraints:
      one 1 in 0-th column: 1.0 <= x_ + x_ <= 1.0
      one 1 in 1-th column: 1.0 <= x_ + x_ <= 1.0
      one 1 in 2-th column: 1.0 <= x_ + x_ <= 1.0
      one 1 in 3-th column: 1.0 <= x_ <= 1.0
    Variables:
      x_ = x_0 is a boolean variable (min=0.0, max=1.0)
      x_ = x_1 is a boolean variable (min=0.0, max=1.0)
      x_ = x_2 is a boolean variable (min=0.0, max=1.0)
      x_ = x_3 is a boolean variable (min=0.0, max=1.0)
**********************************************************************
1 item had failures:
   1 of   8 in sage.combinat.matrices.dancing_links.dancing_linksWrapper.to_milp
    [244 tests, 1 failure, 3.76 s]
----------------------------------------------------------------------
sage -t --random-seed=226504368971247626389933238662858079630 src/sage/combinat/matrices/dancing_links.pyx  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 3.9 seconds
    cpu time: 3.0 seconds
    cumulative wall time: 3.8 seconds
Features detected for doctesting: sage.numerical.mip
pytest is not installed in the venv, skip checking tests that rely on it```