upgrade Normaliz to 3.8.8, PyNormaliz to 2.12, add script package libnauty

[videlec]

Tarballs: see checksums.ini

These new releases include dynamic bitset classes that allow dynamic h-representation computations.

We also install nauty header and library in order that Normaliz compile with nauty support (for computing automorphism group).

Depends on #30262

Upstream: Fixed upstream, but not in a stable release.

CC: @mkoeppe @jplab @isuruf @w-bruns @tscrim @kliem @LaisRast @slel

Component: packages: standard

Keywords: package upgrade days101

Author: Vincent Delecroix, Jonathan Kliem, Matthias Koeppe

Branch: c55a701

Reviewer: Jonathan Kliem, Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/27952

[videlec]

New commits:

fdb67b5

upgrade normaliz to 3.7.3

[videlec]

Commit: fdb67b5

[videlec]

Branch: u/vdelecroix/normaliz

[videlec]

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
 Tarballs

 - Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
-- PyNormaliz: ?
+- PyNormaliz: https://files.pythonhosted.org/packages/b3/2a/6bf3717ff75f4b6dd8cbcba3dffdf75e65162c7aaad4ff95a64b4a768e7e/PyNormaliz-2.7.tar.gz

[videlec]

Changed keywords from none to package upgrade

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

7b7168b

upgrade normaliz to 3.7.3 and PyNormaliz to 2.7

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from fdb67b5 to 7b7168b

[videlec]

Author: Vincent Delecroix

[videlec]

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
 Tarballs

 - Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
-- PyNormaliz: https://files.pythonhosted.org/packages/b3/2a/6bf3717ff75f4b6dd8cbcba3dffdf75e65162c7aaad4ff95a64b4a768e7e/PyNormaliz-2.7.tar.gz
+

[videlec]

Dependencies: #27920

[videlec]

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
 Tarballs

 - Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
-
+- PyNormaliz: https://files.pythonhosted.org/packages/6b/92/f0074f58dc8a36529b374e3b18a7f21557bf113fb3480bb3d327d4fc5a2f/PyNormaliz-2.8.tar.gz

[videlec]

Description changed:

[videlec]

comment:9

Self tests do not pass on my computer

$ python3 setup.py test
/opt/sage/sage-py3-gcc/local/lib/python3.7/distutils/dist.py:274: UserWarning: Unknown distribution option: 'long_description_content_type'
  warnings.warn(msg)
running test
Doctest must_be_matrices-39.txt
  2 tests
Doctest test_rational_cones.txt
  34 tests
Doctest segfault-45.txt
  3 tests
Doctest autom.txt
Doctest failures

[videlec]

comment:10

And I confirm the segfault in a Python3 console

$ python3
Python 3.7.3 (default, May 26 2019, 12:00:16) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyNormaliz import Cone, NmzCone, NmzResult
>>> V = [[-1, -1, 1], [-1, 0, 1], [-1, 1, 1], [0, -1, 1], [0, 0, 1], [0, 1, 1], [1, -1, 1], [1, 0, 1], [1, 1, 1]]
>>> cube2 = NmzCone(vertices=V)
>>> NmzResult(cube2,"Automorphisms") == [8, [[[], []], []], [[[0, 2, 1, 3], [1, 0, 3, 2]], [[0, 1, 2, 3]]], [[[1, 0, 3, 2], [0, 2, 1, 3]], [[0, 1, 2, 3]]]]
Erreur de segmentation (core dumped)

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

72af741

normaliz 3.7.3 upgrade / PyNormaliz 2.8 upgrade

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 7b7168b to 72af741

[videlec]

Description changed:

--- 
+++ 
@@ -1,4 +1,5 @@
 Tarballs

+- e-antic: TO COME
 - Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
 - PyNormaliz: https://files.pythonhosted.org/packages/6b/92/f0074f58dc8a36529b374e3b18a7f21557bf113fb3480bb3d327d4fc5a2f/PyNormaliz-2.8.tar.gz

[videlec]

comment:12

segfault probably due to e-antic version. I am about to release 0.1.3

[videlec]

comment:13

Most probably, the segfault is due to the fact that Normaliz was not built with nauty support

checking whether nauty headers and library are available... checking for densenauty in -lnauty... no

[embray]

comment:14

Tickets still needing working or clarification should be moved to the next release milestone at the soonest (please feel free to revert if you think the ticket is close to being resolved).

[videlec]

Description changed:

--- 
+++ 
@@ -1,5 +1,4 @@
 Tarballs

-- e-antic: TO COME
 - Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
 - PyNormaliz: https://files.pythonhosted.org/packages/6b/92/f0074f58dc8a36529b374e3b18a7f21557bf113fb3480bb3d327d4fc5a2f/PyNormaliz-2.8.tar.gz

[videlec]

Description changed:

--- 
+++ 
@@ -2,3 +2,5 @@

 - Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
 - PyNormaliz: https://files.pythonhosted.org/packages/6b/92/f0074f58dc8a36529b374e3b18a7f21557bf113fb3480bb3d327d4fc5a2f/PyNormaliz-2.8.tar.gz
+
+We also install nauty header and library in order that Normaliz compile with nauty support (for computing automorphism group).

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

026e069	upgrade e-antic to 0.1.3
cef606b	install nauty header and library
36f640e	normaliz 3.7.3 upgrade / PyNormaliz 2.8 upgrade

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 72af741 to 36f640e

[videlec]

Changed dependencies from #27920 to #28006

[mkoeppe]

comment:19

What's this business with nauty.a? Shouldn't a shared library be built?

[videlec]

comment:20

If you suceed in doing so, please go on. Nauty makefile is extremly basic.

[videlec]

Changed keywords from package upgrade to package upgrade days101

[mkoeppe]

comment:22

Debian seems to have an autotoolized build system for nauty. https://sources.debian.org/patches/nauty/2.5r9+ds-1/

[videlec]

comment:23

They also do not build shared library as far a I see.

[mkoeppe]

comment:24

Their "upstream autotoolization.patch " has lib_LTLIBRARIES = libnauty.la ....

[videlec]

comment:25

Then you are welcome to port their patch to Sage and propose a patch to normaliz!

[jplab]

comment:27

FWIW: Normaliz 3.7.4 is out and fixes an f-vector issue (that came up with algebraic polytopes). It might be worth to grab this version right away and not wait for another ticket. This newer version should not be a quantum step...

[embray]

comment:28

Ticket retargeted after milestone closed

[mkoeppe]

comment:29

Also we need a version of e-antic that finds Debian's install name of arb, flint-arb. This seems to be in e-antic master but not in the 0.x branch

[mkoeppe]

comment:30

(see https://github.com/mkoeppe/sage/runs/430688755?check_suite_focus=true, where Sage recognizes libflint-arb but the e-antic install fails.)

[mkoeppe]

Description changed:

--- 
+++ 
@@ -4,3 +4,5 @@
 - PyNormaliz: https://files.pythonhosted.org/packages/6b/92/f0074f58dc8a36529b374e3b18a7f21557bf113fb3480bb3d327d4fc5a2f/PyNormaliz-2.8.tar.gz

 We also install nauty header and library in order that Normaliz compile with nauty support (for computing automorphism group).
+
+We also patch or upgrade e-antic so that it finds `arb` on Debian (https://github.com/videlec/e-antic/pull/82)

[mkoeppe]

comment:33

Replying to @mkoeppe:

Also we need a version of e-antic that finds Debian's install name of arb, flint-arb. This seems to be in e-antic master but not in the 0.x branch

Fixed in merged pull request https://github.com/videlec/e-antic/pull/82; now we only need a new release!

[mkoeppe]

comment:34

Needs to be merged with #28799.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -3,6 +3,10 @@
 - Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
 - PyNormaliz: https://files.pythonhosted.org/packages/6b/92/f0074f58dc8a36529b374e3b18a7f21557bf113fb3480bb3d327d4fc5a2f/PyNormaliz-2.8.tar.gz

+These new releases include dynamic bitset classes that allow dynamic h-representation computations.
+
 We also install nauty header and library in order that Normaliz compile with nauty support (for computing automorphism group).

-We also patch or upgrade e-antic so that it finds `arb` on Debian (https://github.com/videlec/e-antic/pull/82)
+We also patch e-antic so that it finds `arb` on Debian (https://github.com/videlec/e-antic/pull/82).
+
+

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,7 +1,7 @@
 Tarballs

-- Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.7.3/normaliz-3.7.3.tar.gz
-- PyNormaliz: https://files.pythonhosted.org/packages/6b/92/f0074f58dc8a36529b374e3b18a7f21557bf113fb3480bb3d327d4fc5a2f/PyNormaliz-2.8.tar.gz
+- Normaliz: https://github.com/Normaliz/Normaliz/releases/download/v3.8.4/normaliz-3.8.4.tar.gz
+- PyNormaliz: https://files.pythonhosted.org/packages/a6/4b/43fa5e9a993e683b960ade003918bb5fa2180932fc8ac39daea1364163a8/PyNormaliz-2.10.tar.gz

 These new releases include dynamic bitset classes that allow dynamic h-representation computations.

[mkoeppe]

Changed branch from u/vdelecroix/normaliz to u/mkoeppe/normaliz

[kliem]

comment:38

The challenge with upgrading Normaliz and PyNormaliz one basically has to rewrite _cone_from_Vrepresentation_and_Hrepresentation in backend_normaliz.py (and then pickling fails again), as initialization from precomputed data of normaliz cones works a bit different now.

I have mostly figured out how to do this, I think. So unless someone want to, I would like to take care of that part.

---

New commits:

b9b8222	install nauty header and library
b482276	normaliz 3.7.3 upgrade / PyNormaliz 2.8 upgrade
1860af0	Update Normaliz to 3.8.4, PyNormaliz to 2.10
2e1b0e8	normaliz: Remove dependency on boost_cropped
3543b73	Add e-antic patch

[kliem]

Changed commit from 36f640e to 3543b73

[mkoeppe]

comment:39

Replying to @kliem:

The challenge with upgrading Normaliz and PyNormaliz one basically has to rewrite _cone_from_Vrepresentation_and_Hrepresentation in backend_normaliz.py (and then pickling fails again), as initialization from precomputed data of normaliz cones works a bit different now.

I have mostly figured out how to do this, I think. So unless someone want to, I would like to take care of that part.

Sounds good.

[mkoeppe]

comment:40

I assume you are referring to these kinds of errors:

File "src/sage/geometry/polyhedron/backend_normaliz.py", line 675, in sage.geometry.polyhedron.backend_normaliz.Polyhedron_normaliz._cone_from_Vrepresentation_and_Hrepresentation
Failed example:
    cone = P._cone_from_Vrepresentation_and_Hrepresentation(P.vertices(),P.rays(),P.inequalities(),P.equations())   # optional - pynormaliz
Exception raised:
    Traceback (most recent call last):
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.geometry.polyhedron.backend_normaliz.Polyhedron_normaliz._cone_from_Vrepresentation_and_Hrepresentation[2]>", line 1, in <module>
        cone = P._cone_from_Vrepresentation_and_Hrepresentation(P.vertices(),P.rays(),P.inequalities(),P.equations())   # optional - pynormaliz
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_normaliz.py", line 790, in _cone_from_Vrepresentation_and_Hrepresentation
        return self._cone_from_normaliz_data(data, verbose=verbose)
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_normaliz.py", line 414, in _cone_from_normaliz_data
        cone = PyNormaliz.NmzCone(**data)
    PyNormaliz_cpp.NormalizError: Some error in the normaliz input data detected: Input type not allowed with precomputed data

[mkoeppe]

comment:41

I will not make further changes at this point. The branch needs the first part of commit 9dd22ef4f79228fdb9e0e012b98046349922d6fb (from #28799).

[mkoeppe]

Changed author from Vincent Delecroix to Vincent Delecroix, Matthias Koeppe

[kliem]

comment:43

@w-bruns: I'm having trouble with the new version of normaliz for the following reason:

When initializing a cone from precomputed data, it asks me to specify generated_lattice. However, sometimes we do not have access to this (e.g. when we want to change backends and have never done the computations in normaliz or when combinatorially obtaining the new double description is much cheaper).

How do I obtain generated_lattice from extreme_rays and maximal_subspace? (Or even as a cone property from any normaliz cone).

I'm puzzled for example how I could obtain the 2-dimensional permutahedron e.g.. The following doesn't seem to work:

sage: import PyNormaliz
sage: cone = PyNormaliz.NmzCone(
....:     dehomogenization=[[0, 0, 0, 1]],
....:     extreme_rays=[
....:         [1, 2, 3, 1], [1, 3, 2, 1], [2, 1, 3, 1],
....:         [2, 3, 1, 1], [3, 1, 2, 1], [3, 2, 1, 1]],
....:     generated_lattice=[[1, 0, -1, 0], [0, 1, -1, 0], [0, 0, 6, 1]],
....:     maximal_subspace=[],
....:     support_hyperplanes=[
....:         [-1, -1, 0, 5], [-1, 0, 0, 3], [0, -1, 0, 3],
....:         [0, 1, 0, -1], [1, 0, 0, -1], [1, 1, 0, -3]])

It initializes the empty cone for some reason. (At least Dimension of this thing is -1.)

How do I modify the generated lattice to make this work?

[mkoeppe]

Work Issues: patch e-antic configure