doctest killed due to abort in geometry/polyhedron/base.py

[seblabbe]

On a clean develop branch running SageMath version 9.1.beta9, Release Date: 2020-03-29, I get:

$ sage -t --long --memlimit=3600 src/sage/geometry/polyhedron/base.py 
too many failed tests, not using stored timings
Running doctests with ID 2020-04-01-21-12-24-30fbb892.
Git branch: develop
Using --optional=4ti2,build,cbc,ccache,cryptominisat,dochtml,dot2tex,e_antic,glucose,latte_int,lidia,lrslib,memlimit,normaliz,notedown,openssl,pandoc_attributes,pycosat,pynormaliz,rst2ipynb,sage,sage_numerical_backends_coin,sage_numerical_backends_cplex,sage_numerical_backends_gurobi
Doctesting 1 file.
sage -t --long src/sage/geometry/polyhedron/base.py
    [1485 tests, 26.12 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 26.4 seconds
    cpu time: 24.0 seconds
    cumulative wall time: 26.1 seconds

while

sage -t --long --memlimit=3500 src/sage/geometry/polyhedron/base.py 

or

sage -t --long src/sage/geometry/polyhedron/base.py

gives

Using --optional=4ti2,build,cbc,ccache,cryptominisat,dochtml,dot2tex,e_antic,glucose,latte_int,lidia,lrslib,memlimit,normaliz,notedown,openssl,pandoc_attributes,pycosat,pynormaliz,rst2ipynb,sage,sagenb
Doctesting 1 file.
sage -t --long src/sage/geometry/polyhedron/base.py
**********************************************************************
File "src/sage/geometry/polyhedron/base.py", line 7172, in sage.geometry.polyhedron.base.Polyhedron_base.integral_points
Failed example:
    P = 1/10*polytopes.hypercube(14)
Exception raised:
    Traceback (most recent call last):
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.geometry.polyhedron.base.Polyhedron_base.integral_points[12]>", line 1, in <module>
        P = Integer(1)/Integer(10)*polytopes.hypercube(Integer(14))
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/geometry/polyhedron/library.py", line 2749, in hypercube
        return Polyhedron(vertices=list(itertools.product([1, -1], repeat=dim)), base_ring=ZZ, backend=backend)
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/geometry/polyhedron/constructor.py", line 626, in Polyhedron
        return parent(Vrep, Hrep, convert=convert, verbose=verbose)
      File "sage/structure/parent.pyx", line 902, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9245)
        return mor._call_with_args(x, args, kwds)
      File "sage/structure/coerce_maps.pyx", line 180, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_with_args (build/cythonized/sage/structure/coerce_maps.c:5081)
        raise
      File "sage/structure/coerce_maps.pyx", line 175, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_with_args (build/cythonized/sage/structure/coerce_maps.c:4969)
        return C._element_constructor(x, *args, **kwds)
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/geometry/polyhedron/parent.py", line 525, in _element_constructor_
        return self.element_class(self, Vrep, Hrep, **kwds)
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/geometry/polyhedron/base.py", line 121, in __init__
        self._init_from_Vrepresentation(vertices, rays, lines, **kwds)
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_ppl.py", line 93, in _init_from_Vrepresentation
        self._init_Vrepresentation_from_ppl(minimize)
      File "/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_ppl.py", line 161, in _init_Vrepresentation_from_ppl
        gs = self._ppl_polyhedron.minimized_generators()
      File "ppl/polyhedron.pyx", line 335, in ppl.polyhedron.Polyhedron.minimized_generators
    RuntimeError: Aborted
**********************************************************************

...

    Killed due to abort
**********************************************************************
Tests run before process (pid=19555) failed:
sage: p = polytopes.hypercube(2) ## line 71 ##
sage: from sage.geometry.polyhedron.base import is_Polyhedron ## line 72 ##
sage: is_Polyhedron(p) ## line 73 ##
True

...

sage: v = [(1,0,7,-1), (-2,-2,4,-3), (-1,-1,-1,4), (2,9,0,-5), (-2,-1,5,1)] ## line 7164 ##
sage: simplex = Polyhedron(v); simplex ## line 7165 ##
A 4-dimensional polyhedron in ZZ^4 defined as the convex hull of 5 vertices
sage: len(simplex.integral_points()) ## line 7167 ##
49
sage: P = 1/10*polytopes.hypercube(14) ## line 7172 ##
sig_error() without sig_on()

----------------------------------------------------------------------
sage -t --long src/sage/geometry/polyhedron/base.py  # Killed due to abort
----------------------------------------------------------------------

CC: @mkoeppe @jplab @orlitzky

Component: geometry

Author: Jonathan Kliem

Branch: c701c31

Reviewer: Sébastien Labbé, Michael Orlitzky, Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/28866

[orlitzky]

comment:44

There are also a lot of doctest examples in src/sage/geometry/polyhedron/base.py that could be split into separate chunks with :: rather than grouped together, sharing a context. That may help encourage garbage collection of the objects defined in earlier examples.

[kliem]

comment:45

Replying to @seblabbe:

I just checked the most recent branch and

sage -t --long --memlimit=3500 src/sage/geometry/polyhedron/base.py 

now works ok on my machine. But unfortunately, --memlimit=3400 does not (error is raised at the quadrangle line again). But, I guess my computer is an exception?

I would have suggested the following change which is more in the spirit of Python3:

-    new_vertices = map(lambda v : tuple(scalar*x for x in v._vector), self.vertex_generator())
-    new_rays     = map(lambda r : tuple(sign*x   for x in r._vector), self.ray_generator())
+    new_vertices = (tuple(scalar*x for x in v._vector) for v in self.vertex_generator())
+    new_rays     = (tuple(sign*x   for x in r._vector) for r in self.ray_generator())

I agree, you can go ahead and push that change.

[orlitzky]

comment:46

More wild speculation: this is also the only module I've seen that has doctests within a WARNING block. There is a tiny chance that a series of unfortunate events leads to those doctests not being GC'ed. But that still wouldn't explain why you're seeing different results than we are.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 12ecb78 to 93237d6

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

93237d6

28866: avoid map of a lambda function

[seblabbe]

comment:48

Yeah, but this still shouldn't be this hard to figure out. How much RAM does your system have and is swap enabled?

I'm sorry, can you teach me how I can figure this out?

One difference that I suspect between you and (Jonathan and I) is the optimization flags passed to the compiler while building sage. I think we both use -march=native for everything, and you could be getting some SPKG built non-optimally to the point where one of these methods wastes a ton of memory. If you're up for it, you could try

Ok, I will test that.

[orlitzky]

comment:49

Replying to @seblabbe:

I'm sorry, can you teach me how I can figure this out?

There's probably nothing unusual here, but:

$ cat /proc/meminfo

should show the relevant stuff on linux. On mac or windows I would have to resort to google.

[seblabbe]

comment:50

$ cat /proc/meminfo

should show the relevant stuff on linux. On mac or windows I would have to resort to google.

This is the output on the machine having trouble:

$ cat /proc/meminfo
MemTotal:       16356316 kB
MemFree:         3011884 kB
MemAvailable:   14369868 kB
Buffers:         1356356 kB
Cached:          7550684 kB
SwapCached:         1856 kB
Active:          4893152 kB
Inactive:        5302360 kB
Active(anon):     398700 kB
Inactive(anon):   971972 kB
Active(file):    4494452 kB
Inactive(file):  4330388 kB
Unevictable:          48 kB
Mlocked:              48 kB
SwapTotal:       2928636 kB
SwapFree:        2868372 kB
[...]

[seblabbe]

comment:51

(Note that when I pushed the branch on the ticket, it automatically set the ticket to needs_review again (it seems to be a new feature), so I let you review my commit)

[kliem]

Changed reviewer from Michael Orlitzky to Sébastien Labbé, Michael Orlitzky

[orlitzky]

comment:53

Replying to @seblabbe:

This is the output on the machine having trouble:

Nothing unusual.

Another thing I thought of: if recompiling everything with -march=native -O2 doesn't help... maybe there's some system package being used that was built without optimizations...

$ grep 'will use system' config.log

should show which which system packages are being used.

[vbraun]

comment:54

On Python 2:

File "src/sage/geometry/polyhedron/base.py", line 160, in sage.geometry.polyhedron.base.Polyhedron_base.__init__
Failed example:
    p = Polyhedron_field(parent, Vrep, 'nonsense',
                         Vrep_minimal=True, Hrep_minimal=True, pref_rep='Vrep')
Expected:
    Traceback (most recent call last):
    ...
    TypeError: _init_Hrepresentation() takes 3 positional arguments but 9 were given
Got:
    <BLANKLINE>
    Traceback (most recent call last):
      File "/var/lib/buildbot/slave/sage2_git/build/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/var/lib/buildbot/slave/sage2_git/build/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.geometry.polyhedron.base.Polyhedron_base.__init__[13]>", line 2, in <module>
        Vrep_minimal=True, Hrep_minimal=True, pref_rep='Vrep')
      File "/var/lib/buildbot/slave/sage2_git/build/local/lib/python2.7/site-packages/sage/geometry/polyhedron/base.py", line 172, in __init__
        self._init_from_Vrepresentation_and_Hrepresentation(Vrep, Hrep)
      File "/var/lib/buildbot/slave/sage2_git/build/local/lib/python2.7/site-packages/sage/geometry/polyhedron/backend_field.py", line 163, in _init_from_Vrepresentation_and_Hrepresentation
        self._init_Hrepresentation(*Hrep)
    TypeError: _init_Hrepresentation() takes exactly 3 arguments (9 given)
**********************************************************************
1 item had failures:
   1 of  15 in sage.geometry.polyhedron.base.Polyhedron_base.__init__
    [1443 tests, 1 failure, 37.42 s]
----------------------------------------------------------------------
sage -t --long src/sage/geometry/polyhedron/base.py  # 1 doctest failed
----------------------------------------------------------------------

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 93237d6 to c701c31

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

ab4f49a	python2 doctests
c701c31	Merge branch 'public/28866-reb' of git://trac.sagemath.org/sage into public/28866-reb

[mkoeppe]

Changed reviewer from Sébastien Labbé, Michael Orlitzky to Sébastien Labbé, Michael Orlitzky, Matthias Koeppe

[mkoeppe]

comment:57

Looks good to me

[vbraun]

Changed branch from public/28866-reb to c701c31

[kliem]

Changed commit from c701c31 to none

[kliem]

comment:59

I introduced a bug in this ticket: The double description for intervals is completely wrong.

Please review #29904.