Upgrade: lrslib 071b and polymake 4.4, deprecate PolymakeExpect

mkoeppe commented 5 years ago

lrslib is now at 071b.

http://cgm.cs.mcgill.ca/~avis/C/lrs.html
http://cgm.cs.mcgill.ca/~avis/C/lrslib/archive/
Sage package built from https://github.com/mkoeppe/lrslib
To test: make configure && eval ./configure $(./config.status --config) --enable-download-from-upstream-url && make lrslib

polymake 4.4 tarball: https://polymake.org/lib/exe/fetch.php/download/polymake-4.4-minimal.tar.bz2

Polymake 4.4 was released 2021-05-18, we are several releases behind:

For updated instructions regarding the Perl prerequisites, see #31830. Let's keep that a separate ticket.

CC: @slel @yuan-zhou @kliem @videlec @kiwifb @dimpase @simon-king-jena

Component: packages: optional

Keywords: upgrade, lrslib

Author: Matthias Koeppe, Jonathan Kliem, Travis Scrimshaw

Branch/Commit: ff69f28

Reviewer: Thierry Monteil, Matthias Koeppe, Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/27745

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 3504a7d to 06284e9

tscrim commented 3 years ago

Description changed:

--- 
+++ 
@@ -5,6 +5,8 @@
 - Sage package built from https://github.com/mkoeppe/lrslib
 - To test:
 `make configure && eval ./configure $(./config.status --config) --enable-download-from-upstream-url && make lrslib`
+
+polymake 4.4 tarball: https://polymake.org/lib/exe/fetch.php/download/polymake-4.4-minimal.tar.bz2

 Polymake 4.4 was released 2021-05-18, we are several releases behind:
 - https://polymake.org/doku.php/news/release_4_4

edd8e884-f507-429a-b577-5d554626c0fe commented 3 years ago

comment:44

The lrslib issue with openmpi is fixed (self-tests pass).

If i do export MAKE='make -j4' before running sage -i -f polymake, the compilation still eats all my CPU, though spkg-install.in relies on sdh_make.

Also, i have 6G free on my machine, and the compilation ends up with a No space left on device. How much disk space is required to build and install polymake ?

tscrim commented 3 years ago

comment:45

Matthias linked it to SAGE_NUM_THREADS, but it is might indeed be better to link it to what MAKE sets. I am letting mine compile with 2 threads right now since 4 still ate through 10+Gb RAM.

dimpase commented 3 years ago

comment:46

use make -j4 polymake, do not export MAKE. Time is high to drop sage -i, it is an extra hassle.

edd8e884-f507-429a-b577-5d554626c0fe commented 3 years ago

comment:47

Replying to @dimpase:

use make -j4 polymake, do not export MAKE. Time is high to drop sage -i, it is an extra hassle.

But then, how to i run self tests ?

dimpase commented 3 years ago

comment:48

Polymake is huge, it's about as big as Sage, I think. Go figure.

dimpase commented 3 years ago

comment:49

Replying to @sagetrac-tmonteil:

Replying to @dimpase:

use make -j4 polymake, do not export MAKE. Time is high to drop sage -i, it is an extra hassle.

But then, how to i run self tests ?

SAGE_CHECK=yes make -j4 polymake

kliem commented 3 years ago

comment:50

How do I link perl after having installed it with

cpan -i XML::Writer XML::LibXML XML::LibXSLT File::Slurp Term::ReadLine::Gnu JSON SVG MongoDB?

I'm on a debian buster and don't know how to install polymake into sage.

mkoeppe commented 3 years ago

comment:51

Replying to @tscrim:

Matthias linked it to SAGE_NUM_THREADS, but it is might indeed be better to link it to what MAKE sets.

During build, SAGE_NUM_THREADS is actually initialized from the MAKE setting. See build/bin/sage-build-num-threads and build/make/install.

mkoeppe commented 3 years ago

comment:52

Replying to @kliem:

How do I link perl after having installed it with

cpan -i XML::Writer XML::LibXML XML::LibXSLT File::Slurp Term::ReadLine::Gnu JSON SVG MongoDB?

I'm on a debian buster and don't know how to install polymake into sage.

On Debian, you can just install the prerequisite packages listed in build/pkgs/perl_cpan_polymake_prereq/distros/debian.txt

mkoeppe commented 3 years ago

comment:53

Replying to @dimpase:

use make -j4 polymake, do not export MAKE. Time is high to drop sage -i, it is an extra hassle.

The correct way to install an optional package is, of course,

eval ./configure $(./config.status --config) --enable-jupymake && make build

kliem commented 3 years ago

comment:54

Replying to @mkoeppe:

Replying to @kliem:

How do I link perl after having installed it with

cpan -i XML::Writer XML::LibXML XML::LibXSLT File::Slurp Term::ReadLine::Gnu JSON SVG MongoDB?

I'm on a debian buster and don't know how to install polymake into sage.

On Debian, you can just install the prerequisite packages listed in build/pkgs/perl_cpan_polymake_prereq/distros/debian.txt

Thanks. Well that requires root access, but I guess we have a nice IT so they'll do it.

That sage itself can be just installed without root access is a wonderful thing... (But they also have installed the generated list of recommended packages for me, which makes things a lot faster.)

tscrim commented 3 years ago

Changed commit from 06284e9 to 77bce1f

tscrim commented 3 years ago

comment:55

I was able to build both packages (thank you Matthias for your help with this). However, there are issues with interfaces/polymake.py (make sure you run sage -tp --optional=sage,polymake). The biggest issue is that it doesn't even finish evaluating the first doctest:

polymake.eval('print foo;')

It timed out after running for 5 minutes.

I tried running this directly in my Sage shell, which it is still taking seemingly forever. I also tried C = polymake('cube(3)'), which after I hit ctrl-c, any additional attempt to run the interface results in multiple errors of the form:

TypeError: argument should be integer or bytes-like object, not 'str'

Is this an issue with the interface? Is this something with polymake trying to do something for the first time? Something else?

The branch just has one commit that cleans up the interface file a bit.

New commits:

77bce1f Cleaning up the interface file.

tscrim commented 3 years ago

Changed branch from u/mkoeppe/upgrade__lrslib_071a to u/tscrim/lrslib071a_polymake44_upgrade-27745

mkoeppe commented 3 years ago

comment:56

Sounds like you are testing the old pexpect version of the interface. If you install package jupymake, it will test the current version

tscrim commented 3 years ago

comment:57

No, I did not. Once I installed that, then it worked. Since that is a requirement to using the interface, I think we should have jupymake as a dependency.

tscrim commented 3 years ago

comment:58

Now it is stuck on line 128:

p = polymake_expect.rand_sphere(4, 20, seed=5)

I guess we have to rewrite the interface file to remove all tests involving the old interface?

tscrim commented 3 years ago

comment:59

Actually, do we want to just scrap the pexpect interface altogether?

mkoeppe commented 3 years ago

comment:60

Replying to @tscrim:

No, I did not. Once I installed that, then it worked. Since that is a requirement to using the interface, I think we should have jupymake as a dependency.

jupymake can't be a dependency of polymake because it depends on polymake

mkoeppe commented 3 years ago

comment:61

Replying to @tscrim:

Now it is stuck on line 128:
p = polymake_expect.rand_sphere(4, 20, seed=5)

Here's how to debug this:

sage: from sage.interfaces.polymake import PolymakeExpect
sage: polymake = PolymakeExpect(logfile="POLYMAKE.log")
sage: polymake.rand_sphere(4, 20, seed=5)

Then, in a separate terminal, use tail -f POLYMAKE.log

mkoeppe commented 3 years ago

comment:62

There is indeed something going on with bytes vs strings.

mkoeppe commented 3 years ago

Changed branch from u/tscrim/lrslib071a_polymake44_upgrade-27745 to u/mkoeppe/lrslib071a_polymake44_upgrade-27745

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 77bce1f to 4b7616f

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`7a59b3d`	`src/sage/interfaces/polymake.py: More fixes for python3`
`4b7616f`	`src/sage/interfaces/polymake.py: Adjust user input prompt for polymake 4.4`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

332b72f src/sage/interfaces/polymake.py: Disable minkowski_sum_fukuda test, which seems to take forever with polymake 4.4

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 4b7616f to 332b72f

mkoeppe commented 3 years ago

comment:66

More fixes are needed

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`2e41220`	`src/sage/interfaces/polymake.py: Relax version doctest`
`5d5ce56`	`src/sage/interfaces/polymake.py: Another fix for python3`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 332b72f to 5d5ce56

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`2426e60`	`src/sage/interfaces/polymake.py: Yet another fix for python3`
`d1cbbf1`	`src/sage/geometry/polyhedron/backend_polymake.py: Update for changes in sage`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 5d5ce56 to d1cbbf1

tscrim commented 3 years ago

comment:69

Thank you. That has gotten it down to one failure for me:

**********************************************************************
File "src/sage/interfaces/polymake.py", line 2098, in sage.interfaces.polymake.PolymakeExpect._eval_line
Failed example:
    c.N_VERTICES                      # optional - polymake
Expected:
    32768
Got:
    Can't locate object method "description" via package "32768" (perhaps you forgot to load "32768"?) at input line 1.

However, I cannot reproduce this in an interactive session, so it is probably something with the doctester setup. I propose marking it as # known bug with an explanation that it works interactively if it is failing for someone else.

mkoeppe commented 3 years ago

comment:70

Yes, there's something wrong with error recovery still.

For me, also src/sage/geometry/polyhedron/backend_polymake.py reveals additional errors. So there's more work

kliem commented 3 years ago

comment:71

Thanks for the help above. So I have polymake installed now and it works in sage -sh.

However, I'm getting an EOF error whenever I want to do something in sage, e.g. `polytopes.cube(backend='polymake').

TypeError: unable to start polymake: End Of File (EOF). Exception style platform.
Polymake with PID 27045 running /usr/bin/env TERM=dumb polymake
command: /usr/bin/env
args: ['/usr/bin/env', 'TERM=dumb', 'polymake']
buffer (last 100 chars): b''
before (last 100 chars): b'dLine::Gnu" at /srv/public/kliem/sage/local/share/polymake/perllib/Polymake/Core/Shell.pm line 30.\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 27045
child_fd: 16
closed: False
timeout: None
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 4194304
ignorecase: False
searchwindowsize: None
delaybeforesend: None
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
    0: re.compile(b'polytope > ')

sheerluck commented 3 years ago

comment:72

Replying to @kliem:

However, I'm getting an EOF error whenever I want to do something in sage, e.g. `polytopes.cube(backend='polymake').

in sage.interfaces.polymake we have both polymake_expect and polymake_jupymake so which one is used in polytopes.cube(backend='polymake')?

How to tell sage I always want polymake_jupymake for every something(backend='polymake')?

sheerluck commented 3 years ago

comment:73

oh I see

if PythonModule("JuPyMake").is_present():
    polymake = polymake_jupymake
else:
    polymake = polymake_expect

kliem commented 3 years ago

comment:74

For me it is PolymakeExpect apparently.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from d1cbbf1 to 426385d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

426385d Polyhedra: Raise an error if backend 'polymake' is used with an unsupported field

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 426385d to 29c267b

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

29c267b Polyhedron_polymake: Implement pickling

mkoeppe commented 3 years ago

comment:77

With jupymake installed, ./sage -t --optional=sage,polymake,jupymake,perl_mongodb src/sage/geometry/polyhedron/backend_polymake.py now is down to 1 failure:

sage -t --random-seed=0 src/sage/geometry/polyhedron/backend_polymake.py
**********************************************************************
File "src/sage/geometry/polyhedron/backend_polymake.py", line 220, in sage.geometry.polyhedron.backend_polymake.Polyhedron_polymake.__init__
Failed example:
    TestSuite(p).run()                                 # optional - polymake
Expected nothing
Got:
    Failure in _test_lawrence:
    Traceback (most recent call last):
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/src/sage/misc/sage_unittest.py", line 297, in run
        test_method(tester=tester)
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/src/sage/geometry/polyhedron/base.py", line 6312, in _test_lawrence
        tester.assertTrue(Q.is_combinatorially_isomorphic(R))
      File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/case.py", line 682, in assertTrue
        raise self.failureException(msg)
    AssertionError: False is not true
    ------------------------------------------------------------
    The following tests failed: _test_lawrence
**********************************************************************
1 item had failures:
   1 of   7 in sage.geometry.polyhedron.backend_polymake.Polyhedron_polymake.__init__
    [91 tests, 1 failure, 27.73 s]
----------------------------------------------------------------------
sage -t --random-seed=0 src/sage/geometry/polyhedron/backend_polymake.py  # 1 doctest failed
----------------------------------------------------------------------

mkoeppe commented 3 years ago

comment:78

Same without jupymake.

mkoeppe commented 3 years ago

comment:79

Help with the lawrence testcase would be welcome...

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b4366f1 PolymakeElement._sage_, Polyhedron_polymake._from_polymake_polytope: Determine the base ring

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 29c267b to b4366f1

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from b4366f1 to 89614bb

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`3cb6888`	`PolymakeElement: Adjust doctest`
`89614bb`	`build/pkgs/polymake/SPKG.rst: Recommend installing jupymake`

sagemath / sage

Upgrade: lrslib 071b and polymake 4.4, deprecate PolymakeExpect #27745