search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.45k stars 482 forks source link

Merge sage_brial into sagelib #30332

Closed mkoeppe closed 4 years ago

mkoeppe commented 4 years ago

As discussed in #29369 and #28918.

CC: @kiwifb @orlitzky @tscrim

Component: packages: standard

Author: François Bissey

Branch/Commit: 91fe5e1

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/30332

kiwifb commented 4 years ago
comment:1

I propose to copy the current sage-brial code under sage/rings/polynomial. This is based on the fact all (but one) imports of brial happen there.

$ grep -r brial src/sage/*
src/sage/rings/polynomial/pbori.pyx:# distutils: libraries = brial brial_groebner M4RI_LIBRARIES LIBPNG_LIBRARIES
src/sage/rings/polynomial/pbori.pyx:    sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:        #from brial.interpolate import interpolate_smallest_lex
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import BooleanMonomialMonoid, BooleanMonomial
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid, BooleanMonomial
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleSet
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import BooleanPolynomial
src/sage/rings/polynomial/pbori.pyx:        from brial.parallel import _encode_polynomial
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleSet
src/sage/rings/polynomial/pbori.pyx:        from brial import red_tail
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:        from brial.gbcore import groebner_basis
src/sage/rings/polynomial/pbori.pyx:        from brial import red_tail
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import BooleSet
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleSet
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleSet
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import GroebnerStrategy
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import GroebnerStrategy
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import groebner_basis
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import GroebnerStrategy
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanPolynomialVector
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import GroebnerStrategy
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import groebner_basis
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import GroebnerStrategy
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import GroebnerStrategy
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import GroebnerStrategy
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleanMonomialMonoid
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import map_every_x_to_x_plus_one
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import zeros
src/sage/rings/polynomial/pbori.pyx:        sage: from brial.interpolate import *
src/sage/rings/polynomial/pbori.pyx:        sage: from brial.interpolate import *
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import ll_red_nf_redsb
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import ll_red_nf_noredsb
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import ll_red_nf_noredsb_single_recursive_call
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import if_then_else
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import top_index
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import substitute_variables
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import substitute_variables
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import random_set, set_random_seed
src/sage/rings/polynomial/pbori.pyx:        sage: from brial import random_set, set_random_seed
src/sage/rings/polynomial/pbori.pyx:# todo: merge with pickling from brial.parallel
src/sage/rings/polynomial/pbori.pyx:# todo: merge with pickling from brial.parallel
src/sage/rings/polynomial/pbori.pyx:    from brial.parallel import _decode_polynomial
src/sage/rings/polynomial/pbori.pyx:# todo: merge with pickling from brial.parallel
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import BooleConstant
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/pbori.pyx:            sage: from brial import *
src/sage/rings/polynomial/multi_polynomial_sequence.py:        from brial import gauss_on_polys
src/sage/rings/polynomial/multi_polynomial_sequence.py:        from brial.ll import eliminate,ll_encode,ll_red_nf_redsb
src/sage/rings/polynomial/multi_polynomial_sequence.py:            from brial.interred import interred as inter_red
src/sage/sat/converters/__init__.py:from brial.cnf import CNFEncoder as PolyBoRiCNFEncoder

but other suggestions based on best practices will be given the priority.

mkoeppe commented 4 years ago
comment:2

So there's a Cython interface sage.libs.polybori, then a Python library brial, and also sage.rings.polynomial.pbori? What is depending on what?

kiwifb commented 4 years ago
comment:3

I thought that sage-brial depends on sage.libs.polybori but that may not be the case. It imports sage.all and specifically sage.rings.polynomial.pbori and that's the only sage specific bits it imports.

sage.rings.polynomial.pbori itself relies on sage.libs.polybori (through sage.libs.polybori.decl) and both need the C++ brial library.

Does that answer your question?

mkoeppe commented 4 years ago
comment:4

Thanks. I agree that sage.rings.polynomial seems to be a good place for the Python code from sage-brial. Perhaps it would be best to create a package sage.rings.polynomial.brial and to move the current pbori.pyx inside it (leaving deprecated reimport behind). There are only a few places in Sage where things are imported from sage.rings.polynomial.pbori and they could be easily update. Or, if you don't mind keep the old name pbori, the same could be done with sage.rings.polynomial.pbori instead of ...brial.

One thing to keep in mind is the context of the planned modularization of sagelib. In Sage 9.3 I would hope to be able to create a separate distribution (distutils package) sage-brial, which would then package sage.rings.polynomial.{brial,pbori}, sage.libs.polybori, sage.libs.fes, sage.crypto.boolean_function -- everything that has a compile-time dependency on libbrial. This will make it possible to separately compile sagelib on systems without brial.

kiwifb commented 4 years ago
comment:5

On the legal side, the code is GPL2.0 or later. Although I will need to fix the license on github. It seems I changed it from 2+ to 2 carelessly back in 2017 (probably by copying a template). I obviously had no rights to do that.

The only thing I may need guidance with is the deprecated re-import. Otherwise moving stuff, including pbori.{pyx,pxd}, in sage.rings.polynomial.pbori seems to be a good idea and hopefully will help with modularization.

mkoeppe commented 4 years ago
comment:6

Replying to @kiwifb:

On the legal side, the code is GPL2.0 or later. Although I will need to fix the license on github. It seems I changed it from 2+ to 2 carelessly back in 2017 (probably by copying a template). I obviously had no rights to do that.

I think GitHub takes the position that there is no such thing as "GPL 2 only": That the license includes "... or any later version" no matter whether you write these words in the files or not.

kiwifb commented 4 years ago
comment:7

Replying to @mkoeppe:

Replying to @kiwifb:

On the legal side, the code is GPL2.0 or later. Although I will need to fix the license on github. It seems I changed it from 2+ to 2 carelessly back in 2017 (probably by copying a template). I obviously had no rights to do that.

I think GitHub takes the position that there is no such thing as "GPL 2 only": That the license includes "... or any later version" no matter whether you write these words in the files or not.

Very nice of them. But they are not lawyers and I think the "LICENSE" file should be explicit for people who look for these subtle clarification (distros). The GPL FAQ https://www.gnu.org/licenses/old-licenses/gpl-2.0-faq.html#VersionTwoOrLater seem to imply that you should explicitly state it.

kiwifb commented 4 years ago
comment:8

Tests included inside sage-brial will have to be moved to sage doctesting. More generally it will have to be documented properly as much as possible. This will be so much fun.

It looks like that will take much more than just drag and drop in place with a few cosmetic changes.

mkoeppe commented 4 years ago
comment:9

I suppose you could prepare the change to using the sage doctester first, before merging into Sage.

mkoeppe commented 4 years ago
comment:10

Given that sage_brial has no Cython bits, I think it could in fact be removed from the dependencies of sagelib, as it won't really be needed while installing sagelib, only later for docbuild and doctest.

kiwifb commented 4 years ago
comment:11

Replying to @mkoeppe:

Given that sage_brial has no Cython bits, I think it could in fact be removed from the dependencies of sagelib, as it won't really be needed while installing sagelib, only later for docbuild and doctest.

Technically correct.

kiwifb commented 4 years ago
comment:12

This is not ready for review. I am just pushing my work in progress so it is not just on my workstation, it is visible and open to others contribution.

There is a lot to do:

  1. update paths
  2. convert docstrings and tests to sage framework

This may take some time and I doubt it will be ready in time for 9.2 but we can only try.

New commits:

884ca66WIP: move sage-brial files and pbori.{pxd,pyx} in sage/rings/polynomial/pbori/
kiwifb commented 4 years ago

Commit: 884ca66

kiwifb commented 4 years ago

Branch: public/merge_sage_brial

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

f947293replace "from sage.rings.polynomial.pbori" with "from sage.rings.polynomial.pbori.pbori"
e7653a3Replace "from brial" with "from sage.rings.polynomial.pbori.brial"
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 884ca66 to e7653a3

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from e7653a3 to 2226097

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2226097src/sage/rings/polynomial/pbori/__init__.py: Draft of lazy_import call
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

d0bc786transform python test into sage tests - step one.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 2226097 to d0bc786

kiwifb commented 4 years ago
comment:16

I am currently converting docstrings :( I am also doing some selective code deletion as I come along across it. There is a module for memory usage, it is imported once but never used and even less tested. So, I am removing it. It will be worth another look after that first pass.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

ae30cf3Basic conversion to sage doctrings/doctesting. Remove original copyright/license functions.
4c0d80fSome basic docstring in general_boolean_polynomial.py - remove memusage.py and its only import.
bdf1fecFirst pass in converting to docstring and doctests. It is quite likely a lot of doctests are broken at this stage.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from d0bc786 to bdf1fec

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from bdf1fec to 2e94a77

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2e94a77Merge branch 'develop' into brial-merging
kiwifb commented 4 years ago
comment:19

I suspect the building of the html doc will break at this stage. If it doesn't the output may very well be useless. Once I figure that out, I'll do the doctesting to see what's broken. I am expecting a lot of breaking in the newly imported tests.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 2e94a77 to 4e3ed88

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

4e3ed88one too many pbori.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 4e3ed88 to 563e959

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

563e959add miising import of lazy_import
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

7cb8ba1remove excessive .brial from import. The initial replacement was wrong.
408824dFix building of the pbori interface documentation. No "brial" documentation is buiilt as far as I can tell.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 563e959 to 408824d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 408824d to 443a3a8

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

443a3a8Remove addition.py - nothing from this file is used and tests failures point to the file being obsolote.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 443a3a8 to 167ef09

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

167ef09migrate xrange (unsupported in python3) to range
kiwifb commented 4 years ago
comment:25

At this point documentation builds and is identical to the old one. doctesting gets a lot of failures. There is still quite a bit of cleaning to be done, including removing files that should have been trashed long ago.

kiwifb commented 4 years ago
comment:26

More joy! I found a subtle coding error while fixing the test suite. I guess it won't have an impact on the rest of sage because that code path just error-ed out when taken. So it is probably a dead code path anyway.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 167ef09 to e25dd81

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

afdbb9dfix doctests in blocks.py
ec4cddbremove check_claims.py which appears to be a standalone script.
d53714fremove another script
72f8a6aFix doctest and code in cnf.py
e25dd81Fix pbori related doctests and code in multi_polynomial_sequence
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2c9b36fRemove unused files
a0ccf77removing unused files: were used by the previously removed files.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from e25dd81 to a0ccf77

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from a0ccf77 to 20d1f9e

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b3c16a8Revert "removing unused files: were used by the previously removed files." Turns out interred is used somewhere else.
20d1f9eFix doctests in pbori.pyx
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 20d1f9e to bc1f9e2

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

bc1f9e2fix more doctests
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from bc1f9e2 to 64c57d2

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b513981more doctests fixes
1cf36c7Remove VariableFactory from the lazy_import list for now.
64c57d2Last bunch of doctest fixes
kiwifb commented 4 years ago
comment:32

This is ready for a first review. I am not expecting it to make it as is. But other people should comment now.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

878d941Remove the sage_brial package since its meaningful content is merged.