OEIS update (database format change, new entries, incorrect warning handling)

[seblabbe]

This ticket fixes the 34 doctest failures that happen when internet is enabled, see this sage-release thread.

There are 3 kinds or errors:

• the oeis database gets updated with new data (this is a normal update),
• the oeis database changed its internal format, and does not make difference between signed and unsigned sequences anymore (in particular, for signed sequences, the V,W,X fields disapear, and are replaced with the S,T,U fields),
• the warnings triggered when a dead sequence is met were not correctly doctested.

The 3 commits corresponds to these 3 kind of errors

To test the branch:

sage -t --optional=sage,internet src/sage/databases/oeis.py 

Component: combinatorics

Keywords: days94

Author: Thierry Monteil

Branch/Commit: 9889222

Reviewer: Frédéric Chapoton

Issue created by migration from https://trac.sagemath.org/ticket/25471

[seblabbe]

comment:1

shorter title

[seblabbe]

Description changed:

--- 
+++ 
@@ -6,7 +6,6 @@

Running doctests with ID 2018-05-30-22-32-01-a785b24b. -Git branch: 25305 Using --optional=bliss,ccache,mpir,python2,sage Doctesting 1 file. sage -t --warn-long 82.7 src/sage/databases/oeis.py

[seblabbe]

Description changed:

--- 
+++ 
@@ -29,7 +29,6 @@

Running doctests with ID 2018-05-30-22-32-15-89fb8007. -Git branch: 25305 Using --optional=internet,sage Doctesting 1 file. sage -t --warn-long 82.7 src/sage/databases/oeis.py

[seblabbe]

Description changed:

--- 
+++ 
@@ -1,25 +1,4 @@
-This is great:
-
-```
-sage -t src/sage/databases/oeis.py 
-```
-
-```
-Running doctests with ID 2018-05-30-22-32-01-a785b24b.
-Using --optional=bliss,ccache,mpir,python2,sage
-Doctesting 1 file.
-sage -t --warn-long 82.7 src/sage/databases/oeis.py
-    [118 tests, 0.99 s]
-----------------------------------------------------------------------
-All tests passed!
-----------------------------------------------------------------------
-Total time for all tests: 1.0 seconds
-    cpu time: 1.3 seconds
-    cumulative wall time: 1.0 seconds
-```
-
-but as reported on [sage-release 8.3.beta3](https://groups.google.com/d/msg/sage-release/l635YEuT7Hs/WkHCnmWhAQAJ), many doctests depending on internet are failing 
-
+As reported on [sage-release 8.3.beta3](https://groups.google.com/d/msg/sage-release/l635YEuT7Hs/WkHCnmWhAQAJ), many doctests depending on internet are failing 

sage -t --optional=sage,internet src/sage/databases/oeis.py

[edd8e884-f507-429a-b577-5d554626c0fe]

comment:6

Hi,

i will be on this (some might be tricky, e.g. related to dead sequences).

You write:

Note that, as Sloane writes this week on the ​arxiv, the oeis is a now a Wiki which means that Sage doctests should depend as few as possible on the many modifications done on the OEIS database...

Are you suggesting to parse the wiki ???

Note that the issue of the database changing was discussed in previous tickets. Still, we should show part of the answers for having meaningful examples (not only for testing purpose, but also for the user).

[jhpalmieri]

comment:7

By the way, several of us predicted this sort of situation (lots of broken doctests related to OEIS) in #16252.

[edd8e884-f507-429a-b577-5d554626c0fe]

Changed keywords from none to days94

[edd8e884-f507-429a-b577-5d554626c0fe]

Branch: u/tmonteil/34_internet_doctests_failed_in_databases_oeis_py

[edd8e884-f507-429a-b577-5d554626c0fe]

Changed branch from u/tmonteil/34_internet_doctests_failed_in_databases_oeis_py to none

[edd8e884-f507-429a-b577-5d554626c0fe]

Branch: u/tmonteil/oeis_update

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

cf7a0aa

merge

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Commit: cf7a0aa

[edd8e884-f507-429a-b577-5d554626c0fe]

Changed branch from u/tmonteil/oeis_update to none

[edd8e884-f507-429a-b577-5d554626c0fe]

Changed commit from cf7a0aa to none

[edd8e884-f507-429a-b577-5d554626c0fe]

Branch: u/tmonteil/oeis_update

[edd8e884-f507-429a-b577-5d554626c0fe]

Commit: 9889222

[edd8e884-f507-429a-b577-5d554626c0fe]

New commits:

e081bc5	Trac 25471 : fix the 20 trivial upstream changes
6f15c06	Trac 25471 : the OEIS database format changed and does not make difference between signed and unsigned sequences anymore
9889222	Trac 25471 : fix warnings for dead sequences

[edd8e884-f507-429a-b577-5d554626c0fe]

Description changed:

--- 
+++ 
@@ -1,62 +1,17 @@
-As reported on [sage-release 8.3.beta3](https://groups.google.com/d/msg/sage-release/l635YEuT7Hs/WkHCnmWhAQAJ), many doctests depending on internet are failing 
+This ticket fixes the 34 doctest failures that happen when internet is enabled, see [this sage-release thread](https://groups.google.com/d/msg/sage-release/l635YEuT7Hs/WkHCnmWhAQAJ).
+
+There are 3 kinds or errors:
+
+- the oeis database gets updated with new data (this is a normal update),
+- the oeis database changed its internal format, and does not make difference between signed and unsigned sequences anymore (in particular, for signed sequences, the `V,W,X` fields disapear, and are replaced with the `S,T,U` fields),
+- the warnings triggered when a dead sequence is met were not correctly doctested.
+
+The 3 commits corresponds to these 3 kind of errors
+
+To test the branch:

sage -t --optional=sage,internet src/sage/databases/oeis.py

-gives:

-```
-Running doctests with ID 2018-05-30-22-32-15-89fb8007.
-Using --optional=internet,sage
-Doctesting 1 file.
-sage -t --warn-long 82.7 src/sage/databases/oeis.py
-**********************************************************************
-File "src/sage/databases/oeis.py", line 29, in sage.databases.oeis
-Failed example:
-    search = oeis([3, 7, 15, 1], max_results=4) ; search  # optional -- internet
-Expected:
-    0: A001203: Continued fraction expansion of Pi.
-    1: A082495: a(n) = (2^n - 1) mod n.
-    2: A165416: Irregular array read by rows: The n-th row contains those distinct positive integers that each, when written in binary, occurs as a substring in binary n.
-    3: A246674: Run Length Transform of A000225.
-Got:
-    0: A001203: Simple continued fraction expansion of Pi.
-    1: A082495: a(n) = (2^n - 1) mod n.
-    2: A165416: Irregular array read by rows: The n-th row contains those distinct positive integers that each, when written in binary, occurs as a substring in binary n.
-    3: A240698: Partial sums of divisors of n, cf. A027750.
-**********************************************************************
-
-...
-
-**********************************************************************
-17 items had failures:
-   5 of  26 in sage.databases.oeis
-   4 of  21 in sage.databases.oeis.OEIS
-   1 of   5 in sage.databases.oeis.OEIS.find_by_description
-   1 of   3 in sage.databases.oeis.OEIS.find_by_subsequence
-   5 of   8 in sage.databases.oeis.OEISSequence
-   3 of  13 in sage.databases.oeis.OEISSequence.__call__
-   2 of   8 in sage.databases.oeis.OEISSequence.__getitem__
-   1 of   4 in sage.databases.oeis.OEISSequence.__init__
-   1 of   5 in sage.databases.oeis.OEISSequence.author
-   2 of   7 in sage.databases.oeis.OEISSequence.cross_references
-   1 of   5 in sage.databases.oeis.OEISSequence.examples
-   2 of   5 in sage.databases.oeis.OEISSequence.extensions_or_errors
-   1 of  14 in sage.databases.oeis.OEISSequence.first_terms
-   1 of   5 in sage.databases.oeis.OEISSequence.formulas
-   2 of   9 in sage.databases.oeis.OEISSequence.links
-   1 of  22 in sage.databases.oeis.OEISSequence.natural_object
-   1 of   5 in sage.databases.oeis.OEISSequence.show
-    [266 tests, 34 failures, 83.18 s]
-----------------------------------------------------------------------
-sage -t --warn-long 82.7 src/sage/databases/oeis.py  # 34 doctests failed
-----------------------------------------------------------------------
-Total time for all tests: 83.3 seconds
-    cpu time: 1.8 seconds
-    cumulative wall time: 83.2 seconds
-```
-
-Full log is available on [framabin](https://framabin.org/?fd3d1939f986751b#rFxmYLviVoApcqKcKTqkbIeRZnxf6h7I9jKQ2zggVU8=).
-
-Note that, as Sloane writes this week on the [arxiv](https://arxiv.org/abs/1805.10343), the oeis is a now a Wiki which means that Sage doctests should depend as few as possible on the many modifications done on the OEIS database...

[edd8e884-f507-429a-b577-5d554626c0fe]

Author: Thierry Monteil

[fchapoton]

comment:16

ok (but this will need to be done again and again...)

[fchapoton]

Reviewer: Frédéric Chapoton

[vbraun]

Changed branch from u/tmonteil/oeis_update to 9889222