Interface to FindStat

[mantepse]

Sage should have an interface to http://www.findstat.org, the "combinatorial statistics finder". A first version, modelled after the interface to the OEIS is attached. To get you started, you can try the following:

sage: findstat('St000045')                   # optional -- internet
    St000045: The number of linear extensions of the tree. 

    sage: s = findstat(3); s                            # optional -- internet
    St000003: The number of [[/StandardTableaux|standard Young tableaux]] of the partition.

    sage: s.browse()

    sage: stat = {pi: pi.length() for pi in Permutations(3)}
    sage: search = findstat(stat) ; search    # optional -- internet
    0: (St000018: The [[/Permutations/Inversions|number of inversions]] of a permutation., [], 6)
    1: (St000004: The [[/Permutations/Descents-Major|major index]] of a permutation., ['inversion-number to major-index bijection'], 6)
    2: (St000067: The inversion number of the alternating sign matrix., ['to alternating sign matrix'], 6)
    3: (St000224: The sorting index of a permutation., ['first fundamental transformation'], 6)
    4: (St000008: The major index of the composition., ['inversion-number to major-index bijection', 'descent composition'], 6)
    5: (St000059: The inversion number of a standard Young tableau as defined by Haglund and Stevens. [1]  , ['inversion-number to major-index bijection', 'Robinson-Schensted recording tableau'], 6)
    6: (St000153: The number of adjacent cycles of a permutation., ['reverse', 'first fundamental transformation'], 6)
    7: (St000156: The Denert index of a permutation., ['inverse first fundamental transformation', 'foata_bijection'], 6)
    8: (St000174: The flush statistic on semistandard tableaux., ['to alternating sign matrix', 'to semistandard tableau'], 6)

CC: @videlec @nathanncohen @stumpc5 @VivianePons

Component: interfaces: optional

Keywords: FindStat

Author: Martin Rubey

Branch/Commit: ec3bc2a

Reviewer: Christian Stump, Viviane Pons

Issue created by migration from https://trac.sagemath.org/ticket/17818

[mantepse]

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
-Sage should have an interface to FindStat, the "combinatorial statistics finder".  A first version, modelled after the interface to the OEIS is attached.  To get you started, you can try the following:
+Sage should have an interface to http://www.findstat.org, the "combinatorial statistics finder".  A first version, modelled after the interface to the OEIS is attached.  To get you started, you can try the following:

sage: findstat('St000045') # optional -- internet

[jdemeyer]

comment:3

I know this isn't yet set to needs_review, but the documentation is misformatted in many places. It should be like

EXAMPLES::

    sage: ...

Some text to introduce the next example::

    sage: ...

[mantepse]

comment:5

You mean, the text between examples is missing?

[jdemeyer]

comment:7

No, I mean for example

    We can access references provided as follows:
    sage: s.references()

should be

We can access references provided as follows::

    sage: s.references()

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from ce07ce4 to d21fad3

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

d21fad3

fix formatting for EXAMPLES and cache the right method

[6bdad4c1-1e26-4f2f-a442-a01a2292c181]

comment:9

Your new commit has the same problem. When you use "::" somewhere, the next line must be indented. To make sure that everything works as it should:

1) Add your new file to the documentation, i.e. the SAGE_ROOT/src/doc/en/reference/databases/index.rst file

2) compile the documentation sage -b && sage -docbuild reference/databases html

And, of course, read the html page corresponding to your new file to check that it is alright.

See also: http://www.sagemath.org/doc/developer/sage_manuals.html#adding-a-new-file

Nathann

[videlec]

comment:10

Hello,

I really do understand neither your strategy nor your goal with this ticket.

oeis is a Sage completely independent project and we have this ugly parser because we do not have any choice: there is no API to access their database.

Findstat project highly uses Sage and you are one of the author. Instead of integrating findstat code inside Sage you decided to make a closed source project. Right, this is fine with GPL and this is your choice. Now you decide to interface your findstat project with Sage. As a consequence you oblige the user to have an internet access to play with findstat features. Fine, this is again your choice. Instead of providing a clean API from the findstat side (let say json answers from http queries) you chose to implement an ugly HTML parser with regexp to get the information on the Sage side. This is inefficient and counter productive and you should seriously revise your strategy.

Vincent

[mantepse]

comment:11

I agree that the situation could be better from the sage point of view. However, there are a few things to keep in mind:

The current FindStat project really does want to be open to non-SageMath users, and it tries very hard. In fact, I believe that most of the effort went into the web interface. I think this is very important, so I make it boldface :-)

Currently, I (Martin) am mainly interested in using FindStat from within SageMath. I asked Christian whether this would be possible, and so he provided me with the necessary information to create this interface.

I agree that a clean API is desirable, but it is beyond my knowledge, and beyond the human resources of the FindStat project. (As far as I know, there are currently five people who contributed to the FindStat project -- all listed here: http://www.findstat.org/Contributors. My first and so far only contribution (apart from editing statistics) is the interface, but actually, this is more a contribution to SageMath, isn't it?)

Therefore, if you are able to provide such a clean interface, please go for it!

[videlec]

comment:12

Replying to @mantepse:

I agree that the situation could be better from the sage point of view. However, there are a few things to keep in mind:

The current FindStat project really does want to be open to non-SageMath users, and it tries very hard. In fact, I believe that most of the effort went into the web interface. I think this is very important, so I make it boldface :-)

Agreed. The website is nice as a website.

Currently, I (Martin) am mainly interested in using FindStat from within SageMath. I asked Christian whether this would be possible, and so he provided me with the necessary information to create this interface.

This is where I do not agree. The solution to make it Sagemath-friendly is not the interface proposed in this ticket. There should be another web interface for interaction with Sage. This is not the hard part of the work.

I agree that a clean API is desirable, but it is beyond my knowledge, and beyond the human resources of the FindStat project. (As far as I know, there are currently five people who contributed to the FindStat project -- all listed here: http://www.findstat.org/Contributors. My first and so far only contribution (apart from editing statistics) is the interface, but actually, this is more a contribution to SageMath, isn't it?)

In its current version, I am not sure this interface does more good to Sagemath than to Findstat.

Therefore, if you are able to provide such a clean interface, please go for it!

I would if the code was freely available... please go release it!

Vincent

[mantepse]

comment:13

This is where I do not agree. The solution to make it Sagemath-friendly is not the interface proposed in this ticket. There should be another web interface for interaction with Sage. This is not the hard part of the work.

It would be for me. Besides, in which way precisely would this be better for SageMath?

In its current version, I am not sure this interface does more good to Sagemath than to Findstat.

I admit, this is not really important for me, I shouldn't have raised this point. I am mainly a mathematician. If this interface is useful to you, use it, otherwise don't. I was asked to polish the code so it is usable for people other than me, so I spent a few days on polishing. If it doesn't go into SageMath that's fine with me. I hope I will still be able to use it myself.

Therefore, if you are able to provide such a clean interface, please go for it!

I would if the code was freely available... please go release it!

I do not understand why you think that I am in the position to do that. FindStat is not my website, not my code.

[mantepse]

comment:14

Replying to @nathanncohen:

Your new commit has the same problem.

2) compile the documentation sage -b && sage -docbuild reference/databases html

Hi Nathann!

Many thanks for your help, but I'm afraid I need more. Is there a way to make docbuild tell me where I made a mistake? I obtain

[databases] loading pickled environment... not yet created
[databases] building [html]: targets for 10 source files that are out of date
[databases] updating environment: 10 added, 0 changed, 0 removed
[databases] Encoding error:
[databases] 'ascii' codec can't decode byte 0xe2 in position 24871: ordinal not in range(128)
[databases] The full traceback has been saved in /tmp/sphinx-err-YMbReV.log, if you want to report the issue to the developers.
Build finished. The built documents can be found in /home/martin/sage/src/doc/output/html/en/reference/databases

but I have no idea what that traceback is trying to tell me:

Traceback (most recent call last):
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/cmdline.py", line 254, in main
    app.build(force_all, filenames)
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/application.py", line 212, in build
    self.builder.build_update()
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/builders/__init__.py", line 214, in build_update
    'out of date' % len(to_build))
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/builders/__init__.py", line 234, in build
    purple, length):
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/builders/__init__.py", line 134, in status_iterator
    for item in iterable:
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/environment.py", line 480, in update_generator
    self.read_doc(docname, app=app)
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/environment.py", line 627, in read_doc
    pub.publish()
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/core.py", line 217, in publish
    self.settings)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/readers/__init__.py", line 72, in read
    self.parse()
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/__init__.py", line 172, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 170, in run
    input_source=document['source'])
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2301, in explicit_markup
    self.explicit_list(blank_finish)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2331, in explicit_list
    match_titles=self.state_machine.match_titles)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 319, in nested_list_parse
    node=node, match_titles=match_titles)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2604, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2311, in explicit_construct
    return method(self, expmatch)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2054, in directive
    directive_class, match, type_name, option_presets)
  File "/home/martin/sage/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2103, in run_directive
    result = directive_instance.run()
  File "/home/martin/sage/src/doc/common/sage_autodoc.py", line 1166, in run
    documenter.generate(more_content=self.content)
  File "/home/martin/sage/src/doc/common/sage_autodoc.py", line 638, in generate
    self.analyzer = ModuleAnalyzer.for_module(self.real_modname)
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/pycode/__init__.py", line 204, in for_module
    obj = cls.for_file(source, modname)
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/pycode/__init__.py", line 187, in for_file
    obj = cls(fileobj, modname, filename)
  File "/home/martin/sage/local/lib/python2.7/site-packages/Sphinx-1.2.2-py2.7.egg/sphinx/pycode/__init__.py", line 224, in __init__
    self.code = self.source.read().decode(self.encoding)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 24871: ordinal not in range(128)

[6bdad4c1-1e26-4f2f-a442-a01a2292c181]

comment:15

Hello,

Many thanks for your help, but I'm afraid I need more. Is there a way to make docbuild tell me where I made a mistake?

[databases] Encoding error:
[databases] 'ascii' codec can't decode byte 0xe2 in position 24871: ordinal not in range(128)
[databases] The full traceback has been saved in /tmp/sphinx-err-YMbReV.log, if you want to report the issue to the developers.

Well, Sphinx says that there is a non-ascii character at position 24871 in some file. Have you checked whether there might be a non-ascii character at position 24871 in your file ?

Nathann

[mantepse]

comment:16

Replying to @nathanncohen:

Well, Sphinx says that there is a non-ascii character at position 24871 in some file. Have you checked whether there might be a non-ascii character at position 24871 in your file ?

Now I found it! It is a long dash in a reference retrieved from FindStat. I have to redo the doctests anyway, because the descriptions, names, references, etc., are all subject to change...

Thank you!

[6bdad4c1-1e26-4f2f-a442-a01a2292c181]

comment:17

Now I found it! It is a long dash in a reference retrieved from FindStat. I have to redo the doctests anyway, because the descriptions, names, references, etc., are all subject to change...

There must be something wrong with dashes. Every single time I get this error message it is because of a dash between the pages in the reference of a paper, i.e. 234--190.

Let's call this message "the dash message" from now on.

Nathann

[VivianePons]

comment:19

To answer a bit what has been said here. I do agree that it would be better to provide a proper API on the FindStat side, that was my idea at first. But, it's not that easy, we're not that many, and nobody did it! On the other hand, Martin wanted the Sage interface because he needed it. And the dirty html parsing was the quickest way to get it. Even if it's not perfect, I still think it could be useful for other people, that's why he created the ticket.

If you want more of a technical reason: at this stage, FindStat is no more than a customized MoinMoin macro. Especially, we don't really deal with the http request: we just take some info from MoinMoin and give some html back to be included in a MoinMoin html page, we have no interface to deal with non html responses.

For similar reasons, I don't see how to include the FindStat search directly into Sage at the moment, it demands lots of work. I'm not saying it's never going to happen, but probably not just right now.

About open-sourceness, the main reasons it's not been open yet are mostly time and technical issues. FindStat is not a software, it's a webservice. In the meantime, we've welcomed anyone wanting to work on the project!

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

9f16a01	add findstat to documentation
d0ef23a	complete redesign, now using FindStat's ability to provide json data

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from d21fad3 to d0ef23a

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

6caecea

fix oversight

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from d0ef23a to 6caecea

[mantepse]

comment:22

I tried (with a lot of help from the FindStat team) to answer the (technical) objections raised above. In case somebody is interested in having this finalized, I need a little help with the following items:

1) documentation and doctests

2) I believe that statistics, collections and maps should have something similar to a unique representation, but I'm not sure how to achieve this, especially for statistics. I should mention that it is quite possible that at some point collections and maps can be edited, too. So the idea is to have caching enabled for objects with id different from 0...

3) Christian told me that I need to do something like lazy import, but I currently don't really know what that is. I simply followed oeis.py...

4) a design question: it appears that, in sage, python properties are hardly used at all. I have no idea why. Currently, the interface uses properties to allow editing statistics (collections and maps currently cannot be edited). This implies some inconsistencies. Therefore:

• should I change read-only properties like FindStatStatistic.id to ordinary methods?
• should I do the same with the other properties, i.e., provide methods FindStatStatistic.description and FindStatStatistic.set_description?

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

9adb2b5

make collections and maps hashable, fix bug for Cores, fix bug in equality for collections

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 6caecea to 9adb2b5

[videlec]

comment:24

Hello,

Looks better now ;-)

Could you use a lazy import in the "all.py". What it means is that you have to replace

from sage.databases.findstat import findstat

with

from sage.misc.lazy_import import lazy_import
lazy_import('sage.databases.findstat', 'findstat')

What it changes is that you will have:

sage: type(findstat)
<type 'sage.misc.lazy_import.LazyImport'>

but mostly nothing else (access to method, tab completion, documentation is identical). The advantage is that it is not imported on startup. It would be really good since you are doing a lot of global imports in your module.

Vincent

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

8817ab3

lazy import findstat

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 9adb2b5 to 8817ab3

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

05a08ed

more doctesting, fix bug in FindStatCollection.__init__

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 8817ab3 to 05a08ed

[videlec]

comment:27

Plenty of documentation issues! Did you try to run make doc?

1. After INPUT:, OUTPUT: there must be a blankline.

2. TODO sections must be indented, i.e.

.. TODO::

    we definitely should do this and that

3. The first line of the documentation should be an imperative and start with an uppercase. Replace

   returns the name of the collection

   with

   Return the name of the collection

   And no point at the end of this first sentence.

4. This is wrong and will raise an error during the creation of the documentation.

   EXAMPLES::
   
   A particular statistic can be called by its St-number or number::

   There should be two colon only if the next block is indented. In that case it should be

   EXAMPLES:
   
   A particular statistic can be called by its St-number or number::

   And in the following you forgot a colon

   To search for a distribution, send a list of lists, or a single pair:

There is a lot of useful information in the developer guide.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 05a08ed to fefc6c2

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

fefc6c2

improving documentation: make doc passes

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

14def60

better formatting and more documentation

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from fefc6c2 to 14def60

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

52330b9

unicode tempfile

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 14def60 to 52330b9

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 52330b9 to 61d0c69

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

7c407bd	Merge git://trac.sagemath.org/sage into public/findstat-interface
fd6b6c3	Merge branch 'public/findstat-interface' of git://trac.sagemath.org/sage into public/findstat-interface
4c48615	Merge branch 'public/findstat-interface' of git://trac.sagemath.org/sage into public/findstat-interface
61d0c69	properly html-escape

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

0702e72

replace properties by setters, improve documentation

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 61d0c69 to 0702e72

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

4ba6ef0

more documentation, bugfix in in_range, findstat accepts FindStatCollection

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 0702e72 to 4ba6ef0

[stumpc5]

comment:35

Hi Martin, thanks for all your work on that!

1. Several doctests currently fail:

sage -t src/sage/databases/findstat.py
**********************************************************************
File "src/sage/databases/findstat.py", line 637, in sage.databases.findstat.FindStatStatistic.id
Failed example:
    findstat(1).id()                              # optional -- internet
Expected:
    <BLANKLINE>
    (process:...): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
    1
Got:
    1
**********************************************************************
File "src/sage/databases/findstat.py", line 719, in sage.databases.findstat.FindStatStatistic.function
Failed example:
    findstat(lambda pi: pi.length(), "Permutations").function() # optional -- internet
Expected:
    <function <lambda> at ...>
Got:
    <BLANKLINE>
    (process:21380): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
    <function <lambda> at 0x7fb420e806e0>
**********************************************************************
File "src/sage/databases/findstat.py", line 1294, in sage.databases.findstat.FindStatCollection.first_terms
Failed example:
    c.first_terms(lambda x: 1, max_values=10))                       # optional -- internet
Exception raised:
    Traceback (most recent call last):
      File "/home/stumpc5/progs/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 488, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/stumpc5/progs/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 848, in compile_and_execute
        compiled = compiler(example)
      File "/home/stumpc5/progs/sage-git/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 486, in <lambda>
        example.source, filename, "single", compileflags, 1)
      File "<doctest sage.databases.findstat.FindStatCollection.first_terms[2]>", line 1
        c.first_terms(lambda x: Integer(1), max_values=Integer(10)))                       # optional -- internet
                                                                   ^
    SyntaxError: invalid syntax
**********************************************************************
File "src/sage/databases/findstat.py", line 1505, in sage.databases.findstat.FindStatMap.id
Failed example:
    m = findstat(lambda pi: pi.length(), "Permutations")[1][1][0] # optional -- internet
Expected nothing
Got:
    <BLANKLINE>
    (process:21392): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
    <BLANKLINE>
    (process:21402): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
**********************************************************************
4 items had failures:
   1 of   4 in sage.databases.findstat.FindStatCollection.first_terms
   1 of   3 in sage.databases.findstat.FindStatMap.id
   1 of   2 in sage.databases.findstat.FindStatStatistic.function
   1 of   2 in sage.databases.findstat.FindStatStatistic.id
    [77 tests, 4 failures, 12.80 s]
----------------------------------------------------------------------
sage -t src/sage/databases/findstat.py  # 4 doctests failed
----------------------------------------------------------------------
Total time for all tests: 13.1 seconds
    cpu time: 5.7 seconds
    cumulative wall time: 12.8 seconds

2. The documentation doesn't build (unless I made a mistake, I haven't been building docs in a while):

Error building the documentation.

Note: incremental documentation builds sometimes cause spurious
error messages. To be certain that these are real errors, run
"make doc-clean" first and try again.
Traceback (most recent call last):
  File "/home/stumpc5/progs/sage-git/src/doc/common/builder.py", line 1618, in <module>
    getattr(get_builder(name), type)()
  File "/home/stumpc5/progs/sage-git/src/doc/common/builder.py", line 503, in _wrapper
    x.get(99999)
  File "/home/stumpc5/progs/sage-git/local/lib/python/multiprocessing/pool.py", line 558, in get
    raise self._value
OSError: [databases] /home/stumpc5/progs/sage-git/local/lib/python2.7/site-packages/sage/databases/findstat.py:docstring of sage.databases.findstat:16: ERROR: Unexpected indentation.

3. I wonder whether it is okay that doctests open browser windows, @videlec, @nathanncohen.

4. How should doctests that change from the FindStat side over time be handled, @videlec, @nathanncohen. One (possibly not the best) example is

    sage: [FindStatCollection(c.first_terms(lambda x: 0, max_values=1)[0][0]) for c in cl]
        [Cc0001: Permutations,
         Cc0002: Integer partitions,
         Cc0005: Dyck paths,
         Cc0006: Integer compositions,
         Cc0007: Standard tableaux,
         Cc0009: Set partitions,
         Cc0010: Binary trees,
         Cc0012: Perfect matchings,
         Cc0013: Cores,
         Cc0014: Posets,
         Cc0017: Alternating sign matrices,
         Cc0018: Gelfand-Tsetlin patterns,
         Cc0019: Semistandard tableaux,
         Cc0020: Graphs,
         Cc0021: Ordered trees,
         Cc0022: Finite Cartan types,
         Cc0023: Parking functions]

[stumpc5]

comment:36

I was trying to fix the doctests, but stumble across the following problem: I constantly get errors such as

Failed example:
    FindStatCollection("Binary trees").name()               # optional -- internet
Expected:
    u'Binary tree'
Got:
    <BLANKLINE>
    (process:23292): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
    u'Binary tree'

and the problem is that this

    <BLANKLINE>
    (process:23292): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed

only happens sometimes (google search tells me this appears to be a firefox issue, but I wouldn't know how to handle this in Sage doctests.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

598ac1e

- Removed the colons of the end of the first doc lines.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 4ba6ef0 to 598ac1e

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 598ac1e to 5b39d53

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

5b39d53

fixed typo in the documentation that make it fail to build

[mantepse]

comment:39

Christian, the last commit is not following the conventions at http://www.sagemath.org/doc/developer/coding_basics.html#the-docstring-of-a-function-content:

It [the one sentence description] must be followed by a blank line and end in a period.

Please revert.

---

New commits:

5b39d53

fixed typo in the documentation that make it fail to build

[mantepse]

comment:40

I'm still having the problem that I cannot run the doctests at all. As I mentioned on sage-devel, I get, for example:

martin@Martin-Laptop:~/sage$ ./sage -t --optional=internet src/sage/databases/findstat.py
too few successful tests, not using stored timings
Running doctests with ID 2015-03-30-10-41-21-fba377bd.
Git branch: public/findstat-interface
Doctesting 1 file.
sage -t src/sage/databases/findstat.py
**********************************************************************
File "src/sage/databases/findstat.py", line 1152, in sage.databases.findstat.FindStatCollection.__init__
Failed example:
    FindStatCollection("Dyck paths")                 # optional -- internet
Exception raised:
    Traceback (most recent call last):
      File "/home/martin/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 488, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/martin/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 850, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.databases.findstat.FindStatCollection.__init__[0]>", line 1, in <module>
        FindStatCollection("Dyck paths")                 # optional -- internet
    NameError: name 'FindStatCollection' is not defined

How can I fix this? FindStatCollection is imported in the preceding line of the docstring!

[stumpc5]

comment:41

Is I wrote some time earlier, you must run

sage -t --optional=sage,internet path/to/interface.sage

If you only run --optional=internet, you are only running those tests having # optional -- internet, so all the imports (and a few others) are not performed.