Automatically built catalogs

nthiery commented 10 years ago

Sage's catalogs (of groups, monoids, algebras, crystals, ...) are very handy. However it takes quite some time to maintain them, and this won't scale in the long run: we would want to have similar catalogs for Hopf Algebras, rings, coxeter groups, ... with proper inclusions between the catalogs. A natural setting would be that of categories. Namely to be able to do something like:

    sage: catalog = CoxeterGroups().catalog()
    sage: catalog.<tab>
    WeylGroup
    SymmetricGroup
    ...

The core feature is to be able to retrieve all the parents implemented in Sage that belong to a given category.

Florent wrote a prototype of this during the Sage days in Edinburgh in January 2013. It instruments TestSuite, so that running the test suite builds a database of all parent implementations, with their default category. Then, from this category, it's easy to recover all parents in a category by reverse lookup.

CC: @hivert @tscrim @sagetrac-sage-combinat

Component: categories

Issue created by migration from https://trac.sagemath.org/ticket/17219

nthiery commented 10 years ago

comment:1

Florent: could you post your code here?

vbraun commented 10 years ago

comment:2

Not every parent is necessarily intended for the end user, there are base classes and / or internal implementations. Whats the plan for dealing with that? Making it simpler to have a catalog is of course good, but IMHO it requires human judgement for what to include.

6bdad4c1-1e26-4f2f-a442-a01a2292c181 commented 10 years ago

comment:3

Automatically built catalogs "for algebraic structures". You will not make this work with hypergraphs., designs nor graphs/digraphs, there are no categories nor testsuites there.

By the way there is a thing that I do not understand: do you actually intend to make the doctests, though the calls to testsuite, actually fill Sage's database of catalogs ? Does that mean that the doctests will actually have an effect on the code ?

Or is it only to detect that the catalog already contains what is expected to be present ?

Nathann

nthiery commented 10 years ago

comment:4

Replying to @vbraun:

Not every parent is necessarily intended for the end user, there are base classes and / or internal implementations. Whats the plan for dealing with that?

A priori no specific plan: the purpose is to be exhaustive. Something like, for e.g. abstract classes the Java standard library API, exhaustive lists of "all implementations".

Making it simpler to have a catalog is of course good, but IMHO it requires human judgement for what to include.

Definitely: nothing beats a human to highlight the important classes (although we could dream of a system that would automatically gather data from Sage's usage by all users, and automatically learn from that, but that's for later :-)). So, indeed, where we have the manpower, it's good to have manually built indexes. But they could focus on the piece requiring human judgement, and refer to the automaticaly generated ones for exhaustivity. And everywhere else the automatic index are better than nothing.

nthiery commented 10 years ago

comment:5

Replying to @nathanncohen:

Automatically built catalogs "for algebraic structures". You will not make this work with hypergraphs., designs nor graphs/digraphs, there are no categories nor testsuites there.

Well, arguably there should be TestSuite's for graphs and friends :-) Other than that, it should be easy to let the infrastructure also gather data for certain abstract classes (e.g. GenericGraph) in addition to categories.

That being said if the current workflow for building and maintaining the catalogs for graphs and friends is deemed good enough, we don't have to change it!

By the way there is a thing that I do not understand: do you actually intend to make the doctests, though the calls to testsuite, actually fill Sage's database of catalogs ? Does that mean that the doctests will actually have an effect on the code ?

Or is it only to detect that the catalog already contains what is expected to be present ?

Well, I sure would not want every user to have to run the tests to have a full featured Sage :-)

So I am thinking of a two stage process, like for the pickle database:

When Sage would be put in a certain mode, TestSuite would be instrumented to gather the data and fill up the database needed for the automatic catalogs. This could typically be done as an additional step of the release process. Of course this needs to be automatized to not add burden to the release manager; and the specifics of this need to be settled.
This database would be shipped with the Sage distribution, be it in source or binary form, typically as an spkg or something similar.

Cheers, Nicolas

sagemath / sage

Automatically built catalogs #17219