Global module organization

[GoogleCodeExporter]

The current module organization in SymPy is far from optimal. The "core" is
too big (several hundred kilobytes of code, much unrelated functionality
thrown in one place), and some closely related functions are spread out
across several modules.

I suggest a small set of top-level packages similar to the following (the
details can certainly be changed):

core
 * general classes like Basic, Function, Symbol, MutableCompositeDict
 * miscellaneous non-math utilities (memoization, etc)

logic
 * logical operators
 * quantifier elimination
 * set operations

arithmetic
 * number types
 * Add and Mul
 * essential ops for numbers: Inequality, Max, Min, Abs, ...
 * numerical sets (Reals, ...) and ranges

algebra
 * polynomials
 * equation solving
 * algebraic simplify/rewrite rules
 * matrices/linear algebra
 * number theory

calculus
 * limits
 * series expansion
 * integrals
 * summation
 * differential equations

functions
 * transcendental and other special functions
 * transcendental simplify/rewrite rules

apps
 * geometry
 * physics
 * statistics
 * other "end-use" modules that are not used in an integral way by other
parts of SymPy

interface
 * plotting
 * printing
 * parsing

This needs to be discussed in the context of the main SymPy project, of
course. I'm posting here as the structure I'm suggesting would entail
changes to the "core" part of the SymPy, and we could try it out right away
in sympycore.

In particular, the current sympycore core would be split into core, logic,
arithmetic, and interface. This seems like a reasonable split; the current
sympycore core divides fairly evenly into those sections.

Original issue reported on code.google.com by fredrik....@gmail.com on 22 Oct 2007 at 9:42

[GoogleCodeExporter]

Seems more clean. But a disadvantage is that then the notion of a "core" will
disappear. Sympycore will be then some kind of a subset of sympy.

Original comment by ondrej.c...@gmail.com on 22 Oct 2007 at 9:48

[GoogleCodeExporter]

I think the core is a smallest subset that the rest of modules (all of them, or 
at
least a vast majority of them) depend on. So I think core could be 
"core+arithmetic".
On the other hand, given the fact that currently we don't know how to make all 
the
modules above independent of each other, sympycore could be split into those 4
packages you suggest.

Original comment by ondrej.c...@gmail.com on 22 Oct 2007 at 9:54

[GoogleCodeExporter]

I think if core would include "arithmetic" then it should
also include "logic". The ranges, for instance, are
subclasses of sets.

We can use relative imports that is available
starting at Python version 2.5. This will allow making
many core modules as separate packages.

I think this separation is good also for ending up
in a minimal dependence relations between modules -
if one module needs a feature from another module then it
must use a well-defined interface:

- we already have a good mechanism for determining the information
  of an instance type without the need to import the module that
  defines the instance type. Compare

    * obj.is_Add                 (faster)
    * isinstance(obj, Basic.Add)

  versus

    * from add import Add
      isinstance(obj, Add)

- new types can define special methods, say `try_<some operation>`, so
  that core algorithms will know how to deal with new types without
  explicitely coding the support for a new type in core.

  For example,

    Union(obj1, obj2)

  calls

    obj1.try_union(obj2)

  in canonize method. obj1 may be a Range object that is defined in
  some other module and one does not to have update generic Union
  code to add a support for a new set type.

Original comment by pearu.peterson on 23 Oct 2007 at 7:59

[GoogleCodeExporter]

Operations that cannot be implemented directly on a class can be implemented
modularly by keeping a list rules. For example, an Expand function can be 
defined
like this:

  class Expand:
      rules = []

      @classmethod
      def add_rule(cls, f, name, default):
          cls.rules.append((f, name, default))

      def __call__(self, expr, **kwargs):
          for f, name, default in self.rules:
              if kwargs.get(name, default):
                  expr = f(expr)
          return expr

The arithmetic module defines:

  def expand_powers(expr): ...

  Expand.add_rule(expand_powers, "powers", default=True)

trigonometric.py defines:

  def expand_trig(expr):
      # pseudocode
      if expr == sin(x+y):
        return sin(x)*cos(y) + cos(x)*sin(y)
      return expr

  Expand.add_rule(expand_trig, "trig", default=False)

etc.

Other operations that can be implented in a similar way include Simplify, Solve,
Integral, zero testing...

There is a problem handling the order in which the rules are applied, but that 
can be
solved in a number of ways.

Original comment by fredrik....@gmail.com on 23 Oct 2007 at 1:31

[GoogleCodeExporter]

I have now moved set and logical expressions support from core to
separate packages:

* sympy.logic.symbolic
  - defines: Boolean, DummyBoolean, Predicate, And, Or, Xor, Not, Implies, Equiv

* sympy.logic.sets
  - defines: SetSymbol, Set, SetFunction, Union, Intersection, Minus, Complementary,
EmptySet, UniversalSet, Element, Subset

* sympy.arithmetic.sets
  - defines: ComplexSet, RealSet, IntegerSet, RationalSet, PrimeSet, Positive,
Negative, Divisible, Shifted, Range, RangeOO, RangeOC, RangeCO, RangeCC

The next step is to move Add, Mul, numbers under sympy.arithmetic.
Btw, number theory should be under arithmetic as well.

Original comment by pearu.peterson on 24 Oct 2007 at 12:05

[GoogleCodeExporter]

I'd rather keep exp, log, etc in sympy.functions and not under arithmetic. In 
the
context of the arithmetic package, they are only needed for series and complex 
parts,
right?

Original comment by fredrik....@gmail.com on 26 Oct 2007 at 12:28

[GoogleCodeExporter]

Exp and Log are also needed in evaluating Float ** Float.

I don't acctually mind where they are implemented. It just was more
natural to but them under arithmetic.functions due to Float power.

Original comment by pearu.peterson on 26 Oct 2007 at 2:34

[GoogleCodeExporter]

Float ** Float should only need mpmath's internal exp and log.

Original comment by fredrik....@gmail.com on 26 Oct 2007 at 2:51

[GoogleCodeExporter]

Yes, that's right. So we can switch back to using sympy.functions.

Original comment by pearu.peterson on 26 Oct 2007 at 2:57