ECL memory exhaustion when reading expression

[rwst]

This on a 32GB machine after about six minutes:

sage: var('a b c d', domain='real')
(a, b, c, d)
sage: ex = 6/17*(b*x + a)^(3/2)*(d*x + c)^(4/3)/d + 108/935*3^(3/4)*(b*c - a*d)^3*((d*x + c)^(1/3)*(-b/(b*c - a*d))^(1/3) + 1)*sqrt(-sqrt(3) + 2)*sqrt(((d*x + c)^(2/3)*(-b/(b*c - a*d))^(2/3) - (d*x + c)^(1/3)*(-b/(b*c - a*d))^(1/3) + 1)/((d*x + c)^(1/3)*(-b/(b*c - a*d))^(1/3) - sqrt(3) + 1)^2)*elliptic_f(arcsin(((d*x + c)^(1/3)*(-b/(b*c - a*d))^(1/3) + sqrt(3) + 1)/((d*x + c)^(1/3)*(-b/(b*c - a*d))^(1/3) - sqrt(3) + 1)), 4*sqrt(3) - 7)/(sqrt(a + (d*x + c)*b/d - b*c/d)*b*d^3*(-b/(b*c - a*d))^(1/3)*sqrt(-((d*x + c)^(1/3)*(-b/(b*c - a*d))^(1/3) + 1)/((d*x + c)^(1/3)*(-b/(b*c - a*d))^(1/3) - sqrt(3) + 1)^2)) - 54/187*(b*c - a*d)*sqrt(b*x + a)*(d*x + c)^(4/3)/d^2 + 162/935*(b*c - a*d)^2*sqrt(b*x + a)*(d*x + c)^(1/3)/(b*d^2)
sage: ex.diff(x).simplify_full()

Condition of type: STORAGE-EXHAUSTED
Memory limit reached. Please jump to an outer pointer, quit program and enlarge the
memory limits before executing the program again.
Available restarts:

1. (CONTINUE) Extend heap size

Top level.
> 

Interrupting after a few minutes:

/home/ralf/sage/local/lib/python2.7/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression.simplify_full (build/cythonized/sage/symbolic/expression.cpp:54950)()
   9925         """
   9926         x = self
-> 9927         x = x.simplify_factorial()
   9928         x = x.simplify_rectform()
   9929         x = x.simplify_trig()

/home/ralf/sage/local/lib/python2.7/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression.simplify_factorial (build/cythonized/sage/symbolic/expression.cpp:58690)()
  10405 
  10406         """
> 10407         return self.parent()(self._maxima_().makefact().factcomb().minfactorial())
  10408 
  10409     factorial_simplify = simplify_factorial

/home/ralf/sage/local/lib/python2.7/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression._maxima_ (build/cythonized/sage/symbolic/expression.cpp:8032)()
    799             # Maybe not such a great idea because the "default" interface is another one
    800             from sage.calculus.calculus import maxima
--> 801             return super(Expression, self)._interface_(maxima)
    802         else:
    803             return super(Expression, self)._interface_(session)

/home/ralf/sage/local/lib/python2.7/site-packages/sage/structure/sage_object.pyx in sage.structure.sage_object.SageObject._interface_ (build/cythonized/sage/structure/sage_object.c:5894)()
    741             except Exception:
    742                 raise NotImplementedError("coercion of object %s to %s not implemented:\n%s\n%s" % (repr(self), I))
--> 743         X = I(s)
    744         if c:
    745             try:

/home/ralf/sage/local/lib/python2.7/site-packages/sage/interfaces/interface.pyc in __call__(self, x, name)
    278 
    279         if isinstance(x, string_types):
--> 280             return cls(self, x, name=name)
    281         try:
    282             return self._coerce_from_special_method(x)

/home/ralf/sage/local/lib/python2.7/site-packages/sage/interfaces/interface.pyc in __init__(self, parent, value, is_name, name)
    693                 self._name = parent._create(value, name=name)
    694             except (TypeError, RuntimeError, ValueError) as x:
--> 695                 raise TypeError(x)
    696 
    697     def _latex_(self):

Component: symbolics

Issue created by migration from https://trac.sagemath.org/ticket/25789

[rwst]

comment:1

The workaround would be to not use simplify_full but the specific simplify_ functions that have an effect with this expression.

[rwst]

comment:2

Well no. With simplify_rectform or just simplify I get the same. So this happens in the interface / conversion.

[nbruin]

comment:3

You may be running into the limitations of string-based interface conversions. We can do better with maxima_lib and this may be an example where this really pays off:

from sage.interfaces.maxima_lib import  max_to_sr, sr_to_max
dex=ex.diff(x)
f=maxima_calculus(sr_to_max(dex))
fr=f.rectform()
g=max_to_sr(fr.ecl())

seems to work OK (I didn't wait for the last command to finish, though). It's still very slow, though, because while max_to_sr and sr_to_max will convert expression-trees directly, the max_to_sr one will use the standard function constructors for many of the functions (we have nothing else in sage), which may well call into maxima again for simplification operations (which is what we're just coming from, so no benefit will be obtained). Also, some of those calls could end up calling maxima via the string interface again.

The expression-tree conversion tools have been available for ages. They are used in some specific spots (integration, limits, etc.), but the need didn't seem to be there to use it everywhere, and it's a bit of a chore to go through all the calculus/sr machinery to change the way conversions to maxima_lib are done. There used to be a slight advantage that sticking with strings would allow us to move back to an expect-based maxima for calculus, in case ecl would be abandoned. I don't think we need to consider that anymore.

UPDATE: it looks like the result in fr is rather big. Just printing it already causes the error to happen, so indeed trying to convert it from maxima to sage via producing and then parsing a string will fail. It seems that converting the expression through max_to_sr is also quite challenging, but it did complete using less than 6Gb of resident memory. It's not clear to me anything useful can be done with the expression afterwards...