improve FriCAS interface

[mantepse]

The FriCAS interface is currently very rudimentary. In particular, converting the results of a computation into sage types is available only in very few special cases.

Depends on #21209

CC: @sagetrac-bpage @hemmecke @dkrenn @dimpase

Component: interfaces

Keywords: FriCAS

Author: Martin Rubey

Branch/Commit: e281742

Reviewer: Bill Page, Emmanuel Charpentier

Issue created by migration from https://trac.sagemath.org/ticket/21231

[mantepse]

comment:1

1) implement a method that (reliably!) yields a tuple (type, 1d algebra output, 2d algebra output, error message, etc) as strings. 1d algebra output should come in "unparsed inputform" as one very long string, I'd say. 2d algebra output should be empty if 1d output is available.

It should be able to deal with very long lines, too, possibly be using file io. This involves use of ioHook and some regexps.

2) implement a method that translates fricas output into sage types.

I think this could work as follows: we want to map fricas types to sage constructors, possibly recursively. Recall that what's really sent to fricas is something like "sage23 := [x<sup>n/y</sup>n for n in 1..3]", so that sage can access the result using the variable "sage23". This string is referred to as self._name

Now what I propose is a method which takes a parsed type, e.g., "Integer" or ("List", "Integer") or ("UnivariatePolynomial", "x", ("Fraction", "Integer"))

def to_sage(type):
    if type == "Integer":
         return to_sage_integer()
    elif type == "String":
         return to_sage_string()
    ...
    elif isinstance(type, tuple):
        if type[0] == "List":
            return to_sage_list(type[1])
        elif type[0] == "Fraction":
            return to_sage_fraction(type[1])
        ....

and so on.

So, if the type is ("List" ("Fraction" ("UnivariatePolynomial" "x" "Integer"))), this method would call

    to_sage_list(("Fraction" ("UnivariatePolynomial" "x" "Integer"))

which might be something like

def to_sage_list(self, type):
    n = fricas.eval("#" + self._name).to_sage_integer()
    return [fricas.eval(self._name + ".%n").to_sage(type) for n in range(1,n+1)]

[mantepse]

Commit: 5b7e6ab

[mantepse]

Branch: /u/mantepse/21231

[mantepse]

New commits:

5fa5c04	Merge branch 'u/mantepse/generating_function_in_findstat_interface' of git://trac.sagemath.org/sage into develop
4b555c5	Merge branch 'develop' of git://github.com/sagemath/sage into develop
17f7dbe	Merge branch 'u/dkrenn/16137' of git://trac.sagemath.org/sage into develop
08494a2	Merge branch 'u/vdelecroix/16137' of git://trac.sagemath.org/sage into develop
f077211	Merge branch 'develop' of git://github.com/sagemath/sage into develop
5b7e6ab	initial version of new fricas interface

[mantepse]

comment:4

This is very beta, but I'd be very happy to receive some comments.

[mantepse]

Author: Martin Rubey

[mantepse]

Changed commit from 5b7e6ab to d8a26e1

[mantepse]

New commits:

d8a26e1

Merge remote-tracking branch 'origin/develop' into fricas-interface

[vbraun]

Changed branch from /u/mantepse/21231 to u/mantepse/21231

[vbraun]

comment:6

Branch name doesn't have leading slash

---

New commits:

b2ee8cc

make it build

[vbraun]

Changed commit from d8a26e1 to b2ee8cc

[mantepse]

comment:7

Replying to @vbraun:

Branch name doesn't have leading slash

Howto? I used

git push --set-upstream trac HEAD:u/mantepse/21231

[dimpase]

comment:8

Replying to @mantepse:

Replying to @vbraun:

Branch name doesn't have leading slash

Howto? I used

git push --set-upstream trac HEAD:u/mantepse/21231

This is fine, this does not do anything to the webpage contents. You made a typo, this leading slash, while editing the latter.

[dimpase]

Dependencies: #21209

[mantepse]

comment:10

(it actually doesn't depend on #21209, since it works with any installation of FriCAS but never mind)

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from b2ee8cc to cbcb94f

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

cbcb94f

quick dirty fix for bug reported by Bill Page, add idea for better type handling

[6d95573c-b4ed-47d1-8c51-cb633c034739]

Sage worksheet under 7.2

[6d95573c-b4ed-47d1-8c51-cb633c034739]

Attachment: sage-7.2-fricas.pdf.gz

Attachment: sage-7.4.beta0-fricas.pdf.gz

Sage worksheet under 7.4.beta0 (typeset output fails)

[6d95573c-b4ed-47d1-8c51-cb633c034739]

comment:12

In Sage Worksheet typeset output or FriCAS object fails with this patch on 7.4.beta0. See attached pdf files.

https://github.com/sagemath/sage-prod/files/10658907/sage-7.2-fricas.pdf.gz (typeset output OK)

https://github.com/sagemath/sage-prod/files/10658908/sage-7.4.beta0-fricas.pdf.gz (typeset output fails)

I am not sure whether or not this example worked under 7.4.beta0 without the patch.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from cbcb94f to 5b4a95d

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

5b4a95d

switch to better type treatment

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 5b4a95d to 0ab0f92

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

0ab0f92

further improve translation to sage objects, make restart work, fix doctests

[mantepse]

comment:15

I have a question about the _fricas_init_ and _fricas_ methods defined in various files, in particular in ~/sage-develop/src/sage/rings/rational_field.py and in ~/sage-develop/src/sage/rings/real_mpfr.pyx, see below.

The question is: the doctests work without adapting the methods. What are the methods supposed to do?

~/sage-develop/src/sage/rings/rational_field.py

    def _axiom_init_(self):
        r"""
        Return the axiom/fricas representation of `\QQ`.

        EXAMPLES::

           sage: axiom(QQ)    #optional - axiom # indirect doctest
           Fraction Integer
           sage: fricas(QQ)   #optional - fricas # indirect doctest
           Fraction(Integer)

        """
        return 'Fraction Integer'

    _fricas_init_ = _axiom_init_

in ~/sage-develop/src/sage/rings/real_mpfr.pyx

    def _axiom_(self, axiom):
        """
        Return ``self`` as a floating point number in Axiom.

        EXAMPLES::

            sage: R = RealField(100)
            sage: R(pi)
            3.1415926535897932384626433833
            sage: axiom(R(pi))  # optional - axiom # indirect doctest
            3.1415926535 8979323846 26433833
            sage: fricas(R(pi)) # optional - fricas
            3.1415926535_8979323846_26433833

        """
        prec = self.parent().prec()

        #Set the precision in Axiom
        old_prec = axiom('precision(%s)$Float'%prec)
        res = axiom('%s :: Float'%self.exact_rational())
        axiom.eval('precision(%s)$Float'%old_prec)

        return res

    _fricas_ = _axiom_

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 0ab0f92 to d2d0fb8

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

d2d0fb8

fix tab completion

[mantepse]

comment:17

The once missing thing is the error handling. I was expecting that raising an error in eval would be sufficient, as indicated by the patch below. However, if I then do fricas("something stupid"), no error is raised. What am I doing wrong?

--- a/src/sage/interfaces/fricas.py
+++ b/src/sage/interfaces/fricas.py
@@ -385,8 +385,7 @@ class FriCAS(ExtraTabCompletion, Expect):
                 return "\r\n".join(line[FRICAS_MULTI_LINE_START:] for line in lines)

         else:
-            print(output)
-            return
+            raise RuntimeError("FriCAS has not recognized '%s' as a valid operation: %s" % (code, output))

     def set(self, var, value):
         """Set the variable var to the given value.

[6d95573c-b4ed-47d1-8c51-cb633c034739]

comment:18

Calling fricas("something stupid") ends up in fricas.set which calls expect._eval_line not fricas.eval. This interface stuff is truly twisted. Check especially the inheritance. FriCAS inherits from Expect that inherits Interface. Check __call__ and _create in Interface and go from there.

fricas.py should probably override _eval_line in the same way as axiom.py.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from d2d0fb8 to 148bf77

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

148bf77

better to ask for forgiveness than permission in `__getitem__`, some series examples, raise error in eval

[6d95573c-b4ed-47d1-8c51-cb633c034739]

comment:20

fricas.py used to inherit some functions from axiom.py. The new version no longer uses any part of the Axiom interface. One of the casualties of this change was _latex_ from PanAxiomElement. The function _latex_ is responsible for producing LaTeX output in typeset mode. The following patch incorporates the _latex_ function from axiom.py and corrects the problem reported above in #21231 comment:12

diff --git a/src/sage/interfaces/fricas.py b/src/sage/interfaces/fricas.py
index 8831c6c..5c1bc64 100644
--- a/src/sage/interfaces/fricas.py
+++ b/src/sage/interfaces/fricas.py
@@ -201,6 +201,7 @@ from __future__ import print_function

 from sage.interfaces.tab_completion import ExtraTabCompletion
 from sage.interfaces.expect import Expect, ExpectElement, FunctionElement, ExpectFunction
+from sage.misc.multireplace import multiple_replace
 from sage.env import DOT_SAGE
 import re
 import six
@@ -571,6 +572,31 @@ class FriCASElement(ExpectElement):
             raise IndexError("index out of range")
         return self.elt(n+1)

+    def _latex_(self):
+        r"""
+        EXAMPLES::
+
+            sage: a = fricas(1/2) #optional - fricas
+            sage: latex(a)       #optional - fricas
+             1 \over 2
+
+        """
+        self._check_valid()
+        P = self.parent()
+        s = P._eval_line('outputAsTex(%s)'%self.name())
+        if not '$$' in s:
+            raise RuntimeError("Error texing axiom object.")
+        i = s.find('$$')
+        j = s.rfind('$$')
+        s = s[i+2:j]
+        s = multiple_replace({'\r':'', '\n':' ',
+                              ' \\sp ':'^',
+                              '\\arcsin ':'\\sin^{-1} ',
+                              '\\arccos ':'\\cos^{-1} ',
+                              '\\arctan ':'\\tan^{-1} '},
+            re.sub(r'\\leqno\(.*?\)','',s)) # no eq number!
+        return s
+
     def __int__(self):
         return int(self.sage())

I am not sure of the relevance of the particular substitutions that were performed by the old axiom.py code.

Sorry for just including a patch but I am not sure whether or not I could have just pushed this change to trac since this branch seems to be private.

[6d95573c-b4ed-47d1-8c51-cb633c034739]

comment:21

FriCAS uses

sage: fricas('asin(x)')
asin(x)
sage: fricas('acos(x)')
acos(x)
sage: fricas('atan(x)')
atan(x)

Sage replaces asin, acos and atan with arcsin, arccos and arctan

sage: asin(x)
arcsin(x)
sage: acos(x)
arccos(x)
sage: atan(x)
arctan(x)

But FriCAS does not use these names so the following results in an error:

sage: fricas(asin(x))
...
   There are no library operations named arcsin 
      Use HyperDoc Browse or issue
                                                                                                                   )what op arctan
      to learn if there is any operation containing " arctan " in its name.
 ...
sage: fricas(acos(x))
...
   There are no library operations named arccos 
      Use HyperDoc Browse or issue
                                                                                                                   )what op arctan
      to learn if there is any operation containing " arctan " in its name.
 ...
sage: fricas(atan(x))
...
   There are no library operations named arctan 
      Use HyperDoc Browse or issue
                                                                                                                   )what op arctan
      to learn if there is any operation containing " arctan " in its name.
 ...

[mantepse]

comment:22

Sorry for just including a patch but I am not sure whether or not I could have just pushed this change to trac since this branch seems to be private.

Putting patches here is perfect for me, actually! Many thanks!

[dimpase]

comment:23

Replying to @mantepse:

Sorry for just including a patch but I am not sure whether or not I could have just pushed this change to trac since this branch seems to be private.

Putting patches here is perfect for me, actually! Many thanks!

although you could have just pushed your own (i.e. u/bpage) or public (i.e. public/) branch; then your commit(s) could be cherry-picked. An advantage is that it's no error-prone cutting/pasting, (and it is known whom to blame :-)).

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 148bf77 to ca3277c

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

ca3277c

fix eval

[6d95573c-b4ed-47d1-8c51-cb633c034739]

comment:26

Typeset output from FriCAS in the notebook is not working. Apparently the most recent commit is still missing _latex_.

[6d95573c-b4ed-47d1-8c51-cb633c034739]

comment:27

I think the markers |startKeyedMsg| and |endOfKeyedMsg| look strange in the error message and traceback. In the notebook only |endOfKeyedMsg| is shown as the output until clicking to expand the traceback. These markers should probably be removed from the output before displaying the message to the user.

[mantepse]

comment:28

Thanks for reminding me of LaTeX, I'm embarassed - I forgot!

Removing |startKeyedMsg| and |endOfKeyedMsg| and the like is not completely trivial, but I'll try!

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from ca3277c to b5f5a1c

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

b5f5a1c

fix latex, translate Factored, conversions for expressions

[6d95573c-b4ed-47d1-8c51-cb633c034739]

comment:31

Thanks, Martin. Here is a small patch that fixes a few problems with FriCAS-generated LaTeX. For example:

sage: latex(fricas("integrate(sin(x+1/x),x)"))
 \int ^{\displaystyle x} {{\sin  \left( {{{{{ \%A} ^{2}}+1} \over \%A}}  \right)} \  {d \%A}}   

and also subscripts and subscripts in a few other cases.

diff --git a/src/sage/interfaces/fricas.py b/src/sage/interfaces/fricas.py
index 8d2ac3f..594d5b8 100644
--- a/src/sage/interfaces/fricas.py
+++ b/src/sage/interfaces/fricas.py
@@ -598,7 +598,10 @@ class FriCASElement(ExpectElement):
         j = s.rfind('$$')
         s = s[i+2:j]
         s = multiple_replace({'\r':'', '\n':' ',
-                              ' \\sp ':'^',
+                              '\\sp ':'^',
+                              '\\sp{':'^{',
+                              '\\sb ':'_',
+                              '\\sb{':'_{',
                               '\\arcsin ':'\\sin^{-1} ',
                               '\\arccos ':'\\cos^{-1} ',
                               '\\arctan ':'\\tan^{-1} '},

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from b5f5a1c to ccb26de

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

ccb26de

Merge branch 'develop' of git://trac.sagemath.org/sage into fricas-interface

[mantepse]

comment:34

Hi Bill,

I don't understand your patch: why would you replace \sp{ by ^{ if you replace \sp by ^ anyway. Also, isn't \sp valid LaTeX?

But more importantly: shouldn't this patch really applied to FriCAS itself?

[hemmecke]

comment:35

Maybe tex.spad from my mathjax branch might be interesting. It spits out true latex. https://github.com/hemmecke/fricas/commits/mathjax But maybe you really want mathjax or even 1d output of FriCAS objects, then look into mathjax.spad or 1d.spad. Most importantly, the output form can be changed at runtime.

Note that these files are GPL3+ and are therefore not (yet) part of FriCAS.

[mantepse]

comment:36

Hi Bill and Ralf!

I'd like to get this ticket in soon (and in particular, stop working on it very soon). So if FriCAS produces only TeX currently, I'd like to leave it at this. As soon as the new tex.spad is in FriCAS, we can pick it up here, OK?

[mantepse]

comment:37

                               '\\arcsin ':'\\sin^{-1} ',
                               '\\arccos ':'\\cos^{-1} ',
                               '\\arctan ':'\\tan^{-1} '},

another question: why are you replacing \arctan and friends? As far as I know, this is perfectyl valid LaTeX, no?

[hemmecke]

comment:38

Yes, it is. And given the usual meaning of sin²(x) = (sin(x))² and not sin(sin(x)), I even think that writing sin^(-1) for arcsin is even confusing, no?