sage-shell-view-mode no longer working

[TMueller83]

Hi, the sage-shell-view-mode no longer works for me, using sage-shell-mode 20201225.1011, SageMath version 9.3 (Release Date: 2021-05-09), Emacs 27.2 and dvips 2021.1. Is was working before but at some point it stopped. There is no error message produced of any kind in emacs, just no LaTeX-picture overlay is produced and shown.

[supertrianguloid]

Does not work for me either. I have all of the latex packages installed. Emacs 27.2.

[EmmanuelCharpentier]

Do you mean that the inline typesetting does not work ? The rest seems OK (including inline plots...).

[TMueller83]

@EmmanuelCharpentier yes, exactly. The inline typesetting does not work.

[EmmanuelCharpentier]

@EmmanuelCharpentier yes, exactly. The inline typesetting does not work.

I started to look at it. The original author used multithreaded functions to build and display the images. That's a serious pain to debug....

Ssome help from someone more versed in emacs-lisp than I am would be extremely welcome...

[aikrahguzar]

@EmmanuelCharpentier yes, exactly. The inline typesetting does not work.

I started to look at it. The original author used multithreaded functions to build and display the images. That's a serious pain to debug....

Ssome help from someone more versed in emacs-lisp than I am would be extremely welcome...

@EmmanuelCharpentier , I started working on an simpler implementation of sage-mode here https://github.com/aikrahguzar/sage-mode where the idea is to basically offload as much as possible to python-mode and inferior-python-mode. I have got the basic features like completion, documentation lookup and sending code to sage process working.

Displaying latex and images is the next thing I want to do but I have the opposite problem that I don't what is happening on the sage. Is there an overview available somewhere of what is needed to get sage to emit latex and png images? I think I know more or less what to do on emacs side (it should be mostly comint-*-filter-functions). Any pointers are appreciated.

[slotThe]

@EmmanuelCharpentier yes, exactly. The inline typesetting does not work.

I started to look at it. The original author used multithreaded functions to build and display the images. That's a serious pain to debug....

Ssome help from someone more versed in emacs-lisp than I am would be extremely welcome...

The Emacs lisp side is fine; it seems that sage is not sending the "correct" type of the expression to the BackendEmacs anymore.

For example, the following renders correctly:

x = var('x')
u = function('u')(x)
de = desolve(diff(u, x) - x * u + 2 * x^2, [u, x], ics=[0,-1])

from sage.repl.rich_output.output_basic import OutputLatex, OutputPlainText
from emacs_sage_shell_view import BackendEmacs

BackendEmacs().display_immediately(OutputPlainText("pretty printing…"), OutputLatex(latex(de).encode()))

I don't know why, but pretty much every rich_output save when plotting is just reported as OutputPlainText.

[aikrahguzar]

I don't know why, but pretty much every rich_output save when plotting is just reported as OutputPlainText.

You are right and I figured out why: it is due to this line https://github.com/sagemath/sage/blob/c000c953eb6355a93fd4ac43dda4a93e9eab1960/src/sage/repl/rich_output/display_manager.py#L556

You can't get latex unless the backend claims to support OutputHtml. BackendEmacs doesn't because BackIPythonCommandLine doesn't. This seems to works for me (I haven't checked if it causes trouble since I don't use sage-shell-view-node anymore but it prints the latex code as expected),

class BackendEmacs(BackendIPythonCommandline):

    def __init__(self, text=True, plot=True):
        super(BackendEmacs, self).__init__()
        if text:
            self.__text = "latex"
        else:
            self.__text = None
        self.__plot = plot

    def default_preferences(self):
        return DisplayPreferences(text=self.__text)

    def _repr_(self):
        return "Emacs babel"

    def supported_output(self):
        return [OutputLatex , OutputPlainText , OutputHtml , OutputImagePng]

    def displayhook(self, plain_text, rich_output):
        if self.__plot and isinstance(rich_output, OutputImagePng):
            msg = rich_output.png.filename(ext='png')
            msg = "BEGIN_PNG:%s:END_PNG" % msg
            return ({u'text/plain': msg}, {})

        elif isinstance(rich_output, OutputHtml):
            text = "BEGIN_TEXT:" + str(plain_text.text.get(), 'utf-8') + ":END_TEXTBEGIN_LATEX:" + \
                   str(rich_output.latex.get(), 'utf-8') + ":END_LATEX"
            return ({'text/plain': text}, {})

        else:
            return super(BackendEmacs, self).displayhook(plain_text, rich_output)

[EmmanuelCharpentier]

This seems to works for me

Would you propose a patch ?

(I haven't checked if it causes trouble since I don't use sage-shell-view-node anymore

May I inquire why ?

but it prints the latex code as expected),

Do you mean it renders the expected typeset output ?

[aikrahguzar]

Would you propose a patch ?

Yes, I can do that here in a day or two or someone can beat me to it :)

May I inquire why ?

I wanted to change somethings so at one point I started looking at code and I found it hard to figure out what the code was doing. After some time I realized that it was written at a time when the current python-mode was not in emacs so the code had long swathes of code from there so a lot of it could be just removed by depending on the functionality for python built into emacs and the rest of what I wanted was doable quite simply with some comint functionality. So I have less than 500 lines of elisp which more or less do what I want and not much else.

but it prints the latex code as expected),

Do you mean it renders the expected typeset output ?

No, it is the code. It would be the job of a function in comint-output-filter-functions to replace it with the typeset output. I assume sage-shell-view-mode installs such a function but I haven't checked. I think it should work but that needs to be tested. Which is why I think it would be better if someone using the sage-shell-view-mode does the pull request but if not I will get to it eventually.

[apresta]

I got #70 to work by:

1. Adding the missing import as described here: https://github.com/sagemath/sage-shell-mode/pull/70#issuecomment-1500541029
2. Making the following change to sage/repl/rich_output/output_browser.py in the __init__ method of OutputHtml: replace OutputBuffer('$' + latex_string + '$') with OutputBuffer(latex_string).

The issue was that we were getting $ $ delimiters nested inside \begin{math} \end{math}. I'm not sure if this is the correct fix though, since it might break other uses of OutputHtml. I don't understand where the \begin{math} is coming from: the only instance I found is in OutputLatex.inline_equatio, but that's not being called according to my logging.

[EmmanuelCharpentier]

I got https://github.com/sagemath/sage-shell-mode/pull/70 to work by:

[ Snip... ]

Could you submit a "clean" patch ?

[aikrahguzar]

I don't understand where the \begin{math} is coming from: the only instance I found is in OutputLatex.inline_equatio, but that's not being called according to my logging.

It is coming from sage-shell-view-latex-str. I think the input this package expects and what sage produces has diverged and you should add the changes there to #70 to get something working.

[EmmanuelCharpentier]

I got #70 to work by:

1. Adding the missing import as described here: Fix the sage-shell-view-mode #70 (comment)
2. Making the following change to sage/repl/rich_output/output_browser.py in the __init__ method of OutputHtml: replace OutputBuffer('$' + latex_string + '$') with OutputBuffer(latex_string).

No, no, no ! Sagemath has a lot of possible uses, and using its output in sage-shell-mode is but a marginal one. We shouldn't impose our convention.

The issue was that we were getting $ $ delimiters nested inside \begin{math} \end{math}. I'm not sure if this is the correct fix though, since it might break other uses of OutputHtml. I don't understand where the \begin{math} is coming from: the only instance I found is in OutputLatex.inline_equatio, but that's not being called according to my logging.

These spurious dollars can be intercepted in sage-shell-view.el.

Patch follows. If I can figure out how to upload it...

[EmmanuelCharpentier]

Pull request posted.

[EmmanuelCharpentier]

Patch is #72 ...

[apresta]

I got #70 to work by:

1. Adding the missing import as described here: Fix the sage-shell-view-mode #70 (comment)
2. Making the following change to sage/repl/rich_output/output_browser.py in the __init__ method of OutputHtml: replace OutputBuffer('$' + latex_string + '$') with OutputBuffer(latex_string).

No, no, no ! Sagemath has a lot of possible uses, and using its output in sage-shell-mode is but a marginal one. We shouldn't impose our convention.

The issue was that we were getting $ $ delimiters nested inside \begin{math} \end{math}. I'm not sure if this is the correct fix though, since it might break other uses of OutputHtml. I don't understand where the \begin{math} is coming from: the only instance I found is in OutputLatex.inline_equatio, but that's not being called according to my logging.

These spurious dollars can be intercepted in sage-shell-view.el.

Patch follows. If I can figure out how to upload it...

I think you missed #71, which also addressed the issue without affecting Sage. However, I did the opposite than you: kept the output from Sage intact, including the $ delimiters, and removed the math environment wrapping from sage-view. I don't know which of these is preferable.

[BenjaminMoraga]

There isn't a solution to this problem yet?

[apresta]

There isn't a solution to this problem yet?

The solution is ready for merging: https://github.com/sagemath/sage-shell-mode/pull/71

I'm not sure if there are any active maintainers watching this repository though.

[EmmanuelCharpentier]

@apresta's fix was somehow dropped after I reviewed it about 2 weeks ago. I reintegrated it (and did some cosmetic fix of "too wide" docstrings. Please review #74 before I merge it.