can't run drepl-ipython

[pank]

Hi,

I have installed v0.3 from ELPA on windows using Anaconda python v3.11 or v3.12.

I wanted to try this project as I miss comint when using emacs-jupyter.

However, the python process stops before drepl is started:

Python 3.11.5 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:26:23) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.15.0 -- An enhanced Interactive Python. Type '?' for help.

Do you really want to exit ([y]/n)? 

Process dnatplot/*IPython* finished

I don't fully understand the initialization process, but it seems drepl--init calls python -c "import sys; exec(''.join(sys.stdin)); Drepl.instance().mainloop()" via the ipython drepl--command. drepl-ipython.py is then sent as stdin (I'm curious why Drepl.instance().mainloop() couldn't just be run if __name__ == '__main__'?).

The log isn't very revealing either:

[11:42:21] starting dnatplot/*IPython*
[11:42:23] read {"op": "getoptions"}
[11:42:23] send msg {"id":1,"op":"setoptions","prompts":["In [{}]: ","...: ","\u001B[31mOut[{}]:\u001B[0m ","\n",""]}
[11:42:23] read {"op": "status", "status": "ready"}
[11:42:23] read {"op": "status", "status": "rawio"}

It seems that the EOFError exception in the Drepl.mainloop method is triggered. when I change the default to 'n' in the exit question, the error get triggered continuously.

I'm not quite sure how to debug this further. Any ideas?

[astoff]

Okay, I'm pretty sure this is related to the fact you're using Windows. Certainly it's necessary that the IPython subprocess runs in a PTY as opposed to via pipes. I guess you can check that by running M-: (process-tty-name (get-buffer-process (current-buffer))) RET in the dREPL buffer.

You could modify the way initialization works, e.g. by just calling python path/to/drepl-ipython.py. The disadvantage (and reason for the more contrived initalization) is that this will not work over Tramp, since drepl-ipython.py is not present in the remote machine.

If we are able to confirm that the contrived initalization cannot work on Windows, then I would add a simpler, Tramp-incompatible workaround for Windons only.

I'm curious why Drepl.instance().mainloop() couldn't just be run if __name__ == '__main__'?

I guess you're right ;-).

[pank]

I can run python -i drepl-ipython.py and it starts. Or in emacs I can eval (setq python-shell-interpreter "python" python-shell-interpreter-args "-i c:/users/me/.emacs.d/elpa/drepl-0.3/drepl-ipython.py") and then use run-python. I don't think I associate it with drepl after. In any case (process-tty-name (get-buffer-process (current-buffer))) returns nil in this buffer.

When I modify the drepl-startup:

(cl-defmethod drepl--command ((_ drepl-ipython))
  `(,python-interpreter "-i"
                        "c:/users/me/.emacs.d/elpa/drepl-0.3/drepl-ipython.py"))

I get funny errors about indentation errors. I don't see any. I tried to change the file encoding and the line endings to the silly ones (windows ones), but that didn't fix the issue.

>>> 'IPython interface for dREPL.'
>>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> ... ... ... ... >>> >>> ... ... ... ... ... ... >>> >>> >>> ... ... >>> >>> ... ... ... ... ... ... ... ... ... ... >>> >>> ... ... >>> >>> ... ... ... ... ... ... >>>   File "<stdin>", line 1
    def write_format_data(self, format_dict, md_dict=None) -> None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    for mime, handler in self.shell.mime_renderers.items():
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if mime in format_dict:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    handler(format_dict[mime], None)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    return
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    super().write_format_data(format_dict, md_dict)
IndentationError: unexpected indent
>>> >>> >>> ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... >>>   File "<stdin>", line 1
    system = InteractiveShell.system_raw
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    displayhook_class = DreplDisplayHook
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def make_mime_renderer(self, type, encoder):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    def renderer(data, meta=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if encoder:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    data = encoder(data)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    header = json.dumps({**(meta or {}), "type": type})
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if len(data) > self.mime_size_limit:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    fdesc, fname = mkstemp()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    with open(fdesc, "wb") as f:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    f.write(data)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    payload = "tmp" + Path(fname).as_uri()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    else:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    payload = base64.encodebytes(data).decode()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print(f"\033]5151;{header}\n{payload}\033\\")
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    return renderer
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def enable_mime_rendering(self, mime_types=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    """Enable rendering of the given mime types; if None, enable all."""
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if mime_types is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    mime_types = MIME_TYPES
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    for t in mime_types:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if t in MIME_TYPES:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.display_formatter.formatters[t].enabled = True
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def ask_exit(self):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.keep_running = False
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def enable_gui(self, gui=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if gui != "inline":
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print("Can't enable this GUI: {}".format(gui))
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def mainloop(self):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    while self.keep_running:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    try:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.run_once()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    except EOFError:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(op="status", status="rawio")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if (not self.confirm_exit) or self.ask_yes_no(
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    "Do you really want to exit ([y]/n)?", "y", "n"
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    ):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.ask_exit()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    except (DreplError, KeyboardInterrupt) as e:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print(str(e) or e.__class__.__name__)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def run_once(self):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    "Print prompt, run REPL until a new prompt is needed."
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if self.current_ps1 is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(op="getoptions")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.current_ps1, separate_in = "", ""
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    else:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    separate_in = self.separate_in if self.current_ps1 else ""
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.current_ps1 = sys.ps1.format(self.execution_count)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print(separate_in + self.current_ps1, end="")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    while True:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    data = readmsg()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    op = data.pop("op")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    fun = getattr(self, "drepl_{}".format(op), None)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if fun is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    raise DreplError("Invalid op: {}".format(op))
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    fun(**data)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if op == "eval":
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.execution_count += 1
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    break
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if op == "setoptions":
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    break
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_eval(self, id, code):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(op="status", status="rawio")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    r = self.run_cell(code)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_complete(self, id, code, pos):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    with provisionalcompleter():
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    completions = rectify_completions(code, self.Completer.completions(code, pos))
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    first = next(completions, None)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if first is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    return
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    prefix = code[first.start: pos]
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    completions = chain([first], completions)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    candidates = [{"text": c.text, "annot": c.signature} for c in completions]
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id, prefix=prefix, candidates=candidates)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_checkinput(self, id, code):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    status, indent = self.check_complete(code)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    prompt = sys.ps2.format(self.execution_count).rjust(len(self.current_ps1))
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id, status=status, indent=indent, prompt=prompt)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_describe(self, id, code, pos):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    name = token_at_cursor(code, pos)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    try:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    info = self.object_inspect(name)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    defn = info["definition"]
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    id=id,
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    name=info["name"],
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    type=" ".join(defn.split()) if defn else info["type_name"],
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    file=info["file"],
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    text=self.object_inspect_text(name),
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    )
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    except Exception:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_setoptions(self, id, prompts=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if prompts:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sys.ps1, sys.ps2, sys.ps3, self.separate_in, self.separate_out = prompts
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>> 

[astoff]

Okay, python -i is not the right way to do it. What you would need is:

1. Add the if __name__ == "__main__" trick to drepl-ipython.py (which is a change I just pushed to Github)
2. Change the following 2 methods:

(cl-defmethod drepl--command ((_ drepl-ipython))
  `(,python-interpreter ,drepl-ipython--start-file))

(cl-defmethod drepl--init ((repl drepl-ipython))
  (cl-call-next-method repl)
  (drepl--adapt-comint-to-mode ".py")
  (push '("5151" . comint-mime-osc-handler) ansi-osc-handlers))

Again, this is what one would need to do to run the process over pipes (as opposed to a PTY). But ideally I would like to find out why the PTY solution is not working on Windows.

[pank]

I will try this on Monday when I turn on my windows pc again.

[pank]

With the new versions of drepl-ipython.{py, el} and the two changes above, I can run drepl-ipython. Completions and plots work as expected (based on very brief testing only).

[astoff]

Okay, thanks for bringing to my attention that Windows doesn't have PTYs. I still need to think how to best work around this limitation.

[astoff]

@pank If you would like to, feel free to test this branch: https://github.com/astoff/drepl/pull/5