utop with emacs

[RogerTarani]

utop works perfectly in shell mode (Debian Stretch). Within emacs, I'm fine with tuareg/merlin/auto-complete. But I can't have utop up&running there.

M-x utop
utop command line: opam config exec -- utop -emac

error in process filter: pcase--u1: Unknown upattern `(quote wait)'
error in process filter: Unknown upattern `(quote wait)'

And the utop buffer shows no more than the following, accepting no input:

stdout:  #require "package";;      to load a package
stdout:  #list;;                   to list the available packages
stdout:  #camlp4o;;                to load camlp4 (standard syntax)
stdout:  #camlp4r;;                to load camlp4 (revised syntax)
stdout:  #predicates "p,q,...";;   to set these predicates
stdout:  Topfind.reset();;         to force that packages will be reloaded
stdout:  #thread;;                 to enable threads
stdout:

When I use the configuration from RWO Dev, I get another error and utop not working.

It looks like the utop page is telling something different from the RWO Dev page. Anyway, I may be tired and confused.

I've used instructions from http://dev.realworldocaml.org/install.html and https://github.com/ocaml-community/utop. Can you pls. tell the recommended ~/.emacs configuration to get utop workin within emacs and other required cofiguration? (in addition to have utop installed correctly via opam, and working via the shell). Thanks

Pls find herafter the part of my ~/.emacs regarding utop:

$ cat  ~/.emacs 

;; -- opam and utop setup --------------------------------
;; Setup environment variables using opam
(dolist
   (var (car (read-from-string
           (shell-command-to-string "opam config env --sexp"))))
 (setenv (car var) (cadr var)))

;; Update the emacs path
(setq exec-path (split-string (getenv "PATH") path-separator))

;; Update the emacs load path
(push (concat (getenv "OCAML_TOPLEVEL_PATH")
          "/../../share/emacs/site-lisp") load-path)

;; Automatically load utop.el
(autoload 'utop "utop" "Toplevel for OCaml" t)
;; the following from RWO dev/install looks like deprecated, so it's commented out
;;(autoload 'utop-setup-ocaml-buffer "utop" "Toplevel for OCaml" t)
;;(add-hook 'tuareg-mode-hook 'utop-setup-ocaml-buffer)
;; the following was added following https://github.com/ocaml-community/utop#integration-with-emacs
(autoload 'utop-minor-mode "utop" "Minor mode for utop" t)
(add-hook 'tuareg-mode-hook 'utop-minor-mode)
;; to avoid manually activate this mode with 
;; M-x utop-minor-mode

;; Use the opam installed utop
(setq utop-command "opam config exec -- utop -emacs")

[ghost]

This code was written a long time ago, it's possible that the emacs API has changed since then. The error about (quote wait) is suspicious.

[rgrinberg]

What is your version of Emacs?

[RogerTarani]

24.5.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.11) of 2017-09-12 on hullmann, modified byDebian

I recently heard about Spacemacs. Is it more reliable or usable than Emacs?

[rgrinberg]

Would you mind upgrading to the latest stable version of Emacs? Or at the very least to Emacs 25. I'm not sure if the problem is Emacs 24, but it would make it easier for us to narrow things down.

[RogerTarani]

I upgraded Emacs to 26.1 and could immediately launch utop without error messages and with the same ~/.emacs file: M-x utop

Welcome to utop version 2.2.0 (using OCaml version 4.06.1)! ... utop[0]>

However I've not the menu bar (yet) in the bottom as in shell mode. And the following message is not displayed as in shell mode:

Type #utop_help for help about using utop.

From there, is there a specific setup to be fully up&running with utop in Emacs?

AFAIK, utop has history and can save to a file all valid expressions, and is an "enhanced toplevel for OCaml". But is it possible to use utop features in an Emacs buffer with a .ml file, an ask for evaluation of a phrase/region/buffer or is utop just designed for writing expressions and get instant their evaluation? (i.e. a toplevel, as in shell mode)

[ghost]

utop behaves a bit differently in emacs, I didn't know how to display the completion bar so I just used normal emacs completion mechanism.

Have you tried the utop-minor-mode? https://github.com/ocaml-community/utop#usage-1

[RogerTarani]

In utop/Emacs : the TAB key does the the job. But the possible modules/functions appear in another buffer that is kept visible/busy for that purpose in one case described hereafter: Typing Lis TAB gives List Adding .me gives List.me , thenTAB shows the completions possibilities in a buffer : clicking on mem gives List.mem and the completion buffer disappears. Ok. completing manually to get List.mem then pressing TAB and the completion buffer disappears. Ok. but completing manually List.mem then pressing SPC let the completion buffer opened and clicking on mem in it adds even a useless mem in utop: List.mem mem So I can forget using SPC there, but is it normal? If not, how can it be fixed?

Other solution : is it possible to enable AC (auto-complete) and keep the same behaviour as with tuareg/merlin/AC ? (where I use SHIFT TAB to get completion popup menu when needed, with their values and types).

Also, in utop in Emacs I don't have the scrolling history as in the shell (with UP&DOWN arrows). When using tuareg mode, I simply do CTRL UP/DOWN.

Also, in utop in Emacs, when pressing UP arrow, the prompt goes up somewhere in the upper lines and it's no more possible to enter an expression. The only ways I found to solve that are: clicking somewhere to the right of the Utop prompt ; or pressing ENTER to get the cursor just under the Utop prompt, then entering text is possible (one line too low but it works, except if you do LEFT ARROW). How is it possible to control the behaviour of the cursor ?

Yes, I tried utop-minor-mode (M-x utop-minor-mode) C-c C-s (Start a utop buffer) works. C-x C-e (Evaluate the current phrase) works. If there is ;; at the end of the expression, utop returns the expression and the evaluated value ; without ;; it returns only the value. In both cases, I get the message "Symbol’s function definition is void: tuareg--after-double-colon)" C-x C-r (Evaluate the selected region) works. C-c C-b (Evaluate the current buffer) doesn't work if there is ;; at the end of the last expression (utop parses the code and underline the double semicolon ;; ) (all ;; at the end of all previous expressions have no impact : utop gives alls the evaluated expressions, even the last one. C-c C-k works (Kill a running utop process)

Is this useful for you to improve utop, or to help me fine tuning this utop/Emacs setup? Thanks

[ghost]

From your description, it seems to me that the emacs integration is working as originally intended. AFAIK, there is no plan to improve the emacs mode.

[RogerTarani]

No utop history in Emacs? It means that I must type again an expression or copy-paste it (copy, AND move cursor to the right of utop prompt (using DOWN key or mouse), then paste). Imagine if I have 50 expressions! This seems not possible utop has been designed that way.

Before closing this issue, can you give me/us a clue about how to fix/improve the cursor issue and history issue (and possibly about the "last ;;" issue which is not blocking). It should be inside the utop.el file. Thanks

[ghost]

There is history support. I can't remember the keybindings, you can do C-h b to see the current keybindings and look for history. Regarding the cursor, you can rebind UP to the history function.

I don't follow the issue about the double semicolon. It would be easier to understand with examples of inputs and the expected behaviour.

[pmetzger]

Regardless, I think:

1. We should update the documentation to say "please use Emacs 24 or higher"
2. If @RogerTarani wishes, he can (and should!) submit improvements to the Emacs mode for us to consider.
3. If (1) happens we can close this trouble report.

[RogerTarani]

You are right : there is history support (M-n and M-p) which is enough. I can rebind UP/DOWN instead if desired.

Regarding the "double semicolon in the last expression/phrase" issue: You can try C-c C-b (Evaluate the current buffer) with the following code:

let a = 1;;
let b = 2;; (* Not Ok *)

let a = 1;; let b = 2;;

;; (underlined and red double semicolon)

let a = 1
let b = 2;; (* Not Ok *)

let a = 1;; let b = 2;;

;; (underlined and red double semicolon)

let a = 1;;
let b = 2

let a = 1;; let b = 2

;; val a : int = 1 val b : int = 2

[RogerTarani]

@pmetzger

1. Emacs 24.5.1 didn't work with utop for me (with the ~/.emacs file herebefore) and Emacs 26.1 worked. And I didn't try Emacs 25.x (as suggested by @rgrinberg) which maybe fine too. I've just tried to setup Emacs 25.1 (on my Debian Stretch) in order to make a test with utop but the compilation failed. (I used http://ftp.gnu.org/gnu/emacs/emacs-25.1.tar.gz. I had no problem compiling 26.1). Can you try utop with Emacs 25.1 ? So the documentation should be "please use Emacs 26 or higher" or "please use Emacs 25.x or higher" once we have tested it.

2. Yes. But do you mean improvement specification or improvement implementation? (I can only do little things in Elisp! )

[RogerTarani]

Here is the code I added in ~/.emacs to get the Up/Down arrows for scrolling utop history (and keep M-n M-p):

;; utop history additional key binding: <up> and <down> arrows (and keep M-n M-p)
(require 'utop-minor-mode)
  (define-key utop-mode-map (kbd "<up>") 'utop-history-goto-prev )
  (define-key utop-mode-map (kbd "<down>") 'utop-history-goto-next )

or if you prefer a simpler Elisp syntax for keys:

  (define-key utop-mode-map [<up>] 'utop-history-goto-prev )
  (define-key utop-mode-map [<down>] 'utop-history-goto-next )

[shakthimaan]

I am using GNU Emacs 26.3 with opam (2.0.5), utop (2.4.3), merlin (3.3.3), merlin-eldoc and tuareg (2.1.0-1), and I am getting the following error with C-x C-e, as mentioned:

Debugger entered--Lisp error: (void-function tuareg--after-double-colon)
  (tuareg--after-double-colon)
  (let* ((pos (tuareg--after-double-colon)) (pos (if pos pos (point))) (phrase (tuareg-discover-phrase pos))) (if phrase (progn (goto-char (car (cdr (cdr phrase)))) (tuareg--skip-double-colon) (tuareg-skip-blank-and-comments))))
  utop-tuareg-next-phrase()
  funcall(utop-tuareg-next-phrase)
  utop-compat-next-phrase-beginning()
  funcall(utop-compat-next-phrase-beginning)
  (progn (goto-char end) (funcall utop-next-phrase-beginning))
  (if utop-skip-after-eval-phrase (progn (goto-char end) (funcall utop-next-phrase-beginning)))
  (let ((end)) (save-excursion (let ((triple (funcall utop-discover-phrase))) (setq end (nth 2 triple)) (utop-eval (nth 0 triple) (nth 1 triple)))) (if utop-skip-after-eval-phrase (progn (goto-char end) (funcall utop-next-phrase-beginning))))
  utop-eval-phrase()
  funcall-interactively(utop-eval-phrase)
  call-interactively(utop-eval-phrase nil nil)
  command-execute(utop-eval-phrase)

The line in test.ml contains - print_string "Hello, World!" and it executes fine, but, the error message pops up. My Emacs init.el setup for OCaml and RWOv2 is as follows:

  ;; Automatically load utop.el
  (setq utop-command "opam config exec -- utop -emacs")
  (autoload 'utop "utop" "Toplevel for OCaml" t)

  ;; -- opam and utop setup --------------------------------
  ;; Setup environment variables using opam
  (dolist
     (var (car (read-from-string
           (shell-command-to-string "opam config env --sexp"))))
    (setenv (car var) (cadr var)))

  ;; Update the emacs path
  (setq exec-path (split-string (getenv "PATH") path-separator))

  ;; Update the emacs load path
   (push (concat (getenv "OCAML_TOPLEVEL_PATH")
            "/../../share/emacs/site-lisp") load-path)

  (require 'tuareg)
  (setq auto-mode-alist
        (append '(("\\.ml[ily]?$" . tuareg-mode)
                  ("\\.topml$" . tuareg-mode))
                auto-mode-alist))

  (autoload 'utop-minor-mode "utop" "Minor mode for utop" t)
  (add-hook 'tuareg-mode-hook 'utop-minor-mode)

  (add-hook 'tuareg-mode-hook 'merlin-mode)
  (setq merlin-use-auto-complete-mode t)
  (setq merlin-error-after-save nil)

  (require 'merlin-eldoc)
  (add-hook 'tuareg-mode-hook 'merlin-eldoc-setup)

Any suggestions?

[ghost]

I'd say that utop.el is calling non-public functions of tuareg.el.

@Chris00 is there a way to move to the next OCaml phrase via emacs lisp in tuareg mode? By looking at the git history, it looks like in the past utop.el was using tuareg-next-phrase, then this function disappeared and someone copy&pasted it utop, but now it's calling tuareg--after-double-colon that doesn't seem to be available anymore.

[bbatsov]

I've been playing with OCaml lately and I happen to know a lot more about Elisp than about the OCaml, so I'll try to help with utop.el. It seems to me it needs a lot of love.

@monnier Any idea what replaced tuareg-next-phrase in Tuareg?

[bbatsov]

One more thing - based on the comments earlier it seems that utop.el probably doesn't work on Emacs 24 & 25. Given that they are ancient at this point perhaps we should update the Emacs requirement to Emacs 26, which would also allow us to simplify and improve the code a bit.

[monnier]

I've been playing with OCaml lately and I happen to know a lot more about Elisp than about the OCaml, so I'll try to help with utop.el. It seems to me it needs a lot of love.

[ Side comment: I see in https://ocaml.org/news/about-utop it says

  AFAIK at the time I wrote it the UTop mode was the only one where it
  was really impossible to edit the something in the frozen part of
  the buffer.

but comint-prompt-read-only was added back in 2004 (i.e. long before utop existed, AFAICT), so I wonder if it's just that the var is not well advertized or that there is/was a bug in comint-prompt-read-only, or that comint-prompt-read-only doesn't cover all the parts that utop-mode covers? ]

@monnier Any idea what replaced tuareg-next-phrase in Tuareg?

Good question. I could never figure out exactly what tuareg-next-phrase was supposed to do, so I don't have a clear answer for that.

The fundamental tools that are provided now to cover "similar" functionality are basically (SMIE-aware) forward-sexp, and beginning/end-of-defun. You can take a look at tuareg-region-of-defun (used by tuareg-discover-phrase) to see what kind of mess is involved :-(

[bbatsov]

but comint-prompt-read-only was added back in 2004 (i.e. long before utop existed, AFAICT), so I wonder if it's just that the var is not well advertized or that there is/was a bug in comint-prompt-read-only, or that comint-prompt-read-only doesn't cover all the parts that utop-mode covers? ]

Yeah, I saw this and I was wondering about the very same thing. I'm still fairly new to the utop.el codebase, so it's hard to comment whether the current approach has any meaningful benefits over just using comint. My experience with comint for inf-clojure was mostly positive, though.

So far the main advantage for me that utop.el has over the ocaml top-level in Tuareg is that it supports autocompletion.

The fundamental tools that are provided now to cover "similar" functionality are basically (SMIE-aware) forward-sexp, and beginning/end-of-defun. You can take a look at tuareg-region-of-defun (used by tuareg-discover-phrase) to see what kind of mess is involved :-(

I figured as much. I'll take a closer look and I'll figure something out.

[monnier]

Bozhidar Batsov [2022-07-17 10:50:34] wrote:

Yeah, I saw this and I was wondering about the very same thing. I'm still fairly new to the utop.el codebase, so it's hard to comment whether the current approach has any meaningful benefits over just using comint. My experience with comint for inf-clojure was mostly positive, though.

I don't know either. comint.el has served us fairly well, but it's old and it shows. In some sense, it's sad that it's just not bad enough (yet?) to justify a rewrite.

The fundamental tools that are provided now to cover "similar" functionality are basically (SMIE-aware) forward-sexp, and beginning/end-of-defun. You can take a look at tuareg-region-of-defun (used by tuareg-discover-phrase) to see what kind of mess is involved :-(

I figured as much. I'll take a closer look and I'll figure something out.

Feel free to ask me for help on specific concrete situations (IIUC it takes time to develop an intuition for SMIE's navigation). [ Probably better via direct email, tho. ]