REPL hangs for a bit after pasting

[hummy123]

Hi all. This is a cool project I’ve found helpful for incremental programming in OCaml. Appreciate the effort that’s gone into it.

I’m currently working on a B-Tree implementation in OCaml (very early - just about 150 lines) and sometimes I copy and paste the code into Utop (running in GNOME Terminal) and experience some hanging for a minute or two while the clipboard’s contents are being fully pasted.

I’m not super familiar with terminal programming (do terminal emulators process one character input at a time?) but I thought perhaps I might be uniquely positioned to help.

I haven’t had a good look at utop’s internals, but I think both it and lambda-term depend on Zed for text edition which isn’t as performant as it could be. (I developed a functional and immutable text buffer based on the one in VS Code which is an order of a magnitude faster - and one of the other text buffers written in OCaml is much faster than Zed too.)

So I was wondering if this project (or would it be better to put this issue in the lambda-term repository?) would accept a pull request that offered a bridge between Zed’s public API and my library that would lead to minimal changes on utop/lambda-term’s code.

I might be totally wrong about Zed being the bottleneck when pasting and my offer might reasonably be rejected as it could look like self-promotion (not my intent) so I thought it was best to ask first.

[tmattio]

Hi! @hummy123! It's very interesting, thanks for starting the discussions!

It would be interesting to see the performance comparison. I'm wondering if we could somehow first validate that Zed is the performance bottleneck here, to avoid having you spend time on this to realise the bottleneck was somewhere else.

I also vaguely remember discussions about this: could we simply change the terminal mode when we intercept Ctrl+C, so we copy the clipboard as a whole instead of characters by characters?

[emillon]

could we simply change the terminal mode when we intercept Ctrl+C, so we copy the clipboard as a whole instead of characters by characters?

I think that the correct solution to that is to implement bracketed paste in utop (basically disabling the input handling while pasting).

[emillon]

(see https://github.com/dbuenzli/down/issues/16 for example)

[hummy123]

After @tmattio mentioned validating, I did some raw text buffer benchmarks (without the overhead of a front-end like lambda-term) which involved inserting each character from the blibo.ml file one-by-one (like how the terminal processes them), and I can confirm Zed's not the bottleneck for the mentioned case.

The difference was only 0.00100 ms at this level (Zed is actually 0.00100 ms faster in this example) so it seems likely that the lack of bracketed paste (as @emillon mentioned) is the bottleneck and not the library used.

Happy if my issue might one day lead to utop being faster at pastes (my knowledge of how terminals/terminal emulators work internally is pretty close to 0 so not sure I can be much help).

[tmattio]

Great, thanks a lot for running the benchmarks @hummy123!