Julia: Evaluating something with huge output causes lag spice

[theplatters]

Hello, Thank you again for the quick fix of the last problem. I found another one :) :

Minimal working example: Evaluating rand(10000) causes a nvim to lag

This is especially a problem when working with huge dataframes, because they can freeze nvim completly.

[Olical]

So normally this is due to the treesitter part of Neovim trying to parse the massive output. Clojure will truncate large output, so will some other languages, it's worth checking if you can set Julia to also only print a portion of the data if you don't absolutely need all of it in your buffer, maybe writing it to a file on disk if you do.

The other thing you can try is setting both of these, especially the tree sitter one:

https://github.com/Olical/conjure/blob/c562d1bc938205feaca1339e0502506533d76f8d/doc/conjure.txt#L685-L695

That will greatly reduce the CPU load of the log when printing very large structures. So if you find yourself doing that a lot in one particular project it might be worth it. You then may struggle to evaluate things from within the log buffer without using visual selection depending on the language (only lisps really work without tree sitter for historical reasons), but it might be fine in your use case.

Conjure doesn't do anything with the output, it basically just takes the lines and feeds them into the buffer with no extra processing, so any performance issues are due to Neovim trying to work it's magic on the output and make it editable for you. So there's not much I can do from my side I don't think, there's not really code to optimise as far as I can tell.

Although there may be some string operations we could improve if the output is one MASSIVE string instead of many lines. I still suspect 95% of the time will be spent on tree sitter parsing though, that'd be a great thing to prove, not sure if Neovim can give us performance diagnostics for that sort of thing.

[Olical]

Interestingly, I can run this command no problem and see no performance drop. I do see some performance issues for a couple of seconds after I run it a few times. I have 13k lines in my log buffer now and it took maybe 1-2 seconds when I entered the buffer for treesitter to parse things, then the rest of the operations were lag free.

I'm using a ThinkPad X1 with a 12th Gen Intel(R) Core(TM) i7-1255U though, is this just a case of "works on my machine" because of my CPU and anything less than that really struggles?

[Olical]

Saying that, I did manage to get things to lock up while trying to show that it was working okay. So maybe there's some race condition here. Something is choking on all those lines being outputted.

I've printed FAR more data out of a Clojure REPL without this much lag, so I suspect this is to do with the stdio nature of the Julia client and something going wrong when many lines are streamed out quickly.

[Olical]

Captured the lags on camera here I think: https://asciinema.org/a/pmIRT0yyaSheYrvQZksBXUleM

[theplatters]

Hm this could be a thing with LazyVim and Baleila or something. I will look into this, thanks.