vimcat is much slower than vanilla nvim

GordianDziwis commented 5 years ago

Open a file in vim takes less than a second, using vimcat in a terminal takes about 9s to "scroll" to the end of the file. vimcat -o - .vimrc >> test takes about 7s.

This is a problem, because I would like to use vimcat as a previewer for ranger. Ranger calls vimcat like this:

# wraps highlight to treat exit code 141 (killed by SIGPIPE) as success
safepipe() { "$@"; test $? = 0 -o $? = 141; }
try safepipe vimcat -u ~/.dotfiles/.vimpagerrc -o - "$path" && { dump | trim; exit 5; }

Does anyone has an idea?

rkitover commented 5 years ago

I will try to speed it up.

nkh commented 2 years ago

@rkitover any progress?

Also an explanation of why you think it's slow would be very helpful.

rkitover commented 2 years ago

I'm sorry I haven't been doing much work on this repo lately.

I need to speed up the streaming system, but it is essential for viewing large files or it would hang at the beginning.

I can do two things here easily:

Check file size, this would not work with pipeline input. Well, perhaps it could if it buffers a chunk at the beginning and it's smaller than that chunk.
Add a --no-stream option to disable streaming.

nkh commented 2 years ago

I played a bit with this today:

script -o 100000 -qfc "vim -R --cmd 'set shortmess+=sSF t_ti= t_te=' +redraw +q "

Blasing fast but there's a few things to fix:

displays a single page
running it in fzf sometimes gave me short pages (24 lines)
the short message that's at the bottom of the display is displayed at the top, nothing perl or sed can't fix
some spurious message like "Top" added

I also tried grabbing the buffer via tmux, didn't go so well.

rkitover commented 2 years ago

The main problem with this approach is like you said, it's only the first page. vimcat can also output a single page very fast, especially if I set the first chunk to something bigger or turn off streaming.

Your approach may also work if you add some vimscript to page downwards through the whole file.

rkitover commented 2 years ago

Actually, now that I think about it, making the first chunk say, 10k, would mostly fix this problem, what do you think?

rkitover commented 2 years ago

I tried setting the chunk size to 1MiB, but it does not make it any faster.

What do you find annoying about it? It seems pretty fast to me, other than the fraction of a second pause at the very beginning.

nkh commented 2 years ago

I need to show a lot of file previews, lots. vimcat takes eons, 18 seconds to get 287 lines of bash, that a test I just ran. It could be that I have something wrong in my installation but then it was wrong on my last 3 computers. cCould you try and let me know what you get for time? https://raw.githubusercontent.com/nkh/ftl/main/ftl

I'd be delighted to use vimcat if it was faster, I used to alias it to cat but I've become impatient with age ;) I certainly hope it will be fast in the future so I can cat with the same colors as my editor, that's much more ergonomic.

As for the short code I posted above, It doesn't even list a whole page in an fzf preview, the spurious elements are a nuisance, speed is just fine. I'll have to spend more time to make it work properly, hopefully you beat me to it and vimcat becomes a speed cat.

rkitover commented 2 years ago

For your ftl script, I get:

real    0m6,996s
user    0m7,285s
sys     0m0,378s

. If I run vimcat on vimpager, which is a 1045 line script, I get:

real    0m2,538s
user    0m3,147s
sys     0m0,391s

. I suspect that the difference here is because your ftl script is extremely syntax-heavy, and either vim highlighting in general or the ANSI encoding I do becomes the bottleneck due to the number of syntax elements. Still, I get 7 seconds not 18.

If it would be possible to get your method to work, that would cut-out the ANSI re-encoding at least, which is done with vimscript, which is not exactly blazing fast.

nkh commented 1 year ago

https://www.reddit.com/r/vim/comments/vzs658/vimkat_a_rewrite_to_speedup_vimcat/

@rkitover, here's what I have, not polished yet but on its way there. I'm a bit surprised by the time difference between your vimcat run and mine (7s vs 18s), could it be that we use different versions of vimpager/vim? I get 0.5s with vimkat.

@BonaBeavis, maybe this fixes your problem too

rkitover commented 1 year ago

@nkh Awesome, I'll take a look.

rkitover / vimpager

vimcat is much slower than vanilla nvim #239