[Feature request] Support texlab language server

krishnakumarg1984 commented 5 years ago

I had lots of fun using vimtex for writing a paper and my PhD thesis :). Thank you for developing and maintaining this modern plugin.

It is 2019 and Language servers are here! VSCode seems to be having a runaway success due to sharing common origins at the same company that formulated the nicely documented, open & editor-agnostic language server protocol (LSP).

Servers

The idea is that all aspects that are specific to a particular programming language are left to a dedicated language server. A compatible client (i.e. a text editor) can simply ask questions to the server and populate its UI with the response received. Various common tasks such as completions, hover-text, and go-to-definitions are thus offloaded to the server. Though not all language servers are mature yet, the well-documented, open specification means that these get continuous refinements from the community. The official set of language servers for various programming languages is available here.

Clients (for vim)

For vim, the language server capability (i.e. the client functionality that understands LSP lingo) is provided by a couple of plugins, notably languageclient-neovim (python) and vim-lsp (vimscript). A pull-request to bring in native LSP functionality to neovim is fairly mature & scheduled to land for v0.5.

LaTeX-specific information

The officially supported language server for LaTeX is TexLab which also handles BibTeX. I have had a positive experience using it in VSCode for which there exists a client plugin bearing the same name (TexLab).

Question

Can vimtex provide support for (i.e. play nicely with) TexLab? Already the modular codebase & design goals of vimtex instruct to use dedicated plugins for tasks such as completion, folding and reference doc. Instead of using a variety of plugins, using a language server might be a really good idea.

I am not sure what vimtex needs to do to support LSP. At the least, this possibility should be mentioned in the documentation (but only after testing).

lervag commented 5 years ago

Thank you for developing and maintaining this modern plugin.

I'm happy you like it! :)

Can vimtex provide support for (i.e. play nicely with) TexLab?

I do not quite see how. That is, vimtex is not a language server client. My initial thought is that it does not make sense for vimtex to adapt/utilize TexLab because there already exists mature LSP clients for Vim that should be able to support TexLab already.

I agree that it may be relevant to mention TexLab and perhaps some information about how to use it with Vim in the docs. But I have not used it myself and I have quite limited time. As such, I will gladly review and accept PRs for this, but I will probably not do anything myself in any time soon.

krishnakumarg1984 commented 5 years ago

@lervag Okay here are my thoughts.

How about vimtex provides a user option vimtex_override_with_lsp which is default set to 0 but the user can explicitly set it to 1 if using LSP.

Now, this option is just a shortcut for disabling the features set by the LSP (to avoid conflict).

As far as I am aware, LSPs won't ever provide 80-90% of the stuff that vimtex currently handles (correctly handling tex root file, folding, setting tex engines and options, LaTeX specific text objects etc.), so I see that using vimtex is an absolute must i.e LSP is not a replacement for vimtex.

IMHO, an LSP complements vimtex's strengths with only a small overlap. My request here is to implement a design whereby vimtex acts merely like a "passthrough" to the LSP for majority of the features. Being a modular codebase, I think this isn't as difficult as it initially appears.

lervag commented 5 years ago

You raise a fair point, and I can agree to adding such an option. But as said, I do not have the time to investigate this in detail myself. If anyone would be able/willing to provide a reproducible example of how to setup Vim with vimtex, an LSP plugin and the TexLab server, and with that a way to reproduce overlap issues, then I could implement a robust version if this option.

Note that vimtex does already provide a lot of g:vimtex_..._enabled options that allows to disable various parts.

teto commented 5 years ago

Looks like texlab is being rewritten in rust so let's wait and see what features get available as it's very immature. As LSP providers improve, vimtex could lose some weight (i.e, lines of code) :)

lervag commented 5 years ago

If (and when) other plugins/providers are made available that renders some of vimtexs features deprecated, then I'm just glad. I would be happy to make vimtex work well alongside LSP plugins.

In the meantime, I'm closing this as there is nothing to do right now.

clason commented 5 years ago

Out of curiosity, I spent some time yesterday playing around with this. TL;DR: it works, offers some neat functions that vimtex doesn't, doesn't do many things that vimtex does, and where there is overlap, vimtex does it better.

If anyone else wants to play with it (or is just interested in what it does), here's more details:

Setup

First, download the binary from https://github.com/latex-lsp/texlab/releases and put it somewhere in your path.

Next, install a LSP client. I used prabirshrestha/vim-lsp for this (there's also natebosch/vim-lsc, but that worked less well for me). It's a pure VimL implementation, so all you have to do is to add the following to your .vimrc:

Plug 'prabirshrestha/async.vim'
Plug 'prabirshrestha/vim-lsp'
au User lsp_setup call lsp#register_server({
        \ 'name': 'texlab',   
        \ 'cmd': {server_info->['PATH_TO_texlab']},
        \ 'whitelist': ['bib','tex'],
        \ })
set hidden

The language server should now start automatically as soon as you open a tex or bib file.

(Alternatives would be neoclide/coc.nvim and autozimu/LanguageClient, which are more full-featured but need a binary install. If anybody has tested this and gotten better results than I report below, I'd be interested to hear about it.)

LSP-only features

You can then do the following things, which may or may not be useful:

Go to definition (LspDefinition) of labels and citations under the cursor. There is also a PeekDefinition that shows the location in a floating window.
Conversely, you can list all occurrences (LspReferences) of a label under the cursor in the quickfix list. This also works in a bib file. (You can also step through occurrences with LspNextReference.)
Get a preview of citations and equations (LspHover) under the cursor. (Citations give you a formatted text, labels a formatted reference (Lemma 1.10, Theorem 2 (Hahn-Banach), Equation (4.2) -- the number requires an aux file, otherwise you just get Lemma, Theorem (Hahn-Banach), Equation). If the editor supports markdown (like VS Code), equations instead get a rendered TeX preview as an image.)
Rename (LspRename) all occurrences of a label or citations under the cursor (think: \eqref{eq1} to \eqref{eq:boundedness} or \cite{c1} to \cite{Clason:2011} -- yes, in the bib file too)
It also does linting (using virtual text if available) for bib (but apparently not tex) files.
Highlight groups for errors, warnings, informations, hints (for bibtex only, I think). It also highlights all references of a symbol under the cursor (seems more useful for actual programming languages, though).
Autoformat bib files (not just indentation, but also capitalization of field names etc.); not tex files yet, though. The possibilities for the latter with proper text transformations would be phenomenal, though (think: replacing \section{A}\label{sec:A} with
```
\section{A}%
\label{sec:A}
```
everywhere.)

I should stress that none of this relies on aux files, so it works even if the project hasn't been compiled yet.

LSP and vimtex features

The biggest overlap that I noticed was completion; you can hook the language server up to omnicomplete via :set omnifunc=lsp#complete (it's not enabled by default). It seems a lot faster than the native VimL from vimtex, but it doesn't have the fuzzy matching (match on equation/theorem label, citation authors/title), which may be part of the reason. On the other hand, it doesn't require aux files, either. The pum entries are also not styled like vimtex's (although that seems possible in part; the language server provides the full formatted citation as a preview for completion, which vim-lsp however doesn't support yet.)

So the LSP has quite some way to go to be on par with vimtex feature-wise (which is a pretty high bar, admittedly).

But it's possible to hook up the language server to autocomplete (using, e.g., ncm2-vim-lsp) for speed and leave vimtex as manual omnicomplete provider for features to get the best of both worlds.

The language server also can (automatically) trigger compilation via latexmk on save. It currently doesn't have any callbacks, though.

krishnakumarg1984 commented 5 years ago

@clason That's awesome! Have you tried to see if it plays well with coc-vimtex, which is an extension to the LSP client coc.nvim? That's a neat little extension written specifically to support the sources from vimtex, which should get rid of most of your Doesn't work yet problems.

clason commented 5 years ago

No; as I wrote, I don't want to mess with binary installs. I only tried pure VimL clients. (EDIT: I did try LanguageClient-neovim in the meantime; it works, but the experience with vim-lsp was smoother.) But if you can check coc, please report your experiences!

(And some of these problems also occur with the vscode extension, so they are definitely texlab bugs. EDIT: I've opened issues about them. EDIT2: And most of them have been fixed.)

lervag commented 5 years ago

I'm happy to see more information about this. I would be happy to document this somehow. If any of you would be willing and able to suggest something, that would be very good. I'm thinking this might earn a spot at the top of vimtex-non-features, at least a small text there, perhaps with a link to a section where we can guide users on how to make the most of a combination of vimtex and texlab?

clason commented 5 years ago

I was planning to offer to write up my notes properly as a section for the Wiki, once texlab has a full release (probably very soon) and I get the last kinks in the setup ironed out (TeX is not a standard language for a server, and vim is not a standard editor for a client, so there are some edge cases both ways. Also, I can't brain.)

lervag commented 5 years ago

Great, and thanks!

Also, I can't brain.

I can relate to this too often these days...

clason commented 5 years ago

Thinking a bit more about this, LSP integration may not have to be a non-feature (although I can definitely see why it may be, see below). What a language server provides is a structured, language-aware, access to the code you're editing. For example, it knows which parts of the code (in its language) are meaningful "symbols" -- for TeX, mostly labels (and bib keys) and keeps track of where they are. Vimtex could leverage this more powerful (and faster!) access for features which it currently uses regex grepping and aux files for -- such as autocompletion or table of contents. This means outsourcing "lower-level" parts of vimtex and concentrating on how to present and manipulate the results, beyond the standard Actions defined by the generic language server protocol. Neovim is currently working on implementing low-level LSP integration that could be used for that (so no extra plugins necessary).

Clearly, that would be a massive refactor, and the reliance on an external binary for important functionality might be distasteful (it certainly goes against the vim principle of ubiquity and platform-independent scripting). So unless you need a new hobby -- "vimtex 2.0" -- it would probably need to be a fork -- "nvimtex", if it uses the new neovim hooks -- and hence "someone elses problem" :)

(Regarding "nvimtex": Neovim is also working on adding multiprocessing, and the first pull request reports an impressive speedup of 20x for vimgrep. Might be worth looking into once it lands!)

lervag commented 5 years ago

This means outsourcing "lower-level" parts of vimtex and concentrating on how to present and manipulate the results, beyond the standard Actions defined by the generic language server protocol.

I thought this was already well provided by the language server plugins such as vim-lsp, ALE, coc-nvim, LanguageServer-nvim, and so on. As such, it feels wrong to implement it as a dedicated LaTeX interface. I feel it should be more worthwhile to instead try and adapt vimtex to a workflow where some features are handled by the language server client. In a (probably quite distant) future, one could foresee that vimtex is greatly simplified in the sense that features like completion, ToC, etc, are handled by the language server.

(Regarding "nvimtex": Neovim is also working on adding multiprocessing, and the first pull request reports an impressive speedup of 20x for vimgrep. Might be worth looking into once it lands!)

Thanks, this is very interesting information! I am happy to see that there are more enhancements in the pipeline for neovim.

clason commented 5 years ago

Well, the point of LSP (the protocol) is that it defines standard actions that all language servers (can) provide and all clients (can) trigger -- independent of the language. There is of course some room for adaptation here -- for example, the "symbols" for python (variables, functions) are different than the ones for TeX (labels, bib keys). But the actions are fixed: rename, list, preview, etc. Many things that vimtex does do not easily fit into that mold (for example, the table of contents functionality is much more nuanced than simply "list all references"). And things like "change surrounding environment" or "toggle star" are far outside the protocol. Nevertheless, the basic infrastructure of the language server (compiled syntax-aware document parsing) might be leveraged in place of interpreted VimL regexp parsing for this if it is exposed to a higher level, which the neovim pull request seems to be allowing (if I interpret it correctly). Imagine what you could do if you had access to a list of all labels with their location in all files in the current project without having to manually parse the TeX (and aux!) files yourself.

Of course, at this time, it's just pie-in-the-sky futurist hot air...

(And if you think that's cool, you should look at the tree-sitter pull-requests -- direct access to the syntax tree of a document generated by a compiled parser -- in other words, super fast highlighting :))

lervag commented 5 years ago

Imagine what you could do if you had access to a list of all labels with their location in all files in the current project without having to manually parse the TeX (and aux!) files yourself.

Good point, that would of course be very nice!

I'm also looking forward to the tree-sitter stuff, although I currently only know about it from reading on reddit. But it sounds very useful!

clason commented 5 years ago

Of course, the first thing I did was google "tree-sitter latex" and was happy to find https://github.com/yitzchak/tree-sitter-latex :)

One thing I'm wondering is how to best pay forward the huge work that went into the vim-based features to the new kids (language servers and tree-sitter grammars)...

astier commented 5 years ago

Would it be a possibility to turn vimtex to a language-server?

clason commented 5 years ago

@astier That doesn't really make sense -- the whole point of a language server is that it's independent of the editor.

Are there any specific features from texlab that you think vimtex should implement directly?

(@lervag If you're using coc, there's now https://github.com/fannheyward/coc-texlab that should make it fairly easy to try out in case you're curious. Watch out for issues with non-ASCII characters -- LSP is based on UTF-16, and that trips up for example vim-lsp. If it works for you, let us know!)

astier commented 5 years ago

Just forget my idea.

clason commented 5 years ago

@astier I did not mean to be dismissive, just asking for more specific details about your idea.

astier commented 5 years ago

No worries. I did not perceive it as dismissive. I just had a thought but it doesn't make sense.

motorto commented 4 years ago

@clason sorry to dig and old thread but vimtex is extraordinary as an plugin with auto-completion , previously I have used coc nvim , but have ever since moved to vim-lsc, is there any plan to support or is already supported ? On help page (vimtex-complete) there are many lsc client supported but vim-lsc is not one .

clason commented 4 years ago

@motorto Vimtex is not a language server (and coc is not exactly an lsp client), so you're looking for the wrong thing, I'm afraid. What you want is a general-purpose completion plugin like deoplete, ncm2, or asyncomplete.

(I personally am not using autocomplete for vimtex and just trigger it manually.)

motorto commented 4 years ago

@clason. You are right ! I am aware that coc.nvim is more than lsc client.

Unfortunately texlab server is not on pair to vimtex at the moment so I am moving back to coc nvim to use coc-vimtex.

clason commented 4 years ago

Why can't you use both? And if you're not using a language server, why do you want to switch to vim-lsc? (But if coc works well for you, no reason to switch from it.)

motorto commented 4 years ago

Vimtex and texlab ?

Well to be honest I am having trouble configuring the texlab, their documentation is not great (at least from my point of view).

Texlab gives me the autocompletition that I want Packages for example (I couldn't make texlab do that).

If its the move from vim-lsc to coc-nvim , well then coc-nvim does everything that vim-lsc does so , it's redundant having both :)

On 2020/09/05 01:39, Christian Clason wrote:

Why can't you use both?

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/lervag/vimtex/issues/1371#issuecomment-687572874

krishnakumarg1984 commented 3 years ago

@lervag I think now is a great time to revisit this. LSPs have become incredibly popular and are frankly quite awesome (I have started using ccls/clangd for my C++ mini-projects, and Gosh, it lends an order of magnitude improvement in quality of life!

@teto texlab has indeed been re-written in pure rust, and have been making regular releases. Indeed, the project repo is super active with frequent commits.

Why is this the best time to revisit this issue?

Neovim 0.5 has been just released. In my view, this is a game changer, with its built-in LSP integration, and the possibility of allowing lua plugins, tree-sitter integration etc.

In fact, @clason and I were briefly discussing an issue this morning about tree-sitter diff-ing in neovim. Frankly, the availability of some high quality plugins leveraging the lua support that neovim 0.5 provides has made previously unimaginable things happen.

I think that vimtex can perhaps look at offloading certain functionality to texlab, and focus on the core parts in which it excels. vimtex may provide a user-meta option to turn on LSP (which in-turn can be an alias for turning off syntax highlighting, formatexpr, etc etc.). For those without access to an LSP client and server (or those who don't wish to use one), then we fall back to the complete functionality and goodness that vimtex offers.

Of course, we need to thoroughly think this through, if there is further interest in this issue.

clason commented 3 years ago

I have been summoned!

Frankly, the arguments against are still the same -- VimTeX targets a much broader audience than just neovim 0.5+ folks, and the overlap with LSP is not that large, considering what VimTeX offers. (The argument may be different from the other side, but again, texlab is primarily -- although happily not exclusively -- aimed at VS Code users.) And as the release newsletter hopefully makes clear, tree-sitter is not ready yet for production use and cannot replace the extensive (in particular, package-specific!) syntax support of VimTeX. It's also very important to keep in mind that texlab is constrained by the LSP specification, which is not written with markup languages like LaTeX in mind (which are also quite difficult to parse for tree-sitter, I might add), while VimTeX has no such constraints.

And there is nothing preventing you from using both texlab and VimTeX (and tree-sitter, if you're feeling brave) in parallel; I've been doing so for more than ~~a year~~ two years now. (I just scrolled up to the first comments for a trip down memory lane. My, how time flies when you're stuck in home office...)

Is there an opportunity for an "NvimTeX" built on top of LSP, tree-sitter, and whatever other goodies Neovim's Lua API offers (Telescope? LuaSnip?) Absolutely. But It Is Not This Day.

clason commented 3 years ago

(And I know you don't intend it that way, but this suggestion has an aftertaste of "now that LSP is out, you can stop trying and leave this to the experts" -- you're essentially suggesting that @lervag abandon years of work and start from scratch!)

krishnakumarg1984 commented 3 years ago

@clason Absolutely not. Karl is one of my favourite open-source developers, and has been an absolutely awesome human being to interact with (I have talked with him once). vimtex is a gem of a plugin, and I used it to write 4 papers and a PhD thesis, and I am ever grateful for it.

My point was to provide a user-flag to selectively disable certain groups of configs that can be better done by an external tool. For instance, it is well-known that latexindent.pl is a better/more customisable indenting tool for tex files even though vimtex's internal formatter is no slouch.

I advocate co-existence of vimtex and LSP. My point was to have users a bit of control through user setting flag, and if no LSP is found, then we fall back to the complete set of features that vimtex provides.

clason commented 3 years ago

It's still not clear what exactly you want changed, then; there's zero conflict between VimTeX and texlab (and tree-sitter), they happily coexist already. You're 100% in control of it by setting your mappings according to taste. No options necessary.

Let me be clear here: I have been doing this for two years now and never noticed a conflict. If you have, you should give a concrete example!

lervag commented 3 years ago

This has already had many words, so I'll try to first make some general responses and comments, then I'll close with coments to the more concrete suggestions.

First, I really do not mind these discussions. I don't want to end up maintaining a plugin that "lives in the stoneage", so I need to be aware of what is happening around me. But my time is quite limited, so it is very good that the community helps me get informed! Tree-sitter and LSPs are huge, and at some point in time I believe it will impact how I shape VimTeX!

General

LSPs have become incredibly popular and are frankly quite awesome (I have started using ccls/clangd for my C++ mini-projects, and Gosh, it lends an order of magnitude improvement in quality of life!

I strongly agree on the usefulness of LSPs!

@teto texlab has indeed been re-written in pure rust, and have been making regular releases. Indeed, the project repo is super active with frequent commits.

I'm glad to hear it! I still don't really use texlab much.

Neovim 0.5 has been just released. In my view, this is a game changer, with its built-in LSP integration, and the possibility of allowing lua plugins, tree-sitter integration etc.

I agree this is all good! :)

I think that vimtex can perhaps look at offloading certain functionality to texlab

I would not really mind offloading things, but it has to be done very carefully. I personally don't see any good reason to look into this right now.

As @clason writes (and I could not write it better myself):

Frankly, the arguments against are still the same -- VimTeX targets a much broader audience than just neovim 0.5+ folks, and the overlap with LSP is not that large, considering what VimTeX offers. ... And there is nothing preventing you from using both texlab and VimTeX (and tree-sitter, if you're feeling brave) in parallel

Concrete

vimtex may provide a user-meta option to turn on LSP (which in-turn can be an alias for turning off syntax highlighting, formatexpr, etc etc.). For those without access to an LSP client and server (or those who don't wish to use one), then we fall back to the complete functionality and goodness that vimtex offers.

I don't really see a very simple way to do this. VimTeX would have to properly detect the settings of the LSP. Some people may want some parts of the LSP, others may want other parts. So, it seems to me to be much better to simply allow people to properly disable parts of VimTeX that they want and let them customize their environment manually.

I advocate co-existence of vimtex and LSP. My point was to have users a bit of control through user setting flag, and if no LSP is found, then we fall back to the complete set of features that vimtex provides.

My last point stands: I don't really see any clear/simple way for VimTeX to understand what the user wants. Like:

How do I detect if a user is running texlab? (Does it even start before VimTeX is initalized? Does this depend on the user config?)
How do I detect which features a user wants from texlab?

HOWEVER: I do see how it could be beneficial to the community to have a better documentation on how to use and configure VimTeX alongside texlab. Currently, there is only a very small section that explains what texlab is. I would be both grateful and happy to get help from someone (perhaps @clason?) to write a better guide on how to make these plugins coexist; e.g. suggested configuration and describe where there is an overlap and so on. Since I still don't really use texlab, I don't know enough to write something like this myself.

clason commented 3 years ago

Well, my

LSP config is here: https://gist.github.com/clason/04b78ccbf49c4e4a3301b18927a83329
VimTeX config is here: https://gist.github.com/clason/12f884907a80baec6bf9c18a467ff300
mappings are here: https://gist.github.com/clason/3156fd207b16afc43cb2d5c911d35dc9

As for overlap, my initial comment still stands. (texlab has improved a lot, and I can't recommend the project enough -- there's something about TeX that attracts great people ;) but the basic functionality is the same.)

In particular, you'll see that I don't have any LSP-motivated changes to the VimTeX cconfiguration. The only thing worthy of mentioning is that I use texlab for autocompletion and reserve VimTeX for omnicompletion.

lervag commented 3 years ago

How does the texlab completion work compared to VimTeX?

clason commented 3 years ago

Cite: (texlab first, then VimTeX)

Here, VimTeX can of course match on menu, which texlab can't.

Labels: (same)

(texlab is much quicker, at least on a cold cache)

lervag commented 3 years ago

Thanks!

clason commented 3 years ago

(you could also link neovim's builtin LSP client: https://github.com/neovim/nvim-lspconfig/blob/master/CONFIG.md#texlab ;)

krishnakumarg1984 commented 3 years ago

@clason Thank you for the useful comments. Can you please clarify what you meant by Here, VimTeX can of course match on menu, which texlab can't.?

@lervag Thanks a lot for providing detailed explanatory answers. I think we can rest this case for now, bearing in mind the awareness that the neovim ecosystem is undergoing some big changes.

00sapo commented 2 years ago

@clason this is an unfair comparison: lsp clients usually provide a preview when a completion entry is selected

clason commented 2 years ago

Sorry, I stand by this, which was a specific and not a general point: VimTeX offers a better preview than texlab. (I use both -- and they're both great at what they do -- so I know.)

krishnakumarg1984 commented 2 years ago

Both texlab & vimtex are being actively developed. The texlab LSP server is a rust application (which, depending on the camp you are in, might either come across as performant & type-safe, or as a mere trend-following/hype of the current era).

Putting texlab to the side and thinking about the LSP ecosystem in general, the protocol is evolving well, and the information exchange between a client and server is a rather elegant JSON RPC dataframe. The simplicity of the protocol is encouraging more developers to contribute to various LSP servers, and generally I have seen better/faster completions, definition support etc in python/C++. The LSP protocol itself is evolving in positive directions.

Why did I say all these? I have truly benefited from VimTeX, and I agree with clason that it is currently superior to texlab. It is imperative that, with the 7.5 years of hard work put into vimtex to make it so brilliant, it needs to remain relevant in the next decade. VimTeX has been tested against so many corner cases, and lervag writes up test cases with each PR to ensure that regressions are rare.

I think, a visionary/prudent approach for VimTeX to thrive would be to:

study exactly what features overlap with texlab (and the LSP protocol), and provide user options to switch to texlab if installed.
use the dissection study from above to integrate beautifully with texlab, and provide user instructions in the documentation on how to integrate the two (It looks like clason has already integrated these two and his config could be a good starting point).

The modus operandi would be to try this out in a feature branch, and announce vimtex 3.0 series with LSP support/integration. That would make the current greatest latex vim plugin even greater, staying ahead of the curve. Ignoring a fledgling entrant with potential has historically led to the downfall of so much software that were considered unsurmountable in their heydays.

clason commented 2 years ago

I think VimTeX is thriving very well by doing what it is doing currently; I don't see any grounds on thinking that this will change in the near future. And at the risk of sounding like a broken record: VimTeX and Texlab coexist very well already; if there is an overlap of functionality, you can set up your mappings to use one over the other as you prefer. There is nothing VimTeX needs to do to support Texlab!

In general: such "philosophical" comments are rarely helpful; it's much better to focus on single concrete issues and tackle them one at a time.

krishnakumarg1984 commented 2 years ago

I disagree that philosophical comments are rarely helpful. The Agile manifesto, which is now considered the backbone of modern software development, is entirely philosophical in nature!

I think major open-source projects welcome discussions about user experiences, and have RFCs regarding its future direction. It would provide a more inclusive environment which will empower less technical people to contribute to the project.

I think VimTeX is thriving very well by doing what it is doing currently;

Yes. I echoed this exact feeling in my post, with currently being a keyword. We were just considering how to keep it the best going into the future. Ignoring a new entrant to the field is not the answer, imho. Can we at least provide an objective comparison of each overlapping capability in the documentation?

you can set up your mappings to use one over the other as you prefer.

Great. That sounds useful. @clason, would you be willing to provide a write-up or documentation update on how these two can co-exist?

Finally, there may not be a 100% right or wrong approach to think about future directions for software. In my interaction with him, I have observed that VimTeX's main developer is flexible and adaptable to acknowledge the potential of LSP servers like texlab, and hoping that he'd use the best approaches from any competitor, or somehow provide ways to co-operate well with any such technologies in the future.

lervag commented 2 years ago

@krishnakumarg1984 Thanks; I actually do appreciate philosophical comments and thoughts, and it is useful to get this kind of input. I agree with @clason's sentiment that concrete issues are in general much more helpful, but sometimes it is also very good to take a step back and consider things in a larger perspective.

Some thoughts to your concerns:

study exactly what features overlap with texlab (and the LSP protocol), and provide user options to switch to texlab if installed.

Well, I'll try, or at least I'll accept PRs with updates to the docs and similar.

use the dissection study from above to integrate beautifully with texlab, and provide user instructions in the documentation on how to integrate the two (It looks like clason has already integrated these two and his config could be a good starting point).

In this I agree with @clason:

I think VimTeX is thriving very well by doing what it is doing currently; I don't see any grounds on thinking that this will change in the near future. And at the risk of sounding like a broken record: VimTeX and Texlab coexist very well already; if there is an overlap of functionality, you can set up your mappings to use one over the other as you prefer. There is nothing VimTeX needs to do to support Texlab!

However, I will admit that this is also somewhat "lucky". I currently (and still) do not use texlab myself, and as such, I do not experience problems that I thus fix for my own needs. So if there are concrete shortcomings, these should be raised by users. And in this particular case; ...

In general: such "philosophical" comments are rarely helpful; it's much better to focus on single concrete issues and tackle them one at a time.

... as mentioned, I agree with @clason. The more "day to day" improvement and maintenance of VimTeX is very much dependent on users raising specific and concrete issues that I and (as I'm very glad to see!) others can address.

The modus operandi would be to try this out in a feature branch, and announce vimtex 3.0 series with LSP support/integration.

I do not fully understand what this integration would mean, specifically.

Ignoring a fledgling entrant with potential has historically led to the downfall of so much software that were considered unsurmountable in their heydays.

I am not knowingly ignoring texlab. But I am not able to spend as much time with VimTeX as I need to get on top of everything (there are still ~10 open issues and a couple of PRs, and some of these will correspond to major efforts). I don't think you mean it that way, of course.

Can we at least provide an objective comparison of each overlapping capability in the documentation?

I think this sounds like a reasonable request. However, I would not call it comparison, instead I would call it a documentation of "integration" in the sense of explaining a "best practise" manner to combine VimTeX and texlab. I would welcome a PR for this. texlab is currently described in :help vimtex-nf-lsp, so a good starting point would be to update this text and more precisely describe the overlap.

In this sense, I think the following are things that would be useful for me to know in order to understand "the future of VimTeX":

Which features of texlab are directly overlapping with VimTeX, now and in the future?
Which of these should I assume will be "better" served by texlab as it progresses? All?
In a similar vein: Will Tree-sitter syntax support for LaTeX eventually become superior? I would not be surprised if it does; if so, how fast is this development going?
The above, combined and inversed: If we assume that texlab and Tree-sitter were "optimal", which VimTeX features would still be relevant?

krishnakumarg1984 commented 2 years ago

@lervag Thanks for your receptive and detailed response. I agree to everything you have said here. Just one minor point

texlab is currently described in :help vimtex-nf-lsp, so a good starting point would be to update this text and more precisely describe the overlap.

The non-features wording might be perceived as sending out a stronger negative signal. I know its pure semantics, but it appears as saying "No. This particular aspect is not vimtex's job. If you want feature X, read this section and DIY".

Instead, I think the section could be titled "Integration with other relevant technologies" or "Interoperation with other plugins" or something which conveys a more inclusive/welcoming tone (reflecting the openness of your response here). In such a documentation section, providing an initial example set of keymaps and configs explaining how to co-operate with texlab shall enrich the user experience, imo.

I have been reassured (even by large-scale projects) that open source is as much about the community as it is about the code. In my experience, fixing a simple sentence typo or providing an entire language translation for the documentation of a software has been welcomed as much as implementing a new technical feature. I believe that some initial config/documentation section for LSP shall encourage others who have experimented here shall submit their own configs as PRs.

@lervag For visibility/discoverability reasons, would you be okay to indicate that you are willing to accept example LSP configs as documentation PRs. It would be good to indicate this inline in the documentation, but I suspect that might be too deeply embedded in the text to make an impact. The README.md itself is a great place to indicate that you are open to LSP-related PRs. Maybe you could also tag this GitHub issue as "help-wanted" etc. etc.

PS: Treesitter for LaTeX is currently nascent, but I suspect texlab development may accelerate treesitter as one feeds on the other.

clason commented 2 years ago

PS: Treesitter for LaTeX is currently nascent, but I suspect texlab development may accelerate treesitter as one feeds on the other.

I didn't want to comment anymore but feel like I must correct you here: the tree-sitter parsers are not used by texlab -- the developers investigated that at some point but in the end decided to use their own, special-purpose, parser. (There is a lesson here.) They polished up and released their parser on my request so that it may be used with nvim-treesitter. They have not committed to maintaining/extending it in any way.

krishnakumarg1984 commented 2 years ago

Thanks for pointing that out, didn't know.

lervag commented 2 years ago

texlab is currently described in :help vimtex-nf-lsp, so a good starting point would be to update this text and more precisely describe the overlap.

The non-features wording might be perceived as sending out a stronger negative signal ...

Good point; I agree.

Instead, I think the section could be titled "Integration with other relevant technologies" or "Interoperation with other plugins" or something which conveys a more inclusive/welcoming tone (reflecting the openness of your response here).

Good suggestion. I think you have good points here. I've pushed an update that I hope you find agreeable; and please, if you have further ideas for improving this, don't hesitate to open a PR!

In such a documentation section, providing an initial example set of keymaps and configs explaining how to co-operate with texlab shall enrich the user experience, imo.

Agreed, at least in part. I believe the keymaps for texlab are not specific. Instead, you define keymaps for the LSP in general, and these should be valid across languages and LSP servers. But I agree it could be expanded upon. Again, feel free to suggest improvements!

For visibility/discoverability reasons, would you be okay to indicate that you are willing to accept example LSP configs as documentation PRs.

I want to keep the docs as concise as possible. But I don't mind PRs that improve and expand upon topics in way that make sense (e.g. for visibility and discoverability). I would be OK to indicate this type of willingness - feel free to suggest how in a PR.

It would be good to indicate this inline in the documentation, but I suspect that might be too deeply embedded in the text to make an impact. The README.md itself is a great place to indicate that you are open to LSP-related PRs. Maybe you could also tag this GitHub issue as "help-wanted" etc. etc.

This issue is both old and closed, so I don't think it will help to tag it. Perhaps better to open a new issue with a "feature" request to improve docs with a suggestion on how to integrate VimTeX with texlab. If you both refer to the current text, this particular issue, and add the tag "help-wanted", then it may attract a contribution.

I don't think the README.md is a good place to indicate this, but you could open a PR to show me what you mean. If I agree, I would merge the PR. :)

lervag commented 2 years ago

(By the way - this thread is now so long that I'll lock it when we've settled these final points.)

krishnakumarg1984 commented 1 year ago

Just wanted to let you know that the links to the gists for configuring for the co-existence of texlab and vimtex are now invalid, and give 404 errors.

@clason Would you mind pointing us to updated links for your texlab and vimtex configuration? These could be a good starting point for users who arrive in this issue thread by following vimtex's documentation.

lervag / vimtex