search

nim-lang / RFCs

A repository for your Nim proposals.
136 stars 26 forks source link

Developer tooling #300

Open mratsim opened 3 years ago

mratsim commented 3 years ago

The steady pace of Nim development, the language and the communication around the new features like ARC and ORC is attracting goodwill and devs are trying Nim out.

However as mentioned in many many places, Nim current success is not supported by the tooling and is actually despite it.

Situation

Additionally this question on Telegram sparked my RFC: image

No data from Telegram and IRC for now feel free to browse https://www.google.com/search?q=nimsuggest+site:irclogs.nim-lang.org

Analysis

  1. I believe there are 2 issues here: nimsuggest and the vscode extensions. AFAIK I didn't see issues with LSP or vim/nvim or emacs that aren't due to nimsuggest.

  2. From the various threads, I understand that the plan is to rewrite nimsuggest once incremental compilation lands.

  3. Visual Studio Code is one of the most popular editor for developers and is the most popular Nim editor by far despite the issues reported over and over with nimsuggest and the extensions: https://nim-lang.org/blog/2020/02/18/community-survey-results-2019.html

image

Opinions

  1. The tools around a language are an integral part of the whole developer experience. The fact that the extension for the number 1 Nim dev environment has so much issues is a problem and will be a minus for developers evaluating Nim against other languages.

  2. Incremental compilation has been deprioritized vs other features even though it is part of Milestones 2020 roadmap at https://github.com/nim-lang/RFCs/milestone/1 (https://github.com/nim-lang/RFCs/issues/46). From the look of it, incremental compilation has architectural unknown that makes it unlikely to be delivered in the next 6 months.

  3. From a prioritization point of view, I believe developers tools are significantly more important than incremental compilation. If compilation is slow, you can always throw faster hardware at it. We're not in C++ land here. However if a developer is slow, throwing more developers doesn't work: there is communication and synchronization friction, getting to know the code base and ... tooling frictions.

  4. Companies like Status or the one screenshoted are recruiting Nim talents but also Go, C++, Rust talents for their team. While those team members are eager to learn Nim, they need help to get started. Tooling can go a long way.

  5. One of the top critic of Nim is the seemingly absence of mandatory namespacing for imports which makes it hard to know where a function came from, there are 2 replies:

    • enforce myModule.myImport(args) syntax in your codebase.
    • use better tooling

    Unfortunately there is no better tooling right now.

  6. I think VSCode has some smart Intellisense that doesn't rely on the language but just the project code. Experimented users probably don't need a full nimsuggest. However they do enjoy syntax highlighting working and working with macros. Unfortunately if nimsuggest crashes due to macros, there is no highlighting and all suggestions disappear even pure Intellisense. Case in point, I just type spawn( in Weave and autocompletion dies and VScode intellisense (cpp-tools) gets stuck at 160% CPU usage (and if in macros it's nimsuggest) image

    edit: It probably is TabNine, a deep learning powered auto-completion tool: https://www.tabnine.com/, https://www.tabnine.com/blog/deep/

Action items

  1. We have a couple of full-time developers working on Nim, I think working on developer tooling should be one of 2021 focus. I do think a VScode extension under Nim umbrella is in order.

  2. An extension with just syntax highlighting and uses VScode auto-complete builtins that doesn't crash or use 100% CPU on Karax, Jester, Weave and other codebase using generics, statics and macros would please seasoned developers.

  3. For a full extension that integrates with or embed the compiler, don't wait for IC, there are workaround for slow machines but no workaround for discouraged developers. This is important for new joiners

  4. Good debugger integration (viewing variable contents, GDB/LLDB) would also be a boon for debugging complex code. debugEcho gets old.

  5. Regarding architecture, I'm not sure if a CLI tool or a service with a RPC API is better so another RFC might be in order to discuss that

piotrtomiak commented 3 years ago

@ShadowElf37 I've released a large update of the plugin over a week ago. Thanks to your feedback I've improved the autocompletion as well. If you have any problems or feature requests it's best to file an issue with our issue tracker - https://youtrack.jetbrains.com/issues/NIM . We are still thinking about open-sourcing the plugin, however, first, I need to bring it to a certain level of completeness before it's even doable.

johnnovak commented 3 years ago

Just confirming, tried nim.nvim with the native NeoVim 0.5 beta LSP today, and it has been a frustrating experience. Just by using follow definition (gd) a couple of times in my small project (less than 20 files and about 5 dependent projects) I get crashes, then quite often it just stops working after a few jumps, sometimes it loses connection with nimsuggest, etc. Cool when it works, but overall unusable for me, it's just more hassle than worth, it's so unreliable.

(And yes, I'm using macros.)

zetashift commented 3 years ago

Just confirming, tried nim.nvim with the native NeoVim 0.5 beta LSP today, and it has been a frustrating experience. Just by using follow definition (gd) a couple of times in my small project (less than 20 files and about 5 dependent projects) I get crashes, then quite often it just stops working after a few jumps, sometimes it loses connection with nimsuggest, etc. Cool when it works, but overall unusable for me, it's just more hassle than worth, it's so unreliable.

(And yes, I'm using macros.)

Yeah I'm on the exact same setup with neovim and it's just not great, go-to definition doesn't work reliable, because nimsuggest can crash out whenever something touches macros...

Highlighting of nim.nvim is pretty sweet though(breaks down if macros are involved), but besides that I don't think I use anything else because it just works too unreliable, I don't know if this is nimsuggest or nimlsp...

johnnovak commented 3 years ago

Just confirming, tried nim.nvim with the native NeoVim 0.5 beta LSP today, and it has been a frustrating experience. Just by using follow definition (gd) a couple of times in my small project (less than 20 files and about 5 dependent projects) I get crashes, then quite often it just stops working after a few jumps, sometimes it loses connection with nimsuggest, etc. Cool when it works, but overall unusable for me, it's just more hassle than worth, it's so unreliable. (And yes, I'm using macros.)

Yeah I'm on the exact same setup with neovim and it's just not great, go-to definition doesn't work reliable, because nimsuggest can crash out whenever something touches macros...

Highlighting of nim.nvim is pretty sweet though(breaks down if macros are involved), but besides that I don't think I use anything else because it just works too unreliable, I don't know if this is nimsuggest or nimlsp...

Yeah, personally I'll just revert back to an old regexp based syntax highlighter and that's it. I'm fine with that and just using the Nim compiler from the command line. It has worked well enough for all those millions of C devs in the past 50+ years :)

The LSP stuff is cool in theory, but if it's not 100% reliable, then it just gets in the way. Autocomplete would also be cool, but it's not a must. A beneficial side effect of NOT having it is that you'll remember a bit better where things are in your project exactly :)

zetashift commented 3 years ago

Yeah, personally I'll just revert back to an old regexp based syntax highlighter and that's it. I'm fine with that and just using the Nim compiler from the command line. It has worked well enough for all those millions of C devs in the past 50+ years :)

I am getting used to it, but the big problem is for beginners it just highers the bar for entry significantly.

The LSP stuff is cool in theory, but if it's not 100% reliable, then it just gets in the way. Autocomplete would also be cool, but it's not a must. A beneficial side effect of NOT having it is that you'll remember a bit better where things are in your project exactly :)

When one browses the issues on nimlsp or nim.nvim they'll see that it eventually comes down to a hard to debug problem that most likely resides in nimsuggest. But yeah I found this spartan way of programming very insightful, but this RFC makes plenty of arguments why the current state is not desireable.

Tooling is hard though, I'm glad this RFC exists and that IC might mean more reliable tooling. Exciting times ahead!

dsrw commented 3 years ago

I just wanted to chime in and say that I've been really happy using the IntelliJ plugin that @piotrtomiak wrote, along with the CLion IDE. Indexing is fast, highlighting is flawless as best as I can tell, and completion/code navigation work well for simple cases. There's plenty that it can't complete currently, but it works often enough to be useful, and when it fails it's pretty graceful about it.

One limitation is the lack of toolchain support. It can only use the Nim compiler it finds on the path, and doesn't support building with nimble. However, CLion helps here, as it makes it easy to throw together a custom build using whatever commands you want, which can then be debugged from inside the IDE if built using --debugger:native. There's also a File Watcher plugin that I'm using to automatically run nimble c check ... on each change, which properly highlights warning/errors in my sources and adds links to the compiler output to navigate directly to issues. I've set it to run my tests whenever I change them, again with error highlighting and links in the test output.

The whole thing was a bit of a pain to setup, but it's my favorite nim development environment now that I mostly have the kinks worked out. At some point I'm going to release a project template that has the custom build and file watcher stuff pre-configured.

For folks who already use Jetbrains IDEs it's definitely worth a look. For everyone else it's probably a bit early, but it should be a 1st class experience soon enough if @piotrtomiak is able to keep working on it.

qaziquza commented 3 years ago

On a Mac with 8gb RAM, VS Code with the Nim extension is unusable. 1 hour and the entire computer crashes. I had to switch to the vastly inferior Geany text editor. I dearly hope that a better extension will be available at some point.

omentic commented 1 year ago

What is the current status of tooling? I've had luck with the new langserver for Sublime and now Helix: occasionally I still get orphaned processed pegging my CPU at 100% but they're few and far between now. I haven't used VSCode in a while but the extension was problematic and buggy back when I used it. I suppose, to enumerate what I'm wondering:

There are various langservers / IDE tooling / etc: the compiler's nimsuggest, and PMunch/nimlsp and nim-lang/langserver (v3, iirc)

What editors have comprehensive Nim support: and how robust is it?

What are the current capabilities of IDE tooling? As far as I can tell:

How do we ease setup?

Some questions going forward:

EchoPouet commented 1 year ago

There is also IntelliJ Nim plugin but I don't know if it's a good alternative. Also the debugging is activate only in Ultimate version of IntelliJ.

IMHO, the team must focus on VSCode because is free and most popular with very good debbuging integration.

I see more and more users are active on Discord and I'm afraid the lack of a quality IDE will discourage them.

hansdampfinger666 commented 1 year ago

Vim works with https://github.com/PMunch/nimlsp. Seems to be having problems rn though. No autocomplete etc. at the moment.

firasuke commented 1 year ago

Vim works with https://github.com/PMunch/nimlsp. Seems to be having problems rn though. No autocomplete etc. at the moment.

Yeah, it appears to be related to nimsuggest https://github.com/PMunch/nimlsp/issues/36.

EchoPouet commented 1 year ago

Have you try langserver ?

firasuke commented 1 year ago

Have you try langserver ?

langserver is only available for VScode via nimsaem's plugin. I haven't found a single vim/nvim plugin that uses langserver as all current implementations (even mason) use nimlsp which doesn't seem to be working atm.

havardjohn commented 1 year ago

Have you try langserver ?

langserver is only available for VScode via nimsaem's plugin. I haven't found a single vim/nvim plugin that uses langserver as all current implementations (even mason) use nimlsp which doesn't seem to be working atm.

@EchoPouet If you use neovim with 'neovim/nvim-lspconfig' plugin, you can just change the language server cmd in your lua-config like so to use langserver.

require'lspconfig'.nimls.setup {
  cmd = { "nimlangserver" };
}
firasuke commented 10 months ago

Have you try langserver ?

langserver is only available for VScode via nimsaem's plugin. I haven't found a single vim/nvim plugin that uses langserver as all current implementations (even mason) use nimlsp which doesn't seem to be working atm.

@EchoPouet If you use neovim with 'neovim/nvim-lspconfig' plugin, you can just change the language server cmd in your lua-config like so to use langserver.

require'lspconfig'.nimls.setup {
  cmd = { "nimlangserver" };
}

I switched to Helix which surprisingly has better Nim support with nimlangserver compared to Neovim.

aMOPel commented 9 months ago

Update on this:

https://github.com/alaviss/tree-sitter-nim

Is now in a very usable state. I have written a bunch of queries for neovim already.

firasuke commented 9 months ago

Update on this:

https://github.com/alaviss/tree-sitter-nim

Is now in a very usable state. I have written a bunch of queries for neovim already.

It's good to see you both @aMOPel and @alaviss work on this together to bring tree-sitter support for Nim.

Your effort and time on improving Nim tooling is highly appreciated. Thanks!