Progress indicator

[sergei-dyshel]

Using newest version from marketplace: 0.1.12, ccls doesn't update progress upon starting but remains "loading" forever. By using log-file I see that loading is finished and all files are indexed but status bar is not updated.

[MaskRay]

The loading progress has was removed during the overhaul of the pipeline (the component that tracks dependencies, loads cache files, dispatches indexer threads and stores cache files). I know very little about vscode and mainly use emacs. This repo started as a mere re-packaging of vscode-cquery. Fortunately recently there are users helping improve it. You may want to try the HEAD version.

This progress bar can be added back, but I want to change it from a notification to a request, which the client should explicitly poll.

[MaskRay]

Pushed 0.1.13. I believe the progress bar has been removed.

[MaskRay]

Stupid vsce. I don't know how to use package-lock.json

Simply name it 0.1.14

[sergei-dyshel]

I this (progress implemented by polling) something you are going to add in near future or some distant feature. How the progress is currently showed in emacs?

[MaskRay]

I've implemented $ccls/info yesterday where you can find pipeline.pendingIndexRequests and project.entries, but I unlikely have energy to implement the polling interface in vscode (as i'm not a user). Would you like to help?

On Mon, Oct 1, 2018, 12:17 Sergei Dyshel notifications@github.com wrote:

I this (progress implemented by polling) something you are going to add in near future or some distant feature. How the progress is currently showed in emacs?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/MaskRay/vscode-ccls/issues/11#issuecomment-426029348, or mute the thread https://github.com/notifications/unsubscribe-auth/AAZaQoeRa6VzRWxHUJiGhLH2NSiBQLLLks5ugmo9gaJpZM4XBNLz .

[sergei-dyshel]

Well, I'm pretty new to vscode/javascript world and currently I'm in a middle of evaluation of both ccls and cquery. If I decide in favor of ccls I'll certainly look into it as progress tracking is very important feature to me.

Honestly, I much prefer usage of clang C++ interface in ccls but cquery's better (IMHO) polishness (at least for vscode) is also very important factor. Also, I'm still trying to run ccls on CentOS which I require....

[sergei-dyshel]

By the way, what was the exact reason to remove current progress bar?

[MaskRay]

You may take a look cquery's wiki/Design and queue_manager. I found it complicated and difficult to maintain/improve thus it was my first target to rewrite after I started ccls and cleaned/simplified some components. I experienced duplicate/missing references frequently in the past. The rewrite turns out to make the pipeline more robust. emacs-cquery has never had the progress bar. The rewrite wasn't easy and for simplicity I just removed it.

[MaskRay]

If you've tried HEAD (with the PreambleStatCache optimization), I am curious to know how you feel about its diagnostics/completion, compared with ycmd.

e.g. how many completion items does ycmd return by default (I just added an initialization option completion.maxNum: 100). emacs-ccls's switch to use-native-json may also help (requires latest lsp-mode)

[sergei-dyshel]

Unfortunately I still can't use ccls for auto-completion for the big project I'm working on at my job. Right now I'm evaluating two options for my production code (posted a question here):

• In VS code: use clangd for auto-completion and cquery/ccls (preferably ccls, once I add progress indicator) for everything else.
• In Spacemacs: use ycmd for auto-completion and cquery/ccls for everything else.

I'm new to both editors (being a hardcore VIM user with very customized configuration) so the evaluation and decision process will take some time.

For small projects (such as ccls or various tools I compile from source) parsing time for clangd/ycmd/cquery/ccls is so fast so that I really can't tell a difference. C++ API based tools (clangd/ccls) give qualitatively better completions of course.

[MaskRay]

Appreciated if'd you like to help.. But you may have over-emphasized the progress bar... In both cquery and ccls, the project does not need to be fully loaded to server LSP requests. You may have noticed that you get semantic highlight when you open a buffer, and probably a refresh after the project is fully loaded. The first semantic highlight may lack some colors for a few external declarations. textDocument/references is usable though the the result may be incomplete before a full load.

I find mixed tools quiet weird, but I know the cquery author mixes clangd and cquery. In Emacs, my theory is that if you care much about completion latency, irony-mode is better than ycmd (I am unclear about its value, sorry). clangd's completion is tuned a lot and they have a whole stack of heuristics for that. I watch their development very closely and have stolen some important stuff into ccls (in-memory preamble, frontendopts tuning, preamblestatcache, etc). I guess a huge number of users will eventually switch from cquery(ccls) to clangd, but i'll very like to know if I have achieved 80% of their quality of completion (80% is good enough to me as the additional heuristics do not attract me so much). (My current feeling is that their cross reference will be a long pole (the memory will also be an outstanding issue))

[sergei-dyshel]

I agree with you in that progress indicator should not be a showstopper. But it's a very helpful feature when I have "fat" C++ files which take up to 20 seconds to parse (C++14 features + lambdas + templates). When I edit such files I want to know when LSP finishes indexing them so that I would use cross-referencing without risk of references being omitted. Anyway if rest of functionality satisfies me, I'll readily implement this feature myself, that's not a big deal.

Mixed tools is something I would like avoid but I have an explanation for this: our product code is so big that indexing it on my 4-core laptop takes too much time. Especially because I change branches frequently so I have to reindex considerable parts of the project. That's why I have to run ccls/cquery on remote (32-core powerful) server using SSH trick described here. This approach, while good for cross-referencing, is not good for completion: single core performance is 2x better on my laptop so for completion purposes I better compile on laptop. Anyway, from experiment, completion with ccls/cquery running remotely is very laggy (practically unusable) compared with running them locally (in a standard way). That's why in addition to remote indexer I have to use some lightweight clang-based tool just for completion. clangd/ycmd/irony are best choices here since running "heavy" ccls/cquery just for completion is overkill.

I've been successfully using this approach with ycmd and remote rtags (in VIM) for a couple of years but now rtags is not "state of the art" indexer anymore (though it was practically the only option for years) so I'm seeking for a modern LSP-based alternative.

Regarding irony vs ycmd I believe that most of the latency comes from parsing by clang and since both tools are based on libclang I beleive their latency and results' quality should be comparable. irony exists more time so I expect it to be more polished from emacs integration perspective. ycmd is standalone server powering up Vim's plugin YouCompleteMe (which is I'm using in Vim that's why ycmd was my first option in emacs). I remember that when ycmd was introduced many users noted that it was considerably less laggy than irony at that time and AFAIR irony didn't support compile_commands.json at that time. I guess it caught up since then :smile:

I would happily use clangd over ycmd/irony in emacs however this requires support for 2 LSP servers (local + remote in my case) at the same time which emacs-lsp does not support yet (and won't support in near future).

[MaskRay]

Since you can browse code locally, I wonder if you can move cquery/ccls to your laptop. https://github.com/MaskRay/ccls/wiki/Initialization-options#clangpathmappings

I have a quite similar configuration: I index clang+llvm on a workstation but have the needs to read code on a laptop which cannot afford indexing the project (it takes 1 hour). I copy cache files from the workstation to the laptop and configure clang.pathMappings.

ycmd has a lot of customized/generic heuristics which I don't know if work well for C++. irony-mode uses a simple wire format which takes significantly less space than JSON (JSON deserialization in Emacs is horribly slow, use-native-json helps but I have some issue with company-lsp)

[sergei-dyshel]

My situation is different: I always work on my laptop, be it in office, in train while commuting or at home. And I'm always connected to compilation server. Therefore copying the cache around is not feasible solution for me - suppose I copied the cache before going home but then I need to modify some header and if this file is included in half of source files the half of projects get recompiled (after I run "freshen index"). So my solution with running LSP remotely suits me better.

[MaskRay]

How about index.trackDependency: 1?

It is 2 by default. See https://github.com/MaskRay/ccls/wiki/Initialization-options#indextrackdependency

I set it to 1 personally. After the project is loaded, I usually don't want to re-index a dependent .cc if any of its included .h has changed timestamp. The file will still be re-indexed when itself changes timestamp, though.

Unless the global index (struct DB) is inconsistent with cache files (which I think should be pretty rare now with ccls, but might still be an issue with cquery), $cquery/freshenIndex ($ccls/reload) is unnecessary.

[MaskRay]

BTW, welcome to the gitter channel to discuss these setup with me.

[MaskRay]

See this for progress reporting https://github.com/Microsoft/language-server-protocol/issues/511

[sergei-dyshel]

Thank you, I've read this conversation. LSP state indicator is indeed a common issue for all language servers and I hope it there will be some standard API for this someday. Anyway, I've already started some coding of ccls progress indicator but then I decided to learn some Javascript/Typescript before I proceed. Eventually I'll make some simple status bar item like the one in cquery. I also want to add some configuration options: to toggle visibility, to change polling interval.

[Yanpas]

Miss this feature since I use big project and it is frequently updated. I have experience with typescript and vscode extensions btw and can implement frontend part

[sergei-dyshel]

@Yanpas I've some fork of vscode-ccls with some simple implementation of progress in status bar. It works well for me for quite a long time. Look at commit https://github.com/sergei-dyshel/vscode-ccls/commit/58d5ebcd14ffc75f776745437addedc17be82830

[Yanpas]

Wow, that's cool, thanks.

Note, I currently do some refactorings on master if you plan to open a PR, you may face some difficulties while merging. And I suggest you to do all the changes in the separate branch, since you will need to update master from upstream. You can rebase last 3 commits to new branch.

[sergei-dyshel]

I'm currently kind of short on time and probably won't open PR in near future as it still needs some work and polishing. Rebasing it should be trivial as it's such a tiny patch.

[MaskRay]

VSCode has built-in support for progress bar. https://github.com/MaskRay/ccls/pull/662 will add server side support. However, the progress bar is dismissed after a WorkDoneProgressEnd notification. src/statusBarIcon.ts (#26) is still useful