microsoft / vscode-cpptools

Official repository for the Microsoft C/C++ extension for VS Code.
Other
5.52k stars 1.55k forks source link

Pre-indexing the project #4687

Open vsjha18 opened 4 years ago

vsjha18 commented 4 years ago

We have a C codebase which is around 9GB in size. It takes a lot of time to create the IPCH database and hence intellisence doesn't work so well for first few hours of the project. Is it possible to index the whole project separately and use the resulting IPCH database for all our work spaces.

We are using "Remote SSH" plugin and the system which we work on are essentially virtual machines but we have a facility of NFS mounts.

Please suggest the right workflow.

sean-mcmanus commented 4 years ago

The short answer is "it's not currently possible" because the database stores paths to files on the local machine. If the files were located on a shared machine with a shared database, then maybe that would work, but I don't think we support remote URIs yet. Maybe if all users shared the same path location, e.g. "/usr/src/project" or "C:/MyProject" then maybe the database could be copied/shared.

The workspace indexing (global symbol) data is stored in a ".db" file and not the "ipch" file. The ".db" file (database icon) would take hours because it applies to all C/C++ files in your workspace, but the "ipch" file (flame icon) is just created for translation units and should only take seconds or minutes (although sometimes under the hood we're actually doing work with the ".db" if recursive includes is being used...BTW a 9 GB codebase could hit performance issues if recursive includes "**" is used, so you might want to try to avoid that).

Also, you might consider adding some folders to the files.exclude to reduce the parsing for sub-folders that are less important.

vsjha18 commented 4 years ago

At my organisation, though everyone has his own linux instance but all of them download the codebase under the same path since that's the only path which can store such a big code base and store the built images. Will pre-indexing not help in such scenarios? I hope you are storing file path as a "key" in the database and I also hope that file path is relative to the workspace root. If you could tell us command which we can run recursively for all the files in the code base, may be that can help.

Our home directories are only 2GB and these ".db" files, which are stored in ".vscode-server" directory inside the home directory, are getting quite big for our code base. Will it not be possible to store them at some other place just like I can change location for "ipch" files?

sean-mcmanus commented 4 years ago

The files in the database are not stored relative to the workspace root and may contain files outside of the workspace root, or (in our in-development branch) the database could have files from multiple workspace folders (multi-root workspace).

However, the .db file is just SQLite file, so theoretically, you could write a script/tool that modifies the database to translate the shared locations to the user's local locations. You can use a SQLite tool to view the data inside.

The location of the .db file can be changed via the C_Cpp.default.browse.databaseFilename to something like "${workspaceFolder}/.vscode/vc.db" or it can be set in c_cpp_properties.json with the browse.databaseFilename setting.

vsjha18 commented 4 years ago

Thanks a lot for mentioning about "c_cpp_properties.json". This information has certainly helped big time.

As I am studying this extension more and more, I have a growing feeling that it is not as open source as it is generally advertised. e.g I am not able to find the codebase for Language Server. This extension certainly doesn't contain that code. Could you please help us on the following.

  1. Where is the code for language server itself? And is it open source?
  2. Can I use this extension but have language server of my own or other sources?

Now on a different topic, I have seen that once the C/CPP codebase is opened in a workspace, indexing starts spontaneously. But if you stop browsing files, at some point of time it stops automatically and again starts if you happen open the file from some other folder. It seems that it is deliberately implemented in a "lazy loading" style. Assuming above hypothesis is correct I have following doubts.

  1. What if I want to index the whole source code? Will I have to click upon all the files. Is there any way that we can leave the workspace overnight and it indexes everything.

  2. I have much more profound question - How indices are built? What are they?

  3. What Clang or gcc command do I need to run if I need to create those indices myself outside the realm of this extension.

sean-mcmanus commented 4 years ago

Only the TypeScript part is open sourced (not the language server executables).

I'm not sure what you mean by "can I use this extension but have language server of my own"? You could try, but it's not something we expect users to do.

The behavior your seeing with "lazy loading" sounds like you're hitting our current multi-root workspace folder behavior, which is in the process of being changed, so that it would parse all the workspace folders. Currently, 1 language server process is launched lazily for every workspace folder, i.e. as files are opened in different workspace folders.

Are language server process builds the indexes/database. They exist at the databaseFilename location. We don't use clang or gcc to build the indexes (we use a sqlite api). You can open the database with a SQLite viewer to see the data that is collected, i.e. it has files and symbol info.

vsjha18 commented 4 years ago

Many settings in the extension has "Default" as one of the options to chose from, along with other options. How do we know what "Default" points to? Is there are way to dump all the eventual values of all the settings parameters?

sean-mcmanus commented 4 years ago

Do you mean the C_Cpp.Default.* settings or the values like "Default" for C_Cpp.intelliSenseEngine?

vsjha18 commented 4 years ago

Values like C_Cpp.intelliSenseEngine.

vsjha18 commented 4 years ago

In the package.json there are few settings with C_Cpp.default.* and few are like C_Cpp.intelliSenseEngine (without the "default") in the name. In what way they differ? It would be great if you can explain.

sean-mcmanus commented 4 years ago

The intelliSenseEngine "Default" means "use the IntelliSense Engine" instead of just the "tag parser" or nothing at all. The usage of the word "default" is completely unrelated to the "C_Cpp.default".

The C_Cpp.default.* values get used when there is no c_cpp_properties.json to overwrite those setting or "in addition" if "${default}" is used in a list of items, like includePath.

sean-mcmanus commented 4 years ago

See also https://code.visualstudio.com/docs/cpp/customize-default-settings-cpp

vsjha18 commented 4 years ago

I have a file called "student.h" with the following code.

struct student {
    char* name;
    int age;
}

main.c has the following code

#include "student.h"

struct student s;

Now while typing main.c following things happen.

PS: I am not a C programmer so my question may sound absurd. My programming background is mostly in Python/JS/Perl/Shell/Tcl. I am neither aspiring to be a C programmer. My job primarily involves adoption of VS Code in my organization amongst the C developer community.

vsjha18 commented 4 years ago

On the similar note for the following simple sunction

long addition(long a, long b)
{
   long result;

   result = a + b;

   return result;
}
long sum = addition(10, 20);

While calling this function, it provides suggestion of the function name which is good, and also shows its signature. But while typing the arguments between parentheses, there is not help provided for typing arguments.

Also there is no error shown if there is a type error or argument mismatch, is there any setting which can help enabling this, just in case I have accidentally disabled it or something.

I think this is very simple feature to expect that in a strongly typed language, IDE will at least provide type checking.

sean-mcmanus commented 4 years ago

We tracking C "struct s" completion with https://github.com/microsoft/vscode-cpptools/issues/4661 .

I get completions with s. Are you not in a function? Your code should get a compile error if it's in a global context ("initializer element is not constant").

image

I get argument info, but only when the code is in a function. As before, your code should get a compile error if it's in a global context ("initializer element is not constant").

image

We normally do show errors when there are type errors. Can you change your C_Cpp.errorSquiggles setting to "Enabled"? Be aware that in C, a non-declared function can be treated as a function returning an int (i.e. you'll see that when you hover over it).

github-actions[bot] commented 4 years ago

This feature request is being closed due to insufficient upvotes. When enough upvotes are received, this issue will be eligible for our backlog.

github-actions[bot] commented 2 years ago

This feature request has received enough votes to be added to our backlog.

mrx23dot commented 3 months ago

I would suggest to disable intellisense for project that big (there should be an option for that), and use text based search. (or maybe limit intellisense to current file only)