xolox
commented
8 years ago I'm aware of the basic problems you describe, although I'm surprised it manifests to the extent that you describe (~7GB). Not easily solved is unfortunately an understatement :-P. Vim script is one of the most limited "high level" programming languages I have ever worked in and the "standard library" of functions available to Vim script is equally limited.
For example there's one function to read a file and one function to write a file, and both functions process a complete file at once. So the only way to write a file is to write all of it at once - by definition we can't use a streaming approach like UNIX pipes and ctags do. I could escape (and have escaped) to e.g. Python and that helps in all but one regard: It significantly impedes the cross-platform / portability aspect of my Vim plug-ins (in fact it pretty much undermines it).
Fundamentally every serious user will hit these limits at one point and be seriously annoyed, however over the years I have tried to provide ways to lessen the problems. For example dynamic tags files and file type specific tags files. Have you looked into these?
Just for the record: I have never heard of anyone working with a 1.4 GB tags file and yes this will blow up the vim-easytags plug-in. To be honest I don't think it's even possible to do what vim-easytags does given such large tags files. What kind of a code base are you working on, the Linux kernel? And you're not working on isolated subsystems, but all of it at once? The only approach here that I can think of is basically divide and conquer (i.e. work on parts of the code base, use dynamic tags files relevant to subtrees of the project and accept that you can't have vim-easytags update and highlight the complete code base as one unit).
dasizeman
Not sure what the process for :UpdateTags is, but when I run it on a large amount of source code, the vim process allocates a ton of memory (~7GB, probably more if it was available). I also noticed that the .vimtags file does not get written at all during the process, which makes me think you are trying to do too much in memory. You should be writing the .vimtags file as ctags is doing its work.
If you would like some help with this issue, let me know.
UPDATE: It seems its not just the :UpdateTags command, the whole system seems to fall to its knees with large codebases/ctags files (The ctags file for the code I'm working with is 1.4GB)
This may not be something that is easily solved, I may just have to update/manage my ctags manually with a codebase of this size.