junegunn / vim-plug

:hibiscus: Minimalist Vim Plugin Manager
https://junegunn.github.io/vim-plug/
MIT License
34.17k stars 1.93k forks source link

Reduce the size of the shallow git repositories further #267

Closed Hotschke closed 9 years ago

Hotschke commented 9 years ago

Even though vim-plug creates only shallow clones, the size of the cloned repositories can be further decreased:

$ find ~/.vim/plugged -maxdepth 1 -type d sh -c '(cd {} && git reflog expire --all --expire=now && git gc --aggressive --prune=now)' ';'

This is based on http://stackoverflow.com/q/2116778/1057593.

In my current setup I could get down from 224.6MiB to 108.6MiB.

What do you think about appending this command to :PlugInstall and :PlugUpdate?

I've noticed that some plugin authors, place gifs and other larger files into the git repository which are not required for installation. I guess in those cases, I have to accept that they are on my HDD. Or does git-clone allow to ignore certain files and directories (e.g. test/*)?

Another idea would be: Some authors provide releases on github or vim.org. Would it be possible to follow only those (not using git anymore)?

junegunn commented 9 years ago

That's good to know. Thanks for the suggestion. But I don't think it's essential to vim-plug and do not wish to add the behavior to the core in order not to further complicate the internals and the test scenario. We could instead make it an external command (e.g. :PlugPrune) and add it to the wiki page so one can say PlugUpdate | PlugPrune. Feel free to edit the page :)

I've noticed that some plugin authors, place gifs and other larger files into the git repository which are not required for installation ... Another idea would be: Some authors provide releases on github or vim.org. Would it be possible to follow only those (not using git anymore)?

A better idea is just to tell them to stop doing that. They probably are not aware of the fact that those files bloat their repository and lead to poor user experience. Since git remembers the entire history, it's already too late for them to remove those files, but the users of vim-plug will still benefit thanks to shallow-clone support.

Hotschke commented 9 years ago

Thanks for your reply. I understand you very well. Actually, the maintainer of the git repositories should run $ git gc.

I am not really sure whether I want to stick with using git as backend for the plugin manager. Even small plugins by tpope (e.g. vim-unimpaired) which do not contain any large files have a .git directory around 1MiB (disc usage). Too many small files in .git.

You don't happen to know a vim plugin manager which tracks releases as .tar.gz on github.com and vim.org/scripts?

junegunn commented 9 years ago

Too many small files in .git.

If that's the concern, you could simply remove all the .git directories after installation. Though you'll have to reinstall everything whenever you want to update (for vim-plug it would be PlugInstall followed by PlugClean) Probably not an option.

You don't happen to know a vim plugin manager which tracks releases as .tar.gz on github.com and vim.org/scripts?

No, I'm not aware of any, and even if there is, I'm not sure how useful it would be, considering that

  1. plugin authors usually do not bother to create tags (vim-unimpaired for example, hasn't been tagged since 2013)
  2. many plugins these days are not even published to vim.org, and solely hosted on github.
starcraftman commented 9 years ago

@hori-ryota Hi. I agree with @junegunn that it isn't vim-plug's job to minimize the footprint of git repos. Apart from YCM, I haven't run into many huge ones. I have a bunch of plugins & YCM takes around 400M, the rest combined take about 60M. Considering most modern IDEs (Eclipse, Netbeans, VisualStudio) take many GBs this doesn't seem terribly unreasonable. Development takes space.

Regarding doing what you want, I might make a suggestion. We allow quite flexible post install hooks. Using a hook you could do something like the following python pseudo code. The actual hook would be in a VimL function/script, though if you have ruby/python support you could use those languages or even invoke a shell script of your choice.

Important: To update with below hook, you would have to PlugClean then PlugInstall. I'd also like to note, I don't recommend, I just like thinking up odd solutions :).

def post_install():
    if install:
        # Delete `.git` folder
        # Recursively delete extensions that are useless: i.e. **/*.png, **/*.jpeg, ...
        # Maybe remove folders you don't want like /doc
Hotschke commented 9 years ago

I agree that by today's standards and in comparison with other tools one can live with it.

However, wasting space for unneeded files is not in the spirit of lightweight tools such as vim. Other text editors (such as atom, sublime) also take much more space, but that's not a reason to waste disc space. I would suspect 90% of your 400MB are the git repos.

Another point I don't like about exclusively distributing software via git on the master/devel branch is that practically all users become beta tester. I don't need all commits instantaneously. Having stable software is a strong feature which I consider superior. The current situation encourages plugin authors to not release their software anymore according to classic software publishing paradigms (e.g. semver.org). At minimum they should have a stable master branch and development should occur on a separate branch (even with continuous integration). And developers should be aware of their release model, take it into account what they check-in, write as commit message and what the user gets by installation through cloning.

From a plugin infrastructure point of view both release models should be equally supported.

junegunn commented 9 years ago

Yeah, with vim-plug you can select branch or tag to use.

starcraftman commented 9 years ago

@Hotschke No just the one plugin YouCompleteMe is 400M, 100M is the .git, the rest is code, binaries and third parties. The 60M is ALL my other plugins.

Your main problem appears to be the ecosystems dev/distribution model. That ship sailed years ago when Vundle began mirroring vim.org scripts to github.com to be used with Vundle's plugin manager. So I don't see any way that will change unless something better rolls around (maybe neovim?). You are always free to fork and try your hand making it smaller/not git based plugin manager. I don't see vim-plug changing its method any time soon, so any further discussion probably moot.