[Enhancement] dash.nvim is 300mb+ (i.e. very large for a nvim plugin)

[akinsho]

Hi,

Thanks for your work on this plugin, been trying it out, and it works really smoothly, which is 👍🏿 . It took me quite a few attempts to install, though, which I narrowed down to packers 60s timeout for git clones (which I had to triple). I then had a look at the folder once downloaded and noticed that this plugin is enormous by nvim standards, weighing in at 300mb

➜ du -sh ./dash.nvim 
306M    ./dash.nvim

➜ du -sh *   
20K Cargo.lock
4.0K    Cargo.toml
20K LICENSE
4.0K    Makefile
16K README.md
4.0K    autoload
99M bin <----------------Ships with binaries for different systems
20K doc
2.0M    images
49M lua
4.0K    plugin
8.0K    scripts
12K spec
84K src
4.0K    stylua.toml
1.1M    vendor <----------------- duplicated all of plenary.

By comparison, the next largest plugin is 3mb, which was already anomalous but the culprit ships a bunch of files which is fine. Most of my other plugins are around 400kb and less. I'm not sure exactly what other rust based plugins do but I've never used one this big before which I think is down to it shipping binaries for every architecture rather than building the binary during installation or maybe there's a way to trim down the size of the program since it being that big anyway is very odd.

[mrjones2014]

Yeah so, the thing is, the way the backend is being delivered is by storing the compiled Rust binaries in the repo at the moment.

I'm open to other ideas on how to deliver the required binaries.

Maybe we could store .zip files containing the binaries in GitHub Releases or something, then have the make install script download the compressed binaries from the latest GitHub Release.

We could also remove the vendored dependencies, and have make test-lua download them if they're missing, then add ./vendor/ to .gitignore. vendor/ not only contains Plenary but also the Lua testing framework. I agree this isn't ideal.

[mrjones2014]

rather than building the binary during installation

this would require all users to have a Rust toolchain installed on their system.

[mrjones2014]

I can also look at if there's something I'm missing to trim down the size of compiled binaries.

[akinsho]

@mrjones2014 I think it's quite common to either fetch the binaries e.g. they are uploaded to GitHub and pulled down as part of an install script or even more commonly to just use rust on the user's machine. I dug out a few rust-based plugins I've historically used.

• https://github.com/michaelb/sniprun
• https://github.com/euclio/vim-markdown-composer
• https://github.com/wfxr/minimap.vim (binary shipped completely separately)
• https://github.com/chipsenkbeil/distant.nvim

I'm less keen on the separate binary as that's much more error prone/more steps involved for the user and less portable, but I think starting a terminal job that pulls the binary is doable with the job api or termopen.

[mrjones2014]

Right but I don't want users to be required to have a Rust toolchain just to use this plugin. I think downloading the binaries at install time is the best option here.

Also removing vendored dependencies.

[akinsho]

That's fair enough (re. the toolchain, biased since I already have it installed 😆 ) anyway an example of a plugin that does this (if it helps) is https://github.com/tzachar/cmp-tabnine

[akinsho]

As a side note (which you might already know) even if you remove them they will be in the git history, so the repo will remain large. Think you'll have to do a git filter, this will unfortunately rewrite history which is the only way to do this, so users will need to deal with the history rewrite which if they use packer in the current nvim ecosystem they are probably familiar with.

[mrjones2014]

Cool, I knew I'd have to purge the git history but wasn't sure how, so thanks for the link.

[mrjones2014]

@akinsho can you test a clean, fresh install from this branch? matjones/131-remove-bin-from-repo

Since I did the git filter thing, GitHub sees them as completely different git histories and won't let me make a PR, so once I can test this out, I'll have to force-push to master I think.

[mrjones2014]

The branch above changes it so that binaries are downloaded as a zip from GitHub Release and unzipped into the right place.

[akinsho]

👍🏿 I'll give it a go now

[mrjones2014]

Seems like this worked. Closing. Plugin now downloads the binaries from latest GitHub Release.

[akinsho]

Thanks @mrjones2014 👍🏿

[S1M0N38]

dash.nvim/.git/objects/pack/pack-*.pack is 199 MB. So when you clone this repo (or your plugin manager clone it) you would also pull this file which it's not really used by Neovim. I suggest to "rewrite" git history in order to drastically reduce the size of this plugin (from 201 MB to 2 MB). Here it's explain the right way to do it.

This optimization should also help with timeouts related issues

[mrjones2014]

I thought I’d done that but maybe I did something wrong. I’ll try again.