[Perf] Parallel `sync` and `update`

nvim-neorocks / rocks.nvim

Neovim plugin management inspired by Cargo, powered by luarocks

GNU General Public License v3.0

563 stars 8 forks source link

[Perf] Parallel `sync` and `update` #413

Open mrcjkb opened 2 weeks ago

mrcjkb commented 2 weeks ago

I think I have an idea for this:

Option 1

First, install (shared?) dependencies sequentially
Then update rocks in parallel

Option 2

Implement a per-package locking mechanism in luarocks

vhyrro commented 2 weeks ago

I'm inclined to start with option 1 and refactor in case option 2 ever becomes a reality. We could pester hisham a bit to try and make that work :p (hell, maybe I can try to contribute, seems like a simplish thing to implement).

The processing time of checking shared dependencies is outweighed by the speed gains of parallel updates.

mrcjkb commented 4 days ago

Played around with this a bit more on the parallel-installs branch. The luarocks --force-lock option is completely broken when doing parallel installs.

luarocks needs either a way to lock installs per directory or an --ignore-lock option.

mrcjkb commented 4 days ago

Added an --ignore-lock flag to my fork of luarocks. Some packages seem to update fine, but others fail with

Warning: Failed searching manifest: Failed loading manifest for https://nvim-neorocks.github.io/rocks-binaries-dev/: Error loading file: [string "/tmp/luarocks_local_cache-7694486/https___nvi..."]:2887: ']' expected near '}'

error: no results matching query were found for lua 5.1.
to check if it is available for other lua versions, use --check-lua-versions.

It looks like we may have to improve the luarocks locking mechanism.