Artifact downloads limited by max-jobs

[bendlas]

When the jar and pom files for a classpath are not in /nix/store, the resulting downloads are treated as regular builds and can be very slow if max-jobs is low.

This is essentially the same problem https://github.com/nix-community/yarn2nix/issues/9#issuecomment-437074958

Possible workarounds

• A command that pre-builds just the classpath and adds --option max-jobs 256
• Maybe builtins.fetchurl wouldcount against http-downloads?

[jlesquembre]

A command that pre-builds just the classpath and adds --option max-jobs 256

That's for sure an option. Is your use case to speed up CI builds?

If not on CI, there is an option to add the dependencies to the nix store when you generate the lock file, search for CLJNIX_ADD_NIX_STORE in the README:

CLJNIX_ADD_NIX_STORE=true nix run github:jlesquembre/clj-nix#deps-lock

Probably I should add a flag to the CLI to do that.

[bendlas]

That's for sure an option. Is your use case to speed up CI builds?

Yes, also

• first-time user: time to command
• existing user: checking out older versions after gc

Probably I should add a flag to the CLI to do that.

Interesting, a flag to just transfer to nix store - without updating any versions - would also solve these cases, right?

[jlesquembre]

@bendlas to clarify it, a new CLI command, like

add-clj-deps deps-lock.json

which adds all the dependencies to the nix store, solves your use case, right?

[bendlas]

Yes, a cli command like preload-deps to fast-track all the dependency downloads would solve my use cases neatly.