Bare compiler should not require a hex registry

[ericmj]

It complains until I run rebar3 update and then works.

[ferd]

Would there be a problem to having the compiler fetch its own registry? The problem is possibly (as you pointed out on IRC) with plugins required in order to properly compile. In such cases, unless the user of the bare compiler were to fetch plugins and compile them on rebar3's behalf then somehow passed in the proper path, I don't think we could avoid the registry being there.

[ericmj]

If the compiler automatically fetches the registry or if mix can pass the path of its own registry to rebar works for me.

[ericmj]

Another option would be for mix to treat plugins as normal dependencies so we fetch. In this case they should be available when rebar runs and fetching the registry would not be necessary for rebar. I will try this out and let you know how it works out.

[gleber]

This also is relevant to making rebar3 useful for building stuff in hermetic environment (e.g. Debian effort to make packages reproducible and Nix/NixOS build processes). Currently I need to work around this by manually providing registry file, as seen here: https://github.com/NixOS/nixpkgs/pull/11651/files#diff-177ea02978ffc72f964033695c5e023eR9

This registry is not really needed, since all dependencies of built project are already provided locally and, I believe, rebar has enough information from rebar.config to validate that deps are the same as requested.

[ericmj]

Ping?

[tsloughter]

Pong. not really sure what to do here.

[gleber]

Currently rebar3 combines two things in one: 1) compiler wrapper (traversing packages based on config file and building them correctly) 2) dependency manager (e.g. downloading, locking, etc.)

I believe Eric means to not do any of (2) if everything mentioned in config files is available locally to be compiled.

[ericmj]

With the bare compiler rebar assumes that the user have provided all dependencies and that they are correct so rebar should not need the registry to proceed.

[tsloughter]

Ah. hm. I suppose doing plugin install in app_discovery is wrong anyway and could be moved to a new provider. I'll look at doing that today.

But plugins then won't be installed or used which isn't right either, and I don' tknow how to resolve that.

[ericmj]

If possible the way I would like plugins to be handled would be that I make sure the plugins rebar need are available in the code path the same way I do for dependencies. Would that make sense?

[tsloughter]

The problem there is rebar3 treats plugins differently from deps so it doesn't accept a dep as a plugin. As in, if you have a dep A at vsn X you may have a plugin that also has dep A but at vsn Y. So plugins are installed to _build/<profile>/plugins and not _build/<profile>/lib.

[tsloughter]

That is unless you pass a plugins_dir in the config that could set it to lib/.

[ericmj]

If rebar handles plugins separately from deps there is no reason for mix to handle them because as far as I can tell they will never conflict with mix deps. So we can close this issue for my part.

Could we tell rebar where the registry is located so we don't have to make multiple requests to fetch it? Keep in mind that mix may call rebar multiple times to compile dependencies.

[tsloughter]

We don't use the original registry for package look up :(. It gets converted to a packages.idx ets table on disk. But still possible if you are ok with it writing packages.idx to the directory with the registry.

[ericmj]

@tsloughter That would be perfectly fine.

[ferd]

@ericmj is this still required? Trying to filter issues.

[ericmj]

It's not implemented yet afaik but it's also not a hard requirement. It works right now, this feature would just make builds faster.

On Tuesday, 7 June 2016, Fred Hebert notifications@github.com wrote:

@ericmj https://github.com/ericmj is this still required? Trying to filter issues.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/erlang/rebar3/issues/958#issuecomment-224150502, or mute the thread https://github.com/notifications/unsubscribe/AATV2od4RHsWt8vbpAp56q_hUXWDnFmYks5qJNi7gaJpZM4GuCua .

Eric Meadows-Jönsson

[tsloughter]

Is this still the case?

[ferd]

I think there was a way to do it with only the local registry, at least since we shipped the v2 of the hex API, since we now work from a partial index rather than a complete one.