search

erlware / relx

Sane, simple release creation for Erlang
http://erlware.github.io/relx
Apache License 2.0
697 stars 233 forks source link

Extend include_erts to let me ignore other apps as well #689

Open Zalastax opened 5 years ago

Zalastax commented 5 years ago

Relx has the option include_erts which prevents system applications from being included in the release. It also has the option exclude_apps which prevents applications I choose from being included in the release and removes the application from the .app file that relx produce. I want some way of preventing any application from being included in the release without removing it from the dependency list in the produced .app file. The use case for this is that I have applications that are available in lib_dirs that will be available on the machine that I deploy, that I want my application to depend on, but that I don't want to be included in the release.

What I want in terms of code change is some way to expand the check made here: https://github.com/erlware/relx/blob/5e7bd9d650d925e9afff13a5bdcea9d6356a2f76/src/rlx_prv_assembler.erl#L205-L215

A simple way to achieve that is to add some new option but I can't come up with a name that won't be confusing. A more heavy modification would be to have an option that subsumes include_erts and exclude_apps with entries like this: [{erts, not_included}, {cowboy, removed}, {poolboy, not_included}] which in this case would produce a relase that does not include erts, poolboy or cowboy, but where erts and poolboy are left in as dependencies and cowboy is removed from the dependencies.

I can do the development if you find this to be an OK idea.

lrascao commented 5 years ago

what if we keep include_erts as is and expand exclude_apps with the option you mentioned (tuple {app, not_included}) while still keeping retro-compat, the default behaviour would be removed, could this work?

ferd commented 5 years ago

that might totally be incompatible with some features like relups and strict loading modes, and will cause the release to crash on start.

Zalastax commented 5 years ago

My organization is just warming up to using rebar3 and relx. The releases we create are merged into a bigger release. For our system to work we want to reference dependencies in rebar3 but not include them. We get the behavior we want from releases via exclude_apps with the only problem being that the excluded app is removed from the released .app. The apps that we don't include are delivered to the machine via some other means so we want them in the .app file and will not cause the release to crash on start - we have verified this by manually adding the dependencies to the generated .app file. Think of it as us having an extended standard library on the machine we deploy to.

tsloughter commented 5 years ago

@Zalastax are you using reltool or something else currently for building releases?

Zalastax commented 5 years ago

@tsloughter it's Make and magic :sweat_smile:

Zalastax commented 5 years ago

@ferd @tsloughter may I go ahead and implement this according to @lrascao's suggestion?

ferd commented 5 years ago
  1. I'm technically not a maintainer here so what I say isn't a blocker
  2. I do maintain that this is a feature that might totally break for regular users who are not transitioning from Custom Make Magic™️ and would be nervous about providing such a feature were I actually a maintainer
Zalastax commented 5 years ago

@ferd your feedback is still valuable and I want to make sure that my contribution will be accepted. I know that it's a bit controversial but it will really help us transition to using rebar3 and relx more. If you have alternative suggestions I'm all ears - I just want to end up with something stable, easy to use, and officially supported. Basically something that doesn't feel like a hack.

One alternative solution would be if ALL dependencies could be excluded from the release (without removing them from the released .app file(s)). Would that be better or does it feel as bad and risky?

ferd commented 5 years ago

The conflict I have is that no matter how I put it, what you are asking for is a hack though. You are literally asking to build a broken release that would not work so you can patch it up elsewhere as far as I understand.

ferd commented 5 years ago

include_erts does not remove system libraries, it drops the whole virtual machine. I think what you had in mind is the system_libs option. The one case I could see is if you somehow redefined which apps are to be seen as part of ERTS by specifying system_libs. Then that would force a mode where you couldn't accidentally build a broken release without explicitly going "cowboy and cowlib are part of the standard distribution I'm going to use and are elsewhere", which might work.

Zalastax commented 5 years ago

In my experiments, it seems like include_erts controls what gets put into the rel directory and system_libs controls whether to put in system libraries into the .tar in addition to what's already in rel, i.e. {system_libs, true} has no effect when {include_erts, true}. With that said, your idea is sound in principle and basically what I want achieve. Extending what counts as an ERTS would be to change https://github.com/erlware/relx/blob/5e7bd9d650d925e9afff13a5bdcea9d6356a2f76/src/rlx_prv_assembler.erl#L218-L219 and that would have my desired effect.

tsloughter commented 5 years ago

@Zalastax in order to make this work the libraries you don't want included in the release need to be in a separate directory from the rest. Look at how make_boot_script_variables(State) works. It tells the script to use $ERTS_LIB_DIR for any library found in code:lib_dir(). So it would be possible to support this if you have a env variable you can set to where these libs are on the host and figure out a way so the libs when relx is running are in a lib directory separate from all the other apps of the project.

See variables option in http://erlang.org/doc/man/systools.html#make_script-1 and var_tar omit in http://erlang.org/doc/man/systools.html#make_tar-2