bitwalker / distillery

Simplify deployments in Elixir with OTP releases!
MIT License
2.96k stars 396 forks source link

`include_src: true` does not include source of Elixir apps #683

Closed hrubi closed 5 years ago

hrubi commented 5 years ago

Steps to reproduce

Reproducing on bare myapp created by mix new myapp with added distillery dependency.

$ mix release.init
$ sed -i 's/include_src: false/include_src: true/' rel/config.exs
$ MIX_ENV=prod mix release
...

$ ls -1 _build/prod/rel/myapp/lib
artificery-0.4.2
compiler-7.2.6
distillery-2.0.14
elixir-1.8.2
iex-1.8.2
kernel-6.1
logger-1.8.2
mix-1.8.2
myapp-0.1.0
runtime_tools-1.13.1
sasl-3.2.1
stdlib-3.6

$ find _build/prod/rel/myapp/lib -mindepth 1 -maxdepth 2 -type d -name src -o -name lib
_build/prod/rel/myapp/lib/sasl-3.2.1/src
_build/prod/rel/myapp/lib/kernel-6.1/src
_build/prod/rel/myapp/lib/runtime_tools-1.13.1/src
_build/prod/rel/myapp/lib/stdlib-3.6/src
_build/prod/rel/myapp/lib/compiler-7.2.6/src

Verbose Logs

``` % MIX_ENV=prod mix release --verbose ==> Loading configuration.. ==> Assembling release.. ==> Building release myapp:0.1.0 using environment prod ==> Discovered applications: > kernel-6.1 | | from: /usr/lib64/erlang/lib/kernel-6.1 | applications: none | includes: none |_____ > artificery-0.4.2 | | from: _build/prod/lib/artificery | applications: | :kernel | :stdlib | :elixir | includes: none |_____ > distillery-2.0.14 | | from: _build/prod/lib/distillery | applications: | :kernel | :stdlib | :elixir | :runtime_tools | :artificery | includes: none |_____ > runtime_tools-1.13.1 | | from: /usr/lib64/erlang/lib/runtime_tools-1.13.1 | applications: | :kernel | :stdlib | includes: none |_____ > stdlib-3.6 | | from: /usr/lib64/erlang/lib/stdlib-3.6 | applications: | :kernel | includes: none |_____ > logger-1.8.2 | | from: /usr/lib64/elixir/bin/../lib/logger | applications: | :kernel | :stdlib | :elixir | includes: none |_____ > myapp-0.1.0 | | from: _build/prod/lib/myapp | applications: | :kernel | :stdlib | :elixir | :logger | :distillery | includes: none |_____ > compiler-7.2.6 | | from: /usr/lib64/erlang/lib/compiler-7.2.6 | applications: | :kernel | :stdlib | includes: none |_____ > sasl-3.2.1 | | from: /usr/lib64/erlang/lib/sasl-3.2.1 | applications: | :kernel | :stdlib | includes: none |_____ > iex-1.8.2 | | from: /usr/lib64/elixir/bin/../lib/iex | applications: | :kernel | :stdlib | :elixir | includes: none |_____ > mix-1.8.2 | | from: /usr/lib64/elixir/bin/../lib/mix | applications: | :kernel | :stdlib | :elixir | includes: none |_____ > elixir-1.8.2 | | from: /usr/lib64/elixir/bin/../lib/elixir | applications: | :kernel | :stdlib | :compiler | includes: none |_____ ==> Running validation checks.. > Mix.Releases.Checks.Erts * PASS > Mix.Releases.Checks.Cookie * PASS > Mix.Releases.Checks.LoadedOrphanedApps * PASS ==> Generated overlay vars: release_name=:myapp release_version="0.1.0" is_upgrade=false upgrade_from=:latest dev_mode=false include_erts=true include_src=true include_system_libs=true erl_opts="" run_erl_env="" erts_vsn="10.1.1" output_dir="_build/prod/rel/myapp" ==> Copying applications to _build/prod/rel/myapp ==> Generating start_erl.data ==> Generating vm.args from rel/vm.args ==> Generating sys.config from config/config.exs ==> Including ERTS 10.1.1 from /usr/lib64/erlang/erts-10.1.1 ==> Generating boot scripts ==> Generating RELEASES ==> Applying overlays ==> Applying mkdir overlay dst: releases/0.1.0/hooks ==> Applying mkdir overlay dst: releases/0.1.0/hooks/pre_configure.d ==> Applying mkdir overlay dst: releases/0.1.0/hooks/post_configure.d ==> Applying mkdir overlay dst: releases/0.1.0/hooks/pre_start.d ==> Applying mkdir overlay dst: releases/0.1.0/hooks/post_start.d ==> Applying mkdir overlay dst: releases/0.1.0/hooks/pre_stop.d ==> Applying mkdir overlay dst: releases/0.1.0/hooks/post_stop.d ==> Applying mkdir overlay dst: releases/0.1.0/hooks/pre_upgrade.d ==> Applying mkdir overlay dst: releases/0.1.0/hooks/post_upgrade.d ==> Applying copy overlay src: _build/prod/lib/distillery/priv/libexec dst: releases/0.1.0/libexec ==> Applying mkdir overlay dst: releases/0.1.0/commands ==> Packaging release.. ==> Archiving myapp-0.1.0 ==> Writing archive to /home/hrubi/tmp/myapp/_build/prod/rel/myapp/releases/0.1.0/myapp.tar.gz ==> Updating archive.. ==> Including system libs from current Erlang installation ==> Saving archive.. ==> Archive saved! Release successfully built! To start the release you have built, you can use one of the following tasks: # start a shell, like 'iex -S mix' > _build/prod/rel/myapp/bin/myapp console # start in the foreground, like 'mix run --no-halt' > _build/prod/rel/myapp/bin/myapp foreground # start in the background, must be stopped with the 'stop' command > _build/prod/rel/myapp/bin/myapp start If you started a release elsewhere, and wish to connect to it: # connects a local shell to the running node > _build/prod/rel/myapp/bin/myapp remote_console # connects directly to the running node's console > _build/prod/rel/myapp/bin/myapp attach For a complete listing of commands and their use: > _build/prod/rel/myapp/bin/myapp help ```

Description of issue

Running mix release with include_src: true includes sources only for the apps which has an OTP compliant directory layout in the place where distillery looks. In my case, that's just system-wide Erlang libraries (e.g. kernel, stdlib).

For the applications built from the repository, Distillery looks only into _build/prod/lib/* where it find only ebin and possibly priv. The source code lies elsewhere, in my case it's lib/ for the myapp (would be apps/myapp/lib in the case of an umbrella app) and deps/*/lib for artificery and distillery. None of them is assembled into the release.

Also the sources for elixir, iex and mix are missing, but that's understandable as in my case the Elixir installation from package (Gentoo) does not include the source code.

The documentation says:

include_src (boolean) - should source code be included in the release so I would expect that all source code that Distillery can find should be included in the release. In my case that would be sources for myapp, distillery and artificery.

Distillery: 2.0.14 Erlang: 21 Elixir: 1.8.2 OS: Gentoo Linux

hrubi commented 5 years ago

Here's a quick workaround via a plugin: https://gist.github.com/hrubi/a2d646d49c2da00e703bb748e1a6a485

I'll try to find time to send a pull request with a proper fix to distillery.