Successive hot upgrade releases produce error

clessg commented 7 years ago

Steps to reproduce

Assuming an app named blog at version 0.0.1...

MIX_ENV=prod mix release --env=prod
PORT=4000 _build/prod/rel/blog/bin/blog start
Bump version to 0.0.2.
MIX_ENV=prod mix release --env=prod --upgrade
_build/prod/rel/blog/bin/blog upgrade '0.0.2'
Bump version to 0.0.3.
MIX_ENV=prod mix release --env=prod --upgrade

Am I doing something wrong? The final step results in an error:

==> Assembling release..
==> Building release blog:0.0.3 using environment prod
==> Including ERTS 8.1 from /usr/local/Cellar/erlang/19.1/lib/erlang/erts-8.1
==> Failed to build release:
    Missing .rel for blog:blog.rel at _build/prod/rel/blog/releases/blog.rel/blog.rel

Verbose Logs

==> Loading configuration..
==> Assembling release..
==> Building release blog:0.0.3 using environment prod
==> Discovered applications:
  connection-1.0.4
    from: _build/prod/lib/connection
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  db_connection-1.1.0
    from: _build/prod/lib/db_connection
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :connection
    includes: none

  postgrex-0.12.1
    from: _build/prod/lib/postgrex
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :db_connection
      :decimal
    includes: none

  poolboy-1.5.1
    from: _build/prod/lib/poolboy
    applications:
      :kernel
      :stdlib
    includes: none

  decimal-1.3.1
    from: _build/prod/lib/decimal
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  ecto-2.0.6
    from: _build/prod/lib/ecto
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :decimal
      :poolboy
    includes: none

  phoenix_ecto-3.0.1
    from: _build/prod/lib/phoenix_ecto
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :ecto
      :plug
    includes: none

  gettext-0.12.2
    from: _build/prod/lib/gettext
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
    includes: none

  cowlib-1.0.2
    from: _build/prod/lib/cowlib
    applications:
      :kernel
      :stdlib
      :crypto
    includes: none

  ranch-1.2.1
    from: _build/prod/lib/ranch
    applications:
      :kernel
      :stdlib
    includes: none

  cowboy-1.0.4
    from: _build/prod/lib/cowboy
    applications:
      :kernel
      :stdlib
      :ranch
      :cowlib
      :crypto
    includes: none

  phoenix_html-2.8.0
    from: _build/prod/lib/phoenix_html
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :plug
    includes: none

  phoenix_pubsub-1.0.1
    from: _build/prod/lib/phoenix_pubsub
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :crypto
    includes: none

  eex-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/eex
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  poison-2.2.0
    from: _build/prod/lib/poison
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  mime-1.0.1
    from: _build/prod/lib/mime
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  logger-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/logger
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  crypto-3.7.1
    from: /usr/local/Cellar/erlang/19.1/lib/erlang/lib/crypto-3.7.1
    applications:
      :kernel
      :stdlib
    includes: none

  plug-1.3.0
    from: _build/prod/lib/plug
    applications:
      :kernel
      :stdlib
      :elixir
      :crypto
      :logger
      :mime
    includes: none

  phoenix-1.2.1
    from: _build/prod/lib/phoenix
    applications:
      :kernel
      :stdlib
      :elixir
      :plug
      :poison
      :logger
      :eex
    includes: none

  compiler-7.0.2
    from: /usr/local/Cellar/erlang/19.1/lib/erlang/lib/compiler-7.0.2
    applications:
      :kernel
      :stdlib
    includes: none

  elixir-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/elixir
    applications:
      :kernel
      :stdlib
      :compiler
    includes: none

  stdlib-3.1
    from: /usr/local/Cellar/erlang/19.1/lib/erlang/lib/stdlib-3.1
    applications:
      :kernel
    includes: none

  kernel-5.1
    from: /usr/local/Cellar/erlang/19.1/lib/erlang/lib/kernel-5.1
    applications: none
    includes: none

  blog-0.0.3
    from: _build/prod/lib/blog
    applications:
      :kernel
      :stdlib
      :elixir
      :phoenix
      :phoenix_pubsub
      :phoenix_html
      :cowboy
      :logger
      :gettext
      :phoenix_ecto
      :postgrex
    includes: none

  iex-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/iex
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  sasl-3.0.1
    from: /usr/local/Cellar/erlang/19.1/lib/erlang/lib/sasl-3.0.1
    applications:
      :kernel
      :stdlib
    includes: none

==> Generated overlay vars:
    release_name=:blog
    release_version="0.0.3"
    is_upgrade=true
    upgrade_from="blog.rel"
    dev_mode=false
    include_erts=true
    include_src=false
    include_system_libs=true
    erl_opts=""
    erts_vsn="8.1"
    output_dir="_build/prod/rel/blog"
==> Copying applications to _build/prod/rel/blog
==> Generating nodetool
==> Generating start_erl.data
==> Generating vm.args
==> Generating sys.config from config/config.exs
==> Including ERTS 8.1 from /usr/local/Cellar/erlang/19.1/lib/erlang/erts-8.1
==> Generating boot script
==> Generating RELEASES
==> Generating start_clean.boot
==> Generating relup for blog
==> Failed to build release:
    Missing .rel for blog:blog.rel at _build/prod/rel/blog/releases/blog.rel/blog.rel

Description of issue

What OS, Erlang/Elixir versions are you seeing this issue on?

Tested on both Mac and Ubuntu using Distillery 1.0 and Distillery 0.10.

MacOS Sierra - erlang 19, elixir 1.3.4 Ubuntu 16.04 - erlang 19.1, elixir 1.3.4

If possible, also provide your rel/config.exs, as it is often my first troubleshooting question, and you'll save us both time :)

It's just the default generated config (Distillery 1.0):

use Mix.Releases.Config,
    # This sets the default release built by `mix release`
    default_release: :default,
    # This sets the default environment used by `mix release`
    default_environment: :dev

# For a full list of config options for both releases
# and environments, visit https://hexdocs.pm/distillery/configuration.html

# You may define one or more environments in this file,
# an environment's settings will override those of a release
# when building in that environment, this combination of release
# and environment configuration is called a profile

environment :dev do
  set dev_mode: true
  set include_erts: false
  set cookie: :";|WDO,|z>fknppaTF&?n7!2e=3aN$&x@[*AzFm8{DN%{]Sk829A:ac:DHdIO_&.|"
end

environment :prod do
  set include_erts: true
  set include_src: false
  set cookie: :"oM:!U^pMIE4jP/!jSe9QS`55|yh}&Y8sO2~t1fqf7zYM.HNn7ovnbN,q?=`lEVnB"
end

# You may define one or more releases in this file.
# If you have not set a default release, or selected one
# when running `mix release`, the first release in the file
# will be used by default

release :blog do
  set version: current_version(:blog)
end

bitwalker commented 7 years ago

I'm unable to reproduce this with master, could you give that a shot? If it works, I'll cut a bugfix release.

JamesLavin commented 7 years ago

First, Paul, THANK YOU for creating and sharing all these great tools! I'm getting started with Elixir and am very lucky to have access to such amazing OSS.

I did exactly the same thing as Chris, with exactly the same results. Version 0.0.1 worked fine. Hot upgrading to version 0.0.2 worked great. But I couldn't build an upgrade version after bumping my app to version 0.0.3.

I'm on a Mac with Distillery 1.0 and running Erlang/OTP 19 [erts-8.2].

I created a repo (https://github.com/JamesLavin/elixir_distillery_hot_code_reloading/commits/master) with my three code versions, each as a separate commit. You could try replicating this using that repo.

Again, thank you for donating your time to build such cool stuff for the community.

--James Lavin


==> Loading configuration..
==> Assembling release..
==> Building release phoenix_env_settings:0.0.3 using environment prod
==> Discovered applications:
  connection-1.0.4
    from: _build/prod/lib/connection
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  db_connection-1.1.0
    from: _build/prod/lib/db_connection
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :connection
    includes: none

  postgrex-0.13.0
    from: _build/prod/lib/postgrex
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :db_connection
      :decimal
    includes: none

  poolboy-1.5.1
    from: _build/prod/lib/poolboy
    applications:
      :kernel
      :stdlib
    includes: none

  decimal-1.3.1
    from: _build/prod/lib/decimal
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  ecto-2.1.1
    from: _build/prod/lib/ecto
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :decimal
      :poolboy
    includes: none

  phoenix_ecto-3.2.1
    from: _build/prod/lib/phoenix_ecto
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :ecto
      :plug
    includes: none

  gettext-0.13.0
    from: _build/prod/lib/gettext
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
    includes: none

  cowlib-1.0.2
    from: _build/prod/lib/cowlib
    applications:
      :kernel
      :stdlib
      :crypto
    includes: none

  ranch-1.2.1
    from: _build/prod/lib/ranch
    applications:
      :kernel
      :stdlib
    includes: none

  cowboy-1.0.4
    from: _build/prod/lib/cowboy
    applications:
      :kernel
      :stdlib
      :ranch
      :cowlib
      :crypto
    includes: none

  phoenix_html-2.9.1
    from: _build/prod/lib/phoenix_html
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :plug
    includes: none

  phoenix_pubsub-1.0.1
    from: _build/prod/lib/phoenix_pubsub
    applications:
      :kernel
      :stdlib
      :elixir
      :logger
      :crypto
    includes: none

  eex-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/eex
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  poison-2.2.0
    from: _build/prod/lib/poison
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  mime-1.0.1
    from: _build/prod/lib/mime
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  logger-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/logger
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  crypto-3.7.2
    from: /usr/local/Cellar/erlang/19.2/lib/erlang/lib/crypto-3.7.2
    applications:
      :kernel
      :stdlib
    includes: none

  plug-1.3.0
    from: _build/prod/lib/plug
    applications:
      :kernel
      :stdlib
      :elixir
      :crypto
      :logger
      :mime
    includes: none

  phoenix-1.2.1
    from: _build/prod/lib/phoenix
    applications:
      :kernel
      :stdlib
      :elixir
      :plug
      :poison
      :logger
      :eex
    includes: none

  compiler-7.0.3
    from: /usr/local/Cellar/erlang/19.2/lib/erlang/lib/compiler-7.0.3
    applications:
      :kernel
      :stdlib
    includes: none

  elixir-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/elixir
    applications:
      :kernel
      :stdlib
      :compiler
    includes: none

  stdlib-3.2
    from: /usr/local/Cellar/erlang/19.2/lib/erlang/lib/stdlib-3.2
    applications:
      :kernel
    includes: none

  kernel-5.1.1
    from: /usr/local/Cellar/erlang/19.2/lib/erlang/lib/kernel-5.1.1
    applications: none
    includes: none

  phoenix_env_settings-0.0.3
    from: _build/prod/lib/phoenix_env_settings
    applications:
      :kernel
      :stdlib
      :elixir
      :phoenix
      :phoenix_pubsub
      :phoenix_html
      :cowboy
      :logger
      :gettext
      :phoenix_ecto
      :postgrex
    includes: none

  iex-1.3.4
    from: /usr/local/Cellar/elixir/1.3.4/bin/../lib/iex
    applications:
      :kernel
      :stdlib
      :elixir
    includes: none

  sasl-3.0.2
    from: /usr/local/Cellar/erlang/19.2/lib/erlang/lib/sasl-3.0.2
    applications:
      :kernel
      :stdlib
    includes: none

==> Generated overlay vars:
    release_name=:phoenix_env_settings
    release_version="0.0.3"
    is_upgrade=true
    upgrade_from="phoenix_env_settings.rel"
    dev_mode=false
    include_erts=true
    include_src=false
    include_system_libs=true
    erl_opts=""
    erts_vsn="8.2"
    output_dir="_build/prod/rel/phoenix_env_settings"
==> Copying applications to _build/prod/rel/phoenix_env_settings
==> Generating nodetool
==> Generating start_erl.data
==> Generating vm.args
==> Generating sys.config from config/config.exs
==> Including ERTS 8.2 from /usr/local/Cellar/erlang/19.2/lib/erlang/erts-8.2
==> Generating boot script
==> Generating RELEASES
==> Generating start_clean.boot
==> Generating relup for phoenix_env_settings
==> Failed to build release:
    Missing .rel for phoenix_env_settings:phoenix_env_settings.rel at _build/prod/rel/phoenix_env_settings/releases/phoenix_env_settings.rel/phoenix_env_settings.rel```

JamesLavin commented 7 years ago

I think I see the bug:

The error message is saying it can't find: _build/prod/rel/phoenix_env_settings/releases/phoenix_env_settings.rel/phoenix_env_settings.rel

I think it should be looking for this file, which exists: _build/prod/rel/phoenix_env_settings/releases/phoenix_env_settings.rel

It added an extra /phoenix_env_settings.rel

JamesLavin commented 7 years ago

@clessg The error message you posted says "Missing .rel for blog:blog.rel at _build/prod/rel/blog/releases/blog.rel/blog.rel". I'm betting you have a file at _build/prod/rel/blog/releases/blog.rel but not at _build/prod/rel/blog/releases/blog.rel/blog.rel. Could you please confirm this?

clessg commented 7 years ago

@JamesLavin Yep, that is correct!

bitwalker commented 7 years ago

@JamesLavin Thanks for the the test case, and the kind words! I'll take a look as soon as possible.

JamesLavin commented 7 years ago

@bitwalker No rush! Please enjoy the holidays! This can definitely wait till the new year.

sorliem commented 7 years ago

@clessg I'm having the same issue. I went through the same scenario with an app called kv:

make a release at 0.1.0 and start it
make an upgrade from 0.1.0 -> 0.2.0
apply upgrade to 0.2.0, no problem
make an upgrade from 0.2.0 -> 0.3.0, error your are seeing

However, I noticed that the kv.rel after applying the 0.2.0 upgrade moved from the releases/0.2.0 directory up one level into releases/. So this is what my releases/ directory looked after applying an upgrade to 0.2.0:

kv[develop !]$ tree -L 2 releases/
releases/
├── 0.1.0
│   ├── commands
│   ├── hooks
│   ├── kv.boot
│   ├── kv.rel
│   ├── kv.script
│   ├── kv.sh
│   ├── kv.tar.gz
│   ├── start_clean.boot
│   ├── sys.config
│   └── vm.args
├── 0.2.0
│   ├── commands
│   ├── hooks
│   ├── kv.boot
│   ├── kv.script
│   ├── kv.sh
│   ├── relup
│   ├── start.boot
│   ├── start_cleanboot
│   ├── sys.config
│   └── vm.args
├── RELEASES
├── kv.rel
└── start_erl.data

To get an upgrade from 0.0.2 -> 0.0.3 to work, I moved the kv.rel back into the 0.2.0 directory. I'm thinking the currently running version of the app's rel file gets moved up a level, but then that causes problems if you want to make an upgrade from an upgrade.

bitwalker commented 7 years ago

@sorliem Are you generating releases and running them from the same directory? This isn't supported because the release handler removes some files and moves others around, making it impossible to do more than one upgrade from there - you must deploy to another directory, and upgrade from there.

bitwalker commented 7 years ago

I only just noticed that the original issue confirms that this issue was occurring because of running from _build. I'm going to close this for now, please reopen it if you see this occur even when deploying to another directory.

michaelwa commented 7 years ago

I seem to be having this same issue with a twist. I'm working through the "Phoenix Walkthrough" and have deployed and run version 0.0.1. I have just completed the steps for version 0.0.2 and run the command and I am getting the following results.

05:28 $ ./node_modules/brunch/bin/brunch b -p && MIX_ENV=prod mix do phoenix.digest, release --env=prod --upgrade
18 Feb 05:30:37 - info: compiling
18 Feb 05:30:37 - info: compiled 6 files into 2 files, copied 3 in 4 sec
Check your digested files at "priv/static"
==> Assembling release..
==> Building release phoenix_distillery:0.0.2 using environment prod
==> Including ERTS 8.1 from /usr/local/Cellar/erlang/19.1/lib/erlang/erts-8.1
==> Failed to build release:
    Missing .rel for phoenix_distillery:.DS_Store at _build/prod/rel/phoenix_distillery/releases/.DS_Store/phoenix_distillery.rel

Note the .DS_Store inserted into the path. The file does exist in the location the path indicates if you remove .DS_Store.

I am running version 1.1. of distillery. I'm not sure what else to provide.

bitwalker commented 7 years ago

Are you reading a version from the file system? The .DS_Store is showing up where the version should be

michaelwa commented 7 years ago

Hi thanks for the quick reply!!

I'm not sure what you mean (complete newbie). I'm not sure how I would change where it reads from. The releases folder has the 0.0.2 folder with the same contents as the 0.0.1 folder except for the tar.gz file and the 'commands' and 'hooks' folders.

michaelwa commented 7 years ago

This is still an issue for me. I have followed the directions in the "Phoenix Walkthrough" and build 0.0.1 works as expected. But when I continue to build 0.0.2 I get the error listed above and the tar.gz file is not generated after calling the command. The commands and hooks folders are also not generated.

I do not see any directions about pointing it to a new location to read from.

bitwalker commented 7 years ago

Could you update Distillery to point to the latest commit on master? Looks like you have a .DS_Store file in your releases directory somehow, and it wasn't being properly filtered.

michaelwa commented 7 years ago

Thanks!!

I updated mix.exs with this:

{:distillery, github: "bitwalker/distillery" },

Then I have run the following commands in all manner of orders: mix deps.clean distillery mix deps.compile distillery mix deps.update distillery

But keep getting this error:

13:17 $ mix deps.compile distillery
==> distillery
Compiling 18 files (.ex)

== Compilation error on file lib/mix/lib/releases/archiver.ex ==
** (CompileError) lib/mix/lib/releases/archiver.ex:152: cannot invoke local when/2 inside match, called as: {:error, {name, reason}} when :erlang.is_list(name)
    (stdlib) lists.erl:1354: :lists.mapfoldl/3
    (stdlib) lists.erl:1355: :lists.mapfoldl/3
    (stdlib) lists.erl:1354: :lists.mapfoldl/3

could not compile dependency :distillery, "mix compile" failed. You can recompile this dependency with "mix deps.compile distillery", update it with "mix deps.update distillery" or clean it with "mix deps.clean distillery"

noizu commented 7 years ago

I'm also running into this problem. It appears that first --upgrade correctly finds (or creates) the necessary rel file but the subsequent call fails while searching for the rel file under _build/%type/rel/%project/releases/%project.rel/%project.rel

adding the directory %project.rel and moving the file %project.rel into that directory solves the problem (although perhaps there are unintended consequences.).

So my guess is something is off with the logic for fetching or saving the %project.rel file.

Possibly upfrom is being set incorrectly and causing problems on line 340 of assembler.ex, looks like a new release just came out so I'll i'll have to upgrade and recheck.

bitwalker commented 7 years ago

@noizu You can't run upgrades from _build. Upgrading a release consumes files which cause subsequent upgrade builds to fail if you are building and deploying them in the same directory. If you want to test upgrades, please follow one of the guides on handling upgrades. My own workflow is this:

Build a release
Deploy it to /tmp/<app> by copying the tarball there and extracting it
Build an upgrade
Deploy it to /tmp/<app>/releases/<up_version> by copying the tarball there (not extracting it)
Run /tmp/<app>/bin/<app> upgrade <up_version>
Rinse/repeat

michaelwa commented 7 years ago

But "Build an upgrade" is failing.

10:25 $ ./node_modules/brunch/bin/brunch b -p && MIX_ENV=prod mix do phoenix.digest, release --env=prod --upgrade 05 Mar 10:25:35 - info: compiled 6 files into 2 files, copied 3 in 3.7 sec ==> distillery Compiling 18 files (.ex)

== Compilation error on file lib/mix/lib/releases/archiver.ex == ** (CompileError) lib/mix/lib/releases/archiver.ex:152: cannot invoke local when/2 inside match, called as: {:error, {name, reason}} when :erlang.is_list(name) (stdlib) lists.erl:1354: :lists.mapfoldl/3 (stdlib) lists.erl:1355: :lists.mapfoldl/3 (stdlib) lists.erl:1354: :lists.mapfoldl/3

could not compile dependency :distillery, "mix compile" failed. You can recompile this dependency with "mix deps.compile distillery", update it with "mix deps.update distillery" or clean it with "mix deps.clean distillery"

bitwalker commented 7 years ago

@michaelwa I'm unable to reproduce that error, could you show me what version is present in your mix.lock?

bitwalker commented 7 years ago

@michaelwa I would recommend updating to the latest version anyway.

trenpixster commented 7 years ago

Facing the same issue on a clean install. I'm adding distillery to the root umbrella project Mix configuration. Forcing 1.1 fixed the issue:

➜  sample_project git:(master) ✗ # version locked at 1.2
➜  sample_project git:(master) ✗ mix deps.update distillery
Running dependency resolution...
Dependency resolution completed:
  distillery 1.2.0
* Updating distillery (Hex package)
  Checking package (https://repo.hex.pm/tarballs/distillery-1.2.0.tar)
  Fetched package
➜  sample_project git:(master) ✗ mix release.init
==> distillery
Compiling 18 files (.ex)

== Compilation error on file lib/mix/lib/releases/archiver.ex ==
** (CompileError) lib/mix/lib/releases/archiver.ex:152: cannot invoke local when/2 inside match, called as: {:error, {name, reason}} when :erlang.is_list(name)
    (stdlib) lists.erl:1354: :lists.mapfoldl/3
    (stdlib) lists.erl:1355: :lists.mapfoldl/3
    (stdlib) lists.erl:1354: :lists.mapfoldl/3

could not compile dependency :distillery, "mix compile" failed. You can recompile this dependency with "mix deps.compile distillery", update it with "mix deps.update distillery" or clean it with "mix deps.clean distillery"
➜  sample_project git:(master) ✗ # switched version to 1.1
➜  sample_project git:(master) ✗ mix deps.update distillery
Running dependency resolution...
Dependency resolution completed:
  distillery 1.1.0
* Updating distillery (Hex package)
  Checking package (https://repo.hex.pm/tarballs/distillery-1.1.0.tar)
  Using locally cached package
➜  sample_project git:(master) ✗ mix deps.compile distillery
==> distillery
Compiling 17 files (.ex)
Generated distillery app

Nothing special around the project besides it being an umbrella project. Running on Elixir v1.3.1 and Erlang 19. Hope this helps!

bitwalker commented 7 years ago

I'm definitely not able to reproduce with master, can you provide me a repo with a sample project which produces this error? My two test projects are not showing this issue.

michaelwa commented 7 years ago

Sorry was out of town just now getting back to my personal laptop.

This is what is in my mix.lock file:

%{"cowboy": {:hex, :cowboy, "1.1.2", "61ac29ea970389a88eca5a65601460162d370a70018afe6f949a29dca91f3bb0", [:rebar3], [{:cowlib, "~> 1.0.2", [hex: :cowlib, optional: false]}, {:ranch, "~> 1.3.2", [hex: :ranch, optional: false]}]}, "cowlib": {:hex, :cowlib, "1.0.2", "9d769a1d062c9c3ac753096f868ca121e2730b9a377de23dec0f7e08b1df84ee", [:make], []}, "distillery": {:git, "https://github.com/bitwalker/distillery.git", "e65eb764a1e47258d0646f9d49c2c059fc0507f3", []}, "fs": {:hex, :fs, "0.9.2", "ed17036c26c3f70ac49781ed9220a50c36775c6ca2cf8182d123b6566e49ec59", [:rebar], []}, "gettext": {:hex, :gettext, "0.13.1", "5e0daf4e7636d771c4c71ad5f3f53ba09a9ae5c250e1ab9c42ba9edccc476263", [:mix], []}, "mime": {:hex, :mime, "1.1.0", "01c1d6f4083d8aa5c7b8c246ade95139620ef8effb009edde934e0ec3b28090a", [:mix], []}, "phoenix": {:hex, :phoenix, "1.2.1", "6dc592249ab73c67575769765b66ad164ad25d83defa3492dc6ae269bd2a68ab", [:mix], [{:cowboy, "~> 1.0", [hex: :cowboy, optional: true]}, {:phoenix_pubsub, "~> 1.0", [hex: :phoenix_pubsub, optional: false]}, {:plug, "~> 1.1", [hex: :plug, optional: false]}, {:poison, "~> 1.5 or ~> 2.0", [hex: :poison, optional: false]}]}, "phoenix_html": {:hex, :phoenix_html, "2.9.3", "1b5a2122cbf743aa242f54dced8a4f1cc778b8bd304f4b4c0043a6250c58e258", [:mix], [{:plug, "~> 1.0", [hex: :plug, optional: false]}]}, "phoenix_live_reload": {:hex, :phoenix_live_reload, "1.0.8", "4333f9c74190f485a74866beff2f9304f069d53f047f5fbb0fb8d1ee4c495f73", [:mix], [{:fs, "~> 0.9.1", [hex: :fs, optional: false]}, {:phoenix, "~> 1.0 or ~> 1.2-rc", [hex: :phoenix, optional: false]}]}, "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.0.1", "c10ddf6237007c804bf2b8f3c4d5b99009b42eca3a0dfac04ea2d8001186056a", [:mix], []}, "plug": {:hex, :plug, "1.3.0", "6e2b01afc5db3fd011ca4a16efd9cb424528c157c30a44a0186bcc92c7b2e8f3", [:mix], [{:cowboy, "~> 1.0.1 or ~> 1.1", [hex: :cowboy, optional: true]}, {:mime, "~> 1.0", [hex: :mime, optional: false]}]}, "poison": {:hex, :poison, "2.2.0", "4763b69a8a77bd77d26f477d196428b741261a761257ff1cf92753a0d4d24a63", [:mix], []}, "ranch": {:hex, :ranch, "1.3.2", "e4965a144dc9fbe70e5c077c65e73c57165416a901bd02ea899cfd95aa890986", [:rebar3], []}}

I am aimed at the master in my mix.exs file:

defp deps do [{:phoenix, "~> 1.2.0"}, {:phoenix_pubsub, "~> 1.0"}, {:phoenix_html, "~> 2.6"}, {:phoenix_live_reload, "~> 1.0", only: :dev}, {:gettext, "~> 0.11"}, {:cowboy, "~> 1.0"}, {:distillery, github: "bitwalker/distillery" }, ] end

bitwalker commented 7 years ago

You need to run mix deps.unlock distillery and mix deps.update distillery in order to pick up the change. You are currently on a commit which is pre-1.2.0

michaelwa commented 7 years ago

hmmm ... same thing?!

Running erlang 19 and elixir 1.3.1

from mix.lock: "distillery": {:git, "https://github.com/bitwalker/distillery.git", "7904d141daf8198e1dc589bdf402697aa66a7987", []},

From mix.exs: #### Is this correct ?? #### {:distillery, github: "bitwalker/distillery" },

Then running mix unlock, update, and compile.

16:10 $ mix deps.unlock distillery
✔ ~/Documents/src/elixir/phoenix_distillery [master L|✚ 4] 
16:12 $ mix deps.update distillery
* Updating distillery (https://github.com/bitwalker/distillery.git)
Running dependency resolution...
✔ ~/Documents/src/elixir/phoenix_distillery [master L|✚ 4] 
16:12 $ mix deps.compile distillery
==> distillery
Compiling 18 files (.ex)

== Compilation error on file lib/mix/lib/releases/archiver.ex ==
** (CompileError) lib/mix/lib/releases/archiver.ex:152: cannot invoke local when/2 inside match, called as: {:error, {name, reason}} when :erlang.is_list(name)
    (stdlib) lists.erl:1354: :lists.mapfoldl/3
    (stdlib) lists.erl:1355: :lists.mapfoldl/3
    (stdlib) lists.erl:1354: :lists.mapfoldl/3

could not compile dependency :distillery, "mix compile" failed. You can recompile this dependency with "mix deps.compile distillery", update it with "mix deps.update distillery" or clean it with "mix deps.clean distillery"
✘-1 ~/Documents/src/elixir/phoenix_distillery [master L|✚ 4]

bitwalker commented 7 years ago

@michaelwa Can you share your project? I can't reproduce this

bradleyd commented 7 years ago

FWIW i ran into same issue. Distillery version 1.2.2. I was building a release in aktsk/docker-elixir:elixir-1.3-erlang-18.3-ubuntu-16.04

Error:

** (CompileError) lib/mix/lib/releases/archiver.ex:152: cannot invoke local when/2 inside match, called as: {:error, {name, reason}} when :erlang.is_list(name)

Stumped on what it could be, I tried using the official elixir image which is 1.4.2 and erlang 19

That seems to have fixed the issue for me.

If I had to take a guess, the issue would lie with the with else clause. Maybe the else cannot use a guard clause in elixir 1.3.x?

https://github.com/bitwalker/distillery/blob/master/lib/mix/lib/releases/archiver.ex#L147

anyway, hope that helps.

bitwalker commented 7 years ago

Ah, I wonder if that's a feature exclusive to 1.4, I'll check it out tomorrow and confirm

michaelwa commented 7 years ago

I can confirm this.

I just upgraded from 1.3.1 to 1.4.2 and it is compiling now just fine.

michaelwa commented 7 years ago

Still getting an error when running the upgrade build though:

08:34 $ ./node_modules/brunch/bin/brunch b -p && MIX_ENV=prod mix do phoenix.digest, release --env=prod --upgrade
11 Mar 08:34:07 - info: compiled 6 files into 2 files, copied 3 in 3.9 sec
==> distillery
Compiling 18 files (.ex)
Generated distillery app
==> phoenix_distillery
Check your digested files at "priv/static"
Compiling 11 files (.ex)
Generated phoenix_distillery app
==> Assembling release..
==> Building release phoenix_distillery:0.0.2 using environment prod
==> Including ERTS 8.2.2 from /usr/local/Cellar/erlang/19.2.3/lib/erlang/erts-8.2.2
==> compiler has an appup file, but it is invalid for this release,
    Backing up appfile with .bak extension and generating new one..
==> Release failed: no match of right hand side value: {:error, {:write_terms, :file, :eacces}}
    (distillery) lib/mix/lib/releases/assembler.ex:409: Mix.Releases.Assembler.generate_appups/2
    (distillery) lib/mix/lib/releases/assembler.ex:284: Mix.Releases.Assembler.generate_relup/2
    (distillery) lib/mix/lib/releases/assembler.ex:199: Mix.Releases.Assembler.create_release_info/1
    (distillery) lib/mix/lib/releases/assembler.ex:33: Mix.Releases.Assembler.assemble/1
    (distillery) lib/distillery/tasks/release.ex:106: Mix.Tasks.Release.do_release/2
    (mix) lib/mix/task.ex:294: Mix.Task.run_task/3
    (elixir) lib/enum.ex:645: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:645: Enum.each/2

The line about the appup file being backed up is interesting though. I located it and the backup and the new one are identical. Both look exactly the same:

%% -*- erlang -*-
%% %CopyrightBegin%
%%
%% Copyright Ericsson AB 2014-2016. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%%     http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%
%% %CopyrightEnd%
{"7.0.3",
 [{<<".*">>,[{restart_application, compiler}]}],
 [{<<".*">>,[{restart_application, compiler}]}]
}.

What else would need to be updated when upgrading elixir framework? It seems like that would be a valid use case that your application version 0.0.1 could be built on elixir 1.3.1 and your upgrade could be build on 1.4.2.

I did delete my 0.0.2 build folder but that still gave the same error.

thiamsantos commented 6 years ago

@bitwalker I'm experiencing the errors reported here, because I'm running upgrades from _build. I would like to know if I still can't do this on version 2.0.

bitwalker commented 6 years ago

@thiamsantos Yes, it is still not supported, and won't be - executing upgrades modifies files on disk that are required to build and run the next upgrade; it is not possible to support this, and defeats the point of building releases if you are running them from _build anyway.

thiamsantos commented 6 years ago

@bitwalker I understand. Do you recommend a workaround for this? What is the correct approach for building upgrade releases?

In our current scenario we would like to build the upgrade release in the CI server.

bitwalker commented 6 years ago

@thiamsantos Oh it's fine to build them there, running them and upgrading in _build is what presents the problem. Your CI should build the tarball and scp that to your target server, or at a minimum, extract to a different directory on the server. The rule is just that build and deploy directories need to be separated

bitwalker / distillery