Cannot connect to running app

kuon commented 6 years ago

Steps to reproduce

Build a release with mix release
Start app with ./bin/myapp start
Issue any RPC command like ./bin/myapp remote_console

Logs

Error when trying to connect to running release:

** (exit) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
    (stdlib) gen.erl:261: :gen.do_for_proc/2
    (stdlib) gen_event.erl:197: :gen_event.rpc/2
    (artificery) lib/console.ex:270: Artificery.Console.init/0
    (distillery) lib/entry.ex:42: Mix.Releases.Runtime.Control.main/1
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
    (elixir) lib/code.ex:232: Code.eval_string/3
Node cardio_data_ops@127.0.0.1 is not running!

My vm.args

## Name of the node
-name ${NODE_NAME}

## Cookie for distributed erlang
-setcookie ${COOKIE}

-smp auto

Description of issue

To ensure I did not mess up node name and cookie I did the following:

Open a terminal, export NODE_NAME, COOKIE and REPLACE_OS_VARS .
Issue ./bin/myapp start
The app is started and works as expected
Issue `./bin/myapp remote_console
The above error shows up
iex --remsh $NODE_NAME --cookie $COOKIE --name "console@127.0.0.1" works as expected and connect to the running app.

The issue was not present with 1.X and it appeared with 2.0.10. With the same app, I have the issue only on the production server (elixir 1.7.3, erlang 19.3.6.7, OS FreeBSD 11.2). On my linux dev machine when building with prod environment I don't have the issue.

bitwalker commented 6 years ago

What is $NODE_NAME in this case?

kuon commented 6 years ago

$NODE_NAME is cardio_data_ops@127.0.0.1 $REPLACE_OS_VARS is true $COOKIE is an hex string

inferiorhumanorgans commented 5 years ago

Yeah I'm getting a similar issue on FreeBSD (in this case in a jail). The app is running and functioning appropriately, if I try to run console it complains the node name is already in use, if I run something like ping or remote_console I get a not running error.

$ elixir --version
Erlang/OTP 19 [erts-8.3.5.6] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Elixir 1.7.4 (compiled with Erlang/OTP 19)

$ uname -a
FreeBSD fotos 10.4-RELEASE-p13 FreeBSD 10.4-RELEASE-p13 #0: Thu Sep 27 10:09:35 UTC 2018     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

inferiorhumanorgans commented 5 years ago

Also seeing it on FreeBSD 12.0

$ uname -a
FreeBSD app-fotos 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64

jfreeze commented 5 years ago

Was this issue solved?

ejpcmac commented 5 years ago

Hi, I’m not hitting this issue in a FreeBSD 11.2 jail with Erlang/OTP 21.1, Elixir 1.7.3 and Distillery 2.0.10 when exporting NODE_NAME before to run the commands. I’ve just tested it on an app, stopping the rc.d service before restarting it by hand, following the way to reproduce the problem.

jfreeze commented 5 years ago

I am on release 11.0-RELEASE-p9 FreeBSD. Exporting NODE_NAME had no effect on recognizing the app.

jfreeze commented 5 years ago

I upgraded to FreeBSD 11.2-RELEASE-p5 and Erlang OTP 21, and am not seeing the issue.

imranismail commented 5 years ago

This issue seems to be happening intermittently for me.

Sometimes it works and sometimes when it doesn't work after an app restart it works again.

imranismail commented 5 years ago

In both cases the app is actually running

siegfried commented 5 years ago

Having the same issue on OpenBSD 6.4.

Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false]

Elixir 1.7.3 (compiled with Erlang/OTP 19)

vanetix commented 5 years ago

I'm not sure if this is related but I was having a similar issue on distillery 2.0.12 where my post-start hook (migrations) was running prior to epmd starting, I was able to temporarily hack a sleep 1 into the script and all seems well now.

This was only happening when running in distributed mode and binding to an IP. Using a non-distributed node name - ie app@app.local had been working.

nulian commented 5 years ago

Could also be part of starting remote console overwriting the vm.args and sys.config.

Using the read_only flag doesn't help because it won't use the already generated vm.args and sys.config.

For us we where able to create a console by doing

SYS_CONFIG_PATH=/opt/app/var/sys.config VMARGS_PATH=/opt/app/var/vm.args RELEASE_READ_ONLY=1 ./bin/synapse remote_console

Change to the appropriate path to the var folder for your configuration.

easco commented 5 years ago

In the off chance it helps others…

I was having this problem. In my case I was running the application in a Docker container and on attaching to a running container could not start a remote shell using ./bin/myapp remote_shell

In this case, I was running the application (and the remote shell command) as a non-privileged user and the user account in question had no home directory. While trying to track down the issue, at one point, the system complained about something I tried because it couldn't write a .erlang.cookie file into the directory that would be the user's home directory if it existed.

I reworked my docker image so that it does create a home directory (and gives the user write permissions). When I first attach to the running container, there is no .erlang.cookie file in the user's home directory, but using ./bin/myapp remote_console will give me a remote console where it would not before. After running the remote console, I do find a .erlang.cookie file in the user's home directory.

penguincoder commented 5 years ago

Making a home directory worked for me, thanks a lot @easco 💯

vDupindev commented 5 years ago

Hi guys, I have the same problem. How to fix that bug ? What do you mean by "Making a home directory" ?

Thanks !!

lasseebert commented 5 years ago

I have the same problem. Like @imranismail this only happens some times to me. It seems to happen when the app has been running for a while (days).

I have tried @nulian's fix with no luck. I also have a home folder and a .erlang.cookie as suggested by @easco.

When I can't remote_console, only a restart helps. The app is running fine when this happens.

lasseebert commented 5 years ago

I figured out that my problem was that I was running two different applications on the same server, both deployed with distillery. They used the same EPMD port, which clashed so that the last deployed application would be present on the EPMD.

Changing the EPMD port on one of the applications solved my problem.

jfreeze commented 5 years ago

Where did you set the EPMD port at?

Sent from my iPhone

On May 8, 2019, at 4:01 AM, Lasse Skindstad Ebert notifications@github.com wrote:

I figured out that my problem was that I was running two different applications on the same server, both deployed with distillery. They used the same EPMD port, which clashed so that the last deployed application would be present on the EPMD.

Changing the EPMD port on one of the applications solved my problem.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

lasseebert commented 5 years ago

@jfreeze

In vm.args:

-env ERL_EPMD_PORT 4370

And I also needed to set it explicit when starting the application (not sure why because the distillery binary should already load vm.args):

ERL_EPMD_PORT=4370 bin/my_app foreground # Or how you start your app

bitwalker commented 5 years ago

Is there an actionable fix/documentation update that we've arrived at here? It seems like the issue is that there are missing $HOME directories, when no cookie is otherwise specified, which silently fails. My suspicion is that there is actually an incompatibility in the shell scripts that may be causing the problem, or at least not surfacing things properly. Can one of you that still has this problem export DEBUG_BOOT=true in the environment when running the various commands, and then paste that output here?

siegfried commented 5 years ago

Now I cannot reproduce it anymore. I did nothing but upgraded my OpenBSD from 6.4 to 6.5, Erlang from Erlang/OTP 19 [erts-8.3] to Erlang/OTP 21 [erts-10.2] and Elixir from 1.7.3 to 1.8.1. Was my Erlang/Elixir too old for Distillery?

egidijusz commented 5 years ago

Seeing this on FreeBSD 11.2-RELEASE-p4.

In my case this was not non-existing $HOME dir or epmd listening on same ip/port and getting confused. My fix is at the bottom of this comment.

Some info about the environments and what was working.

I could deploy and start application with edeliver (albeit the dreaded error message), but couldn't ping/connect to it (neither with edelivery, nor with ./app_name remote_console). But I could connect to it with iex --remsh ....

epmd -names was showing proper output (that is - node was alive and listening on port).

Here is specific error message I was receiving when trying remote_console to running application.

[jailedwebsite@test ~/apps/api/app_name/bin]$ ./app_name remote_console
** (exit) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started                                                                                                                  
    (stdlib) gen.erl:261: :gen.do_for_proc/2
    (stdlib) gen_event.erl:197: :gen_event.rpc/2
    (artificery) lib/console.ex:281: Artificery.Console.init/0
    (distillery) lib/entry.ex:42: Mix.Releases.Runtime.Control.main/1
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
    (elixir) lib/code.ex:232: Code.eval_string/3
Node app_name@127.0.0.1 is not running!

As I've mentioned before, doing iex --remsh app_name@127.0.0.1 --name debug@127.0.0.1 --cookie my_cookie (right after trying remote_console for example) worked just fine. Also I was able to connect to it from separate iex shell (without --remsh), with Node.connect(:"app_name@127.0.0.1").

Exact versions of applications in error message: {artificery,"0.4.2","./lib/artificery-0.4.2"}, {distillery,"2.0.14","./lib/distillery-2.0.14"} Elixir: 1.7.4 Erlang/OTP: 19.3.6.12_1

How I fixed it. Upgrading this particular application's runtime jail did not help, but ugprading build jail fixed the issue.

What exactly got upgraded: elixir 1.7.4 -> 1.8.2 erlang/otp 19.3.6.12_1,4 -> 21.3.8.3,4 (I suppose this "_1,4" is some unusual FreeBSD pkg naming strategy)

Sidenote I have many FreeBSD jails set up - one for building, others for running elixir apps. There are multiple elixir apps running on this server fine (built with previous distillery versions, ranging from 0.8.0 to 1.x).

egidijusz commented 5 years ago

Can one of you that still has this problem export DEBUG_BOOT=true in the environment when running the various commands, and then paste that output here?

I did that and will send the output to your email (from your github profile), if you don't mind.

siegfried commented 5 years ago

I'm able to reproduce this again. I wonder if it is because I'm not running it as root. @bitwalker

$ DEBUG_BOOT=true ./j1605_mqtt/bin/j1605_mqtt remote_console
tput: not enough arguments (3) for capability `setaf'
+1564105131 export TERM=xterm-256color
+1564105131 TERM=xterm-256color
+1564105131 '[' xterm-256color = dumb ']'
+1564105131 exec /home/j1605/j1605_mqtt/releases/0.1.2/j1605_mqtt.sh remote_console
+1564105131 unset CDPATH
+1564105131 ERL_OPTS=
+1564105131 RUN_ERL_ENV=
+1564105131 DISTILLERY_VSN=2.1.1
+1564105131 export ERL_OPTS
+1564105131 export RUN_ERL_ENV
+1564105131 export DISTILLERY_VSN
+1564105131 uname
+1564105131 grep -q Darwin
++1564105131 readlink -f /home/j1605/j1605_mqtt/releases/0.1.2/j1605_mqtt.sh
+1564105131 SCRIPT=/home/j1605/j1605_mqtt/releases/0.1.2/j1605_mqtt.sh
+1564105131 export ESCRIPT_NAME
+1564105131 ESCRIPT_NAME=/home/j1605/j1605_mqtt/releases/0.1.2/j1605_mqtt.sh
++1564105131 dirname /home/j1605/j1605_mqtt/releases/0.1.2/j1605_mqtt.sh
+1564105131 SCRIPT_DIR=/home/j1605/j1605_mqtt/releases/0.1.2
+1564105131 RELEASE_ROOT_DIR=/home/j1605/j1605_mqtt
+1564105131 REL_DIR=/home/j1605/j1605_mqtt/releases/0.1.2
+1564105131 export REL_DIR
+1564105131 RELEASE_LIBEXEC_DIR=/home/j1605/j1605_mqtt/releases/0.1.2/libexec
+1564105131 CONSOLIDATED_DIR=/home/j1605/j1605_mqtt/lib/j1605_mqtt-0.1.2/consolidated
+1564105131 export CONSOLIDATED_DIR
+1564105131 START_ERL_DATA=/home/j1605/j1605_mqtt/var/start_erl.data
+1564105131 export START_ERL_DATA
+1564105131 . /home/j1605/j1605_mqtt/releases/0.1.2/libexec/readlink.sh
++1564105131 set -e
++1564105131 '[' '!' -z true ']'
++1564105131 set +x
+1564105131 . /home/j1605/j1605_mqtt/releases/0.1.2/libexec/logger.sh
++1564105131 set -e
++1564105131 '[' '!' -z true ']'
++1564105131 set +x
tput: not enough arguments (3) for capability `setaf'
tput: not enough arguments (3) for capability `setaf'
tput: not enough arguments (3) for capability `setaf'
+1564105131 . /home/j1605/j1605_mqtt/releases/0.1.2/libexec/erts.sh
++1564105131 set -e
++1564105131 code_paths=()
+++1564105131 erl -noshell -eval 'io:format("~s~n~s~n", [code:root_dir(), erlang:system_info(version)]).' -s erlang halt
++++1564105131 whereis_erts_bin
++++1564105131 '[' -z 10.2 ']'
++++1564105131 '[' -z '' ']'
++++1564105131 __erts_dir=/home/j1605/j1605_mqtt/erts-10.2
++++1564105131 '[' -d /home/j1605/j1605_mqtt/erts-10.2 ']'
++++1564105131 echo /home/j1605/j1605_mqtt/erts-10.2/bin
+++1564105131 __bin=/home/j1605/j1605_mqtt/erts-10.2/bin
+++1564105131 '[' -z /home/j1605/j1605_mqtt/erts-10.2/bin ']'
+++1564105131 __erl=/home/j1605/j1605_mqtt/erts-10.2/bin/erl
+++1564105131 __boot_provided=0
+++1564105131 echo -noshell -eval 'io:format("~s~n~s~n", [code:root_dir(), erlang:system_info(version)]).' -s erlang halt
+++1564105131 grep '\-boot '
+++1564105131 __erts_included=0
+++1564105131 [[ /home/j1605/j1605_mqtt/erts-10.2/bin/erl =~ ^/home/j1605/j1605_mqtt ]]
+++1564105131 __erts_included=1
+++1564105131 '[' 1 -eq 1 ']'
+++1564105131 '[' 0 -eq 1 ']'
+++1564105131 '[' 1 -eq 1 ']'
+++1564105131 /home/j1605/j1605_mqtt/erts-10.2/bin/erl -boot_var ERTS_LIB_DIR /home/j1605/j1605_mqtt/lib -boot /home/j1605/j1605_mqtt/bin/start_clean -pa /home/j1605/j1605_mqtt/lib/j1605_mqtt-0.1.2/consolidated -noshell -eval 'io:format("~s~n~s~n", [code:root_dir(), erlang:system_info(version)]).' -s erlang halt
warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell)
++1564105131 __info='/home/j1605/j1605_mqtt
10.2'
++1564105131 export ROOTDIR
+++1564105131 echo '/home/j1605/j1605_mqtt
10.2'
+++1564105131 head -n1
++1564105131 ROOTDIR=/home/j1605/j1605_mqtt
++1564105131 export ERTS_VSN
++1564105131 '[' -z 10.2 ']'
+++1564105131 echo '/home/j1605/j1605_mqtt
10.2'
+++1564105131 tail -n1
++1564105131 ERTS_VSN=10.2
++1564105131 export ERTS_DIR
++1564105131 ERTS_DIR=/home/j1605/j1605_mqtt/erts-10.2
++1564105131 export BINDIR
++1564105131 BINDIR=/home/j1605/j1605_mqtt/erts-10.2/bin
++1564105131 export ERTS_LIB_DIR
+++1564105131 readlink_f /home/j1605/j1605_mqtt/erts-10.2/../lib
+++1564105131 uname
+++1564105131 grep -q Darwin
+++1564105131 readlink -f /home/j1605/j1605_mqtt/erts-10.2/../lib
++1564105131 ERTS_LIB_DIR=/home/j1605/j1605_mqtt/lib
++1564105131 export EMU=beam
++1564105131 EMU=beam
++1564105131 export PROGNAME=erl
++1564105131 PROGNAME=erl
++1564105131 __set_code_paths
++1564105131 '[' 0 -eq 0 ']'
++1564105131 code_paths=()
+++1564105131 __rel_apps
+++1564105131 __releases=/home/j1605/j1605_mqtt/releases/RELEASES
+++1564105131 __vsn=0.1.2
++++1564105131 sed -E -n '/\{release,[^,]*,"0.1.2"/,/[^po]*(permanent|old)}/p' /home/j1605/j1605_mqtt/releases/RELEASES
+++1564105131 __rel='[{release,"j1605_mqtt","0.1.2","10.2",
          [{elixir,"1.8.1","./lib/elixir-1.8.1"},
           {mix,"1.8.1","./lib/mix-1.8.1"},
           {iex,"1.8.1","./lib/iex-1.8.1"},
           {sasl,"3.3","./lib/sasl-3.3"},
           {compiler,"7.3","./lib/compiler-7.3"},
           {j1605_mqtt,"0.1.2","./lib/j1605_mqtt-0.1.2"},
           {logger,"1.8.1","./lib/logger-1.8.1"},
           {tortoise,"0.9.4","./lib/tortoise-0.9.4"},
           {ssl,"9.1","./lib/ssl-9.1"},
           {public_key,"1.6.4","./lib/public_key-1.6.4"},
           {asn1,"5.0.8","./lib/asn1-5.0.8"},
           {gen_state_machine,"2.0.5","./lib/gen_state_machine-2.0.5"},
           {crypto,"4.4","./lib/crypto-4.4"},
           {stdlib,"3.7","./lib/stdlib-3.7"},
           {runtime_tools,"1.13.1","./lib/runtime_tools-1.13.1"},
           {j1605,"0.3.0","./lib/j1605-0.3.0"},
           {distillery,"2.1.1","./lib/distillery-2.1.1"},
           {artificery,"0.4.2","./lib/artificery-0.4.2"},
           {toml,"0.5.2","./lib/toml-0.5.2"},
           {kernel,"6.2","./lib/kernel-6.2"}],
          permanent}].'
+++1564105131 echo '[{release,"j1605_mqtt","0.1.2","10.2",
+++1564105131 grep -E '[{][A-Za-z_0-9]*,\"[0-9.]*[A-Za-z0-9.\_\+\-]*\"'
          [{elixir,"1.8.1","./lib/elixir-1.8.1"},
           {mix,"1.8.1","./lib/mix-1.8.1"},
+++1564105131 tail -n +2
           {iex,"1.8.1","./lib/iex-1.8.1"},
           {sasl,"3.3","./lib/sasl-3.3"},
           {compiler,"7.3","./lib/compiler-7.3"},
           {j1605_mqtt,"0.1.2","./lib/j1605_mqtt-0.1.2"},
           {logger,"1.8.1","./lib/logger-1.8.1"},
           {tortoise,"0.9.4","./lib/tortoise-0.9.4"},
           {ssl,"9.1","./lib/ssl-9.1"},
+++1564105131 sed '-es/"[^"]*$//' '-es/^[^a-z]*//' -es/,/-/ '-es/"//' '-es/","[^"]*$//'
           {public_key,"1.6.4","./lib/public_key-1.6.4"},
           {asn1,"5.0.8","./lib/asn1-5.0.8"},
           {gen_state_machine,"2.0.5","./lib/gen_state_machine-2.0.5"},
           {crypto,"4.4","./lib/crypto-4.4"},
           {stdlib,"3.7","./lib/stdlib-3.7"},
           {runtime_tools,"1.13.1","./lib/runtime_tools-1.13.1"},
           {j1605,"0.3.0","./lib/j1605-0.3.0"},
           {distillery,"2.1.1","./lib/distillery-2.1.1"},
           {artificery,"0.4.2","./lib/artificery-0.4.2"},
           {toml,"0.5.2","./lib/toml-0.5.2"},
           {kernel,"6.2","./lib/kernel-6.2"}],
          permanent}].'
++1564105131 apps='elixir-1.8.1
mix-1.8.1
iex-1.8.1
sasl-3.3
compiler-7.3
j1605_mqtt-0.1.2
logger-1.8.1
tortoise-0.9.4
ssl-9.1
public_key-1.6.4
asn1-5.0.8
gen_state_machine-2.0.5
crypto-4.4
stdlib-3.7
runtime_tools-1.13.1
j1605-0.3.0
distillery-2.1.1
artificery-0.4.2
toml-0.5.2
kernel-6.2'
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/elixir-1.8.1 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/mix-1.8.1 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/iex-1.8.1 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/sasl-3.3 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/compiler-7.3 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/j1605_mqtt-0.1.2 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/logger-1.8.1 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/tortoise-0.9.4 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/ssl-9.1 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/public_key-1.6.4 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/asn1-5.0.8 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/gen_state_machine-2.0.5 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/crypto-4.4 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/stdlib-3.7 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/runtime_tools-1.13.1 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/j1605-0.3.0 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/distillery-2.1.1 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/artificery-0.4.2 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/toml-0.5.2 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
++1564105131 for app in $apps
++1564105131 '[' -d /home/j1605/j1605_mqtt/lib/kernel-6.2 ']'
++1564105131 code_paths+=(-pa "${ERTS_LIB_DIR}/$app/ebin")
+1564105131 . /home/j1605/j1605_mqtt/releases/0.1.2/libexec/helpers.sh
++1564105131 set -e
+1564105131 . /home/j1605/j1605_mqtt/releases/0.1.2/libexec/env.sh
++1564105131 set -e
++1564105131 RUN_ERL_DISABLE_FLOWCNTRL=true
++1564105131 CODE_LOADING_MODE=embedded
++1564105131 REL_LIB_DIR=/home/j1605/j1605_mqtt/lib
++1564105131 RELEASE_MUTABLE_DIR=/home/j1605/j1605_mqtt/var
++1564105131 RUNNER_LOG_DIR=/home/j1605/j1605_mqtt/var/log
++1564105131 EXTRA_OPTS=
++1564105131 reset_hooks
++1564105131 export PRE_CONFIGURE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_configure.d
++1564105131 PRE_CONFIGURE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_configure.d
++1564105131 export POST_CONFIGURE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_configure.d
++1564105131 POST_CONFIGURE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_configure.d
++1564105131 export PRE_START_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_start.d
++1564105131 PRE_START_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_start.d
++1564105131 export POST_START_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_start.d
++1564105131 POST_START_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_start.d
++1564105131 export PRE_STOP_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_stop.d
++1564105131 PRE_STOP_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_stop.d
++1564105131 export POST_STOP_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_stop.d
++1564105131 POST_STOP_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_stop.d
++1564105131 export PRE_UPGRADE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_upgrade.d
++1564105131 PRE_UPGRADE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/pre_upgrade.d
++1564105131 export POST_UPGRADE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_upgrade.d
++1564105131 POST_UPGRADE_HOOKS=/home/j1605/j1605_mqtt/releases/0.1.2/hooks/post_upgrade.d
++1564105131 export BINDIR=/home/j1605/j1605_mqtt/erts-10.2/bin
++1564105131 BINDIR=/home/j1605/j1605_mqtt/erts-10.2/bin
++1564105131 export LD_LIBRARY_PATH=/home/j1605/j1605_mqtt/erts-10.2/lib:
++1564105131 LD_LIBRARY_PATH=/home/j1605/j1605_mqtt/erts-10.2/lib:
++1564105131 export EMU=beam
++1564105131 EMU=beam
++1564105131 export PROGNAME=erl
++1564105131 PROGNAME=erl
++1564105131 export RELEASE_CONFIG_DIR=/home/j1605/j1605_mqtt
++1564105131 RELEASE_CONFIG_DIR=/home/j1605/j1605_mqtt
++1564105131 '[' -z '' ']'
++1564105131 '[' '!' -d /home/j1605/j1605_mqtt/var ']'
++1564105131 '[' -z '' ']'
++1564105131 '[' '!' -d /home/j1605/j1605_mqtt/var/log ']'
+1564105131 . /home/j1605/j1605_mqtt/releases/0.1.2/libexec/config.sh
++1564105131 set -e
+1564105131 cd /home/j1605/j1605_mqtt
+1564105131 export DISTILLERY_TASK=remote_console
+1564105131 DISTILLERY_TASK=remote_console
+1564105131 COMMAND_DIR=/home/j1605/j1605_mqtt/releases/0.1.2/libexec/commands
+1564105131 COMMAND_NAME=remote_console
+1564105131 COMMAND_PATH=/home/j1605/j1605_mqtt/releases/0.1.2/libexec/commands/remote_console.sh
+1564105131 '[' 1 -ne 0 ']'
+1564105131 shift
+1564105131 case $COMMAND_NAME in
+1564105131 '[' -f /home/j1605/j1605_mqtt/releases/0.1.2/libexec/commands/remote_console.sh ']'
+1564105131 partial_configure_release
+1564105131 '[' -z '' ']'
+1564105131 '[' -f /home/j1605/j1605_mqtt/var/vm.args ']'
+1564105131 export VMARGS_PATH=/home/j1605/j1605_mqtt/var/vm.args
+1564105131 VMARGS_PATH=/home/j1605/j1605_mqtt/var/vm.args
+1564105131 '[' -z '' ']'
+1564105131 '[' -f /home/j1605/j1605_mqtt/var/sys.config ']'
+1564105131 export SYS_CONFIG_PATH=/home/j1605/j1605_mqtt/var/sys.config
+1564105131 SYS_CONFIG_PATH=/home/j1605/j1605_mqtt/var/sys.config
+1564105131 '[' -z /home/j1605/j1605_mqtt/var/vm.args ']'
+1564105131 '[' -z /home/j1605/j1605_mqtt/var/sys.config ']'
+1564105131 _configure_node
+1564105131 '[' '!' -z '' ']'
++1564105131 get_vmargs_path
++1564105131 '[' -z /home/j1605/j1605_mqtt/var/vm.args ']'
++1564105131 echo /home/j1605/j1605_mqtt/var/vm.args
++1564105131 echo ''
+1564105131 vmargs=/home/j1605/j1605_mqtt/var/vm.args
+1564105131 '[' -z /home/j1605/j1605_mqtt/var/vm.args ']'
+1564105131 export NAME_ARG
++1564105131 _replace_os_vars_str /home/j1605/j1605_mqtt/var/vm.args
++1564105131 grep '^-s\{0,1\}name'
++1564105131 awk '
        function escape(s) {
            gsub(/&/, "\\\\&", s);
            return s;
        }
        {
            while(match($0,"[$]{[^}]*}")) {
                var=substr($0,RSTART+2,RLENGTH-3);
                gsub("[$]{"var"}", escape(ENVIRON[var]))
            }
    }1'
+1564105131 NAME_ARG='-name j1605_mqtt@127.0.0.1'
+1564105131 '[' -z '-name j1605_mqtt@127.0.0.1' ']'
+1564105131 export NAME_TYPE
++1564105131 echo '-name j1605_mqtt@127.0.0.1'
++1564105131 awk '{print $1}'
++1564105131 tail -n 1
+1564105131 NAME_TYPE=-name
+1564105131 export NAME
++1564105131 echo '-name j1605_mqtt@127.0.0.1'
++1564105131 tail -n 1
++1564105131 awk '{print $2}'
+1564105131 NAME=j1605_mqtt@127.0.0.1
+1564105131 '[' -z j1605_mqtt@127.0.0.1 ']'
+1564105131 case $NAME in
+1564105131 [[ j1605_mqtt@127.0.0.1 =~ ^[^@]+@[^.]+\..*$ ]]
+1564105131 '[' -name = -sname ']'
+1564105131 . /home/j1605/j1605_mqtt/releases/0.1.2/libexec/commands/remote_console.sh
++1564105131 set -e
++1564105131 require_cookie
++1564105131 _load_cookie
++1564105131 '[' '!' -z '' ']'
+++1564105131 get_vmargs_path
+++1564105131 '[' -z /home/j1605/j1605_mqtt/var/vm.args ']'
+++1564105131 echo /home/j1605/j1605_mqtt/var/vm.args
+++1564105131 echo ''
++1564105131 vmargs=/home/j1605/j1605_mqtt/var/vm.args
++1564105131 '[' -z /home/j1605/j1605_mqtt/var/vm.args ']'
+++1564105131 _replace_os_vars_str /home/j1605/j1605_mqtt/var/vm.args
+++1564105131 grep '^-setcookie'
+++1564105131 awk '
        function escape(s) {
            gsub(/&/, "\\\\&", s);
            return s;
        }
        {
            while(match($0,"[$]{[^}]*}")) {
                var=substr($0,RSTART+2,RLENGTH-3);
                gsub("[$]{"var"}", escape(ENVIRON[var]))
            }
    }1'
++1564105131 COOKIE_ARG='-setcookie rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
++1564105131 DEFAULT_COOKIE_FILE=/home/j1605/.erlang.cookie
++1564105131 '[' '!' -z '-setcookie rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' ']'
+++1564105131 echo '-setcookie rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
+++1564105131 awk '{ print $2 }'
++1564105131 COOKIE='rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
++1564105131 '[' -z 'rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' ']'
++1564105131 require_live_node
++1564105131 require_cookie
++1564105131 _load_cookie
++1564105131 '[' '!' -z 'rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' ']'
++1564105131 return 0
++1564105131 '[' -z 'rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' ']'
++1564105131 release_ctl ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
++1564105131 command=ping
++1564105131 shift
++1564105131 elixir -e Distillery.Releases.Runtime.Control.main --logger-sasl-reports false -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
++1564105131 '[' 8 -eq 0 ']'
++1564105131 '[' -e = --help ']'
++1564105131 '[' -e = -h ']'
++1564105131 MODE=elixir
++1564105131 ERL=
++1564105131 I=1
++1564105131 E=0
++1564105131 LENGTH=8
++1564105131 set -- -e Distillery.Releases.Runtime.Control.main --logger-sasl-reports false -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' -extra
++1564105131 '[' 1 -le 8 ']'
++1564105131 S=1
++1564105131 case "$1" in
++1564105131 S=2
++1564105131 set -- -e Distillery.Releases.Runtime.Control.main --logger-sasl-reports false -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' -extra -e Distillery.Releases.Runtime.Control.main
++1564105131 I=3
++1564105131 shift 2
++1564105131 '[' 3 -le 8 ']'
++1564105131 S=1
++1564105131 case "$1" in
++1564105131 S=2
++1564105131 '[' false = true ']'
++1564105131 '[' false = false ']'
++1564105131 ERL=' -logger handle_sasl_reports false'
++1564105131 I=5
++1564105131 shift 2
++1564105131 '[' 5 -le 8 ']'
++1564105131 S=1
++1564105131 case "$1" in
++1564105131 '[' 5 -le 8 ']'
++1564105131 I=6
++1564105131 set -- -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' -extra -e Distillery.Releases.Runtime.Control.main --
++1564105131 shift
++1564105131 '[' 6 -le 8 ']'
++1564105131 I=7
++1564105131 set -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' -extra -e Distillery.Releases.Runtime.Control.main -- ping
++1564105131 shift
++1564105131 '[' 7 -le 8 ']'
++1564105131 I=8
++1564105131 set -- --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' -extra -e Distillery.Releases.Runtime.Control.main -- ping --name=j1605_mqtt@127.0.0.1
++1564105131 shift
++1564105131 '[' 8 -le 8 ']'
++1564105131 I=9
++1564105131 set -- '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4' -extra -e Distillery.Releases.Runtime.Control.main -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
++1564105131 shift
++1564105131 '[' 9 -le 8 ']'
++1564105131 break
++1564105131 I=-1
++1564105131 '[' -1 -ge 0 ']'
++1564105131 '[' elixir '!=' iex ']'
++1564105131 ERL='-noshell -s elixir start_cli  -logger handle_sasl_reports false'
++1564105131 erl -noshell -s elixir start_cli -logger handle_sasl_reports false -extra -e Distillery.Releases.Runtime.Control.main -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
+++1564105131 whereis_erts_bin
+++1564105131 '[' -z 10.2 ']'
+++1564105131 '[' -z '' ']'
+++1564105131 __erts_dir=/home/j1605/j1605_mqtt/erts-10.2
+++1564105131 '[' -d /home/j1605/j1605_mqtt/erts-10.2 ']'
+++1564105131 echo /home/j1605/j1605_mqtt/erts-10.2/bin
++1564105131 __bin=/home/j1605/j1605_mqtt/erts-10.2/bin
++1564105131 '[' -z /home/j1605/j1605_mqtt/erts-10.2/bin ']'
++1564105131 __erl=/home/j1605/j1605_mqtt/erts-10.2/bin/erl
++1564105131 __boot_provided=0
++1564105131 echo -noshell -s elixir start_cli -logger handle_sasl_reports false -extra -e Distillery.Releases.Runtime.Control.main -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
++1564105131 grep '\-boot '
++1564105131 __erts_included=0
++1564105131 [[ /home/j1605/j1605_mqtt/erts-10.2/bin/erl =~ ^/home/j1605/j1605_mqtt ]]
++1564105131 __erts_included=1
++1564105131 '[' 1 -eq 1 ']'
++1564105131 '[' 0 -eq 1 ']'
++1564105131 '[' 1 -eq 1 ']'
++1564105131 /home/j1605/j1605_mqtt/erts-10.2/bin/erl -boot_var ERTS_LIB_DIR /home/j1605/j1605_mqtt/lib -boot /home/j1605/j1605_mqtt/bin/start_clean -config /home/j1605/j1605_mqtt/var/sys.config -pa /home/j1605/j1605_mqtt/lib/j1605_mqtt-0.1.2/consolidated -noshell -s elixir start_cli -logger handle_sasl_reports false -extra -e Distillery.Releases.Runtime.Control.main -- ping --name=j1605_mqtt@127.0.0.1 '--cookie=rk_>3z[%3/gVfy43x=5&pqkFDhPB>H3y3bJ>K[&Gc`$TMypQ7vAV21vta[OJvov4'
warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell)
++1564105132 fail 'Node j1605_mqtt@127.0.0.1 is not running!'
++1564105132 printf '\e[31m%s\n' 'Node j1605_mqtt@127.0.0.1 is not running!'
Node j1605_mqtt@127.0.0.1 is not running!
++1564105132 exit 1

bitwalker / distillery