bitwalker / distillery

Simplify deployments in Elixir with OTP releases!
MIT License
2.96k stars 396 forks source link

remote_console is broken under elixir 1.9 #692

Closed bryanhuntesl closed 5 years ago

bryanhuntesl commented 5 years ago

Steps to reproduce

Under k8s/Docker this works :

iex --erl "-hidden -kernel" \ --name "elixir_plug_poc111@elixir-plug-poc-0.elixir-plug-poc.default.svc.cluster.local" \ --cookie "cookie" \ --remsh "elixir_plug_poc@elixir-plug-poc-0.elixir-plug-poc.default.svc.cluster.local"

But this fails with error :

/opt/app # ./bin/elixir_plug_poc remote_console 
Node elixir_plug_poc@elixir-plug-poc-0.elixir-plug-poc.default.svc.cluster.local is not running!

I can see from instrumenting releases/0.0.1/libexec/commands/remote_console.sh that it isn't getting past the require_live_node check.

#!/usr/bin/env bash

## Connect a remote shell to a running node

set -e

require_cookie
echo "got to cookie"
require_live_node

echo "got past live node"

I also get the error "--logger-sasl-reports : Unknown option" reported in #687


Here follows two examples you can try for yourself - one built using 'bitwalker/alpine-elixir:1.8.2' the other using 'bitwalker/alpine-elixir:1.9' - the build with 1.8.2 works correctly, with 1.9.0 it exibits the failure of remote_console and --logger-sasl-reports : Unknown option message when running ping (which succeeds).

docker run --rm  --hostname foo.local --name=elixir_plug_poc_1.9.0 -e "REPLACE_OS_VARS=true" -e "ERLANG_COOKIE=cookie" -ti binarytemple/elixir_plug
_poc:1.9.0 foreground                                                                                                                                            

14:47:50.022 [debug] Tzdata polling for update.                                                                                                                  

14:47:50.461 [debug] Tzdata polling shows the loaded tz database is up to date.               
/code/binarytemple/elixir_plug_poc/ docker exec -ti elixir_plug_poc_1.9.0 /bin/bash
bash-5.0# 
bash-5.0# bin
bin/  bind  
bash-5.0# bin/elixir_plug_poc
elixir_plug_poc             elixir_plug_poc.bat         elixir_plug_poc_rc_exec.sh  
bash-5.0# bin/elixir_plug_poc remote_console
--logger-sasl-reports : Unknown option
Node elixir_plug_poc@66379c4306da is not running!
bash-5.0# bin/elixir_plug_poc ping
pong
--logger-sasl-reports : Unknown option

Then using image built from bitwalker/alpine-elixir:1.8.2

docker run --rm --hostname foo.local --name=elixir_plug_poc_1.8.2 -e "REPLACE_OS_VARS=true" -e "ERLANG_COOKIE=cookie" -ti bin
arytemple/elixir_plug_poc:latest foreground                                     

14:51:47.109 [debug] Tzdata polling for update.

14:51:47.840 [debug] Tzdata polling shows the loaded tz database is up to date.

Remote console works

docker exec -ti elixir_plug_poc_1.8.2 /bin/bash
bash-4.4# hostname -f
foo.local
bash-4.4# bin/elixir_plug_poc ping
pong
bash-4.4# bin/elixir_plug_poc remote_console
Erlang/OTP 22 [erts-10.4] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [hipe]

Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(elixir_plug_poc@foo.local)1> *** ERROR: Shell process terminated! (^G to start new job) ***
bitwalker commented 5 years ago

Sorry, thought this was related to the other issue, can you try and reproduce with 2.1.1?

rupurt commented 5 years ago

@bitwalker I can also confirm that this is still an issue with 2.1.1

bryanhuntesl commented 5 years ago

Tested with 2.1.1 - works -

docker run --rm --hostname foo.local --name=elixir_plug_poc -e "REPLACE_OS_VARS=true" -e "ERLANG_COOKIE=cookie" -ti binarytemple/elixir_plug_poc:dist-2.1.1-elixir-1.9.0
docker exec -ti elixir_plug_poc /bin/bash 
bash-5.0# 
bash-5.0# bin/elixir_plug_poc ping
pong
bash-5.0# bin/elixir_plug_poc remote_console
Erlang/OTP 22 [erts-10.4.3] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [hipe]

Interactive Elixir (1.9.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(elixir_plug_poc@foo.local)1> 

Thanks @bitwalker !

rupurt commented 5 years ago

I fixed my problem. I was getting an error with tzdata because the format changed. Bumping the version to include https://github.com/bitwalker/timex/issues/542 now allows me to connect via remote_console.

However I think there may be an underlying problem here that prevented remote_console from working. The beam process was still running after tzdata threw the error on startup. I would like to be able to connect for debugging purposes, should I be able to? or is it working as expected and not allowing me to connect?