bitwalker / distillery

Simplify deployments in Elixir with OTP releases!
MIT License
2.96k stars 398 forks source link

Release with custom args in vm.args -name not starting after upgrade to 2.10 #562

Closed lukaszsamson closed 5 years ago

lukaszsamson commented 5 years ago

Steps to reproduce

In vm.args -name ${NODE_NAME}@${NODE_IP} NODE_NAME and NODE_IP are in process env REPLACE_OS_VARS is set to true

release fails to start with Failed setting -name! The hostname in '@' is not fully qualified

Works fine in distillery <= 2.9

mmrobins commented 5 years ago

I got this error as well in a kubernetes job where I had forgotten to set these environment variables in my yaml. I had set them in the yaml for my deployment

          env:
            -
              name: NODE_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP

But had forgotten to do that in the job yaml. Before distillery 2.10 it didn't complain about that, but it seems to now, which seems like better behavior to be honest.

Hopefully that's helpful for you or someone else for debugging this kind of error.

lukaszsamson commented 5 years ago

@mmrobins I'm not using kubernates. I'm using the official approach to dynamic vm.args as described in https://github.com/bitwalker/distillery/blob/master/docs/config/vm.args.md#custom-args. I cannot hardcode hostname as it is not known at build time.

The defect become visible in https://github.com/bitwalker/distillery/commit/c83332804bce88089f627d4753bfcfb2c31f2504 but IMO the real error is in https://github.com/bitwalker/distillery/blob/c83332804bce88089f627d4753bfcfb2c31f2504/priv/libexec/config.sh#L159 where NAME_ARG is read by grep directly from vm.args with no env nor template substitution. This invalid value is then validated in following lines resulting in the error. In my case this results in

NODE_ARG='-name ${NODE_NAME}@${NODE_IP}'
NODE_TYPE='-name'
NAME='${NODE_NAME}@${NODE_IP}'

Which obviously does not match ^[^@]+@[^\.]+\..*$

bitwalker commented 5 years ago

@lukaszsamson Your evaluation of what happens with NAME_ARG is incorrect; by the time that function is called, VMARGS_PATH has already been set to a substituted version of the contents of that file.

In any case, this code has changed anyway, could you report back on whether this is still an issue in master?

bitwalker commented 5 years ago

The other thing I forgot to mention is that the reason no substitutions happened in your example is because neither NODE_NAME or NODE_IP are exported in the environment (in such cases the replacements are left alone, which I believe is what @mmrobins was trying to say.

lukaszsamson commented 5 years ago

@bitwalker perhaps, I was just a guess. Anyway, NODE_NAME and NODE_IP are definitely exported and work fine on v2.9. I also did a quick test on master and managed to start the release. It looks like https://github.com/bitwalker/distillery/commit/3bfd7a0cd7d2be69318869bc72d1be699f211995 fixed this regression. I'll do some more tests and get back.

lukaszsamson commented 5 years ago

@bitwalker after more testing I have to say it is still broken. Now the release starts fine but I'm not able to run migrations.

export NODE_NAME="$APP_NAME"
export NODE_IP
NODE_IP=$(/sbin/ip route|awk '/scope/ { print $9 }')

echo "Node: ${NODE_NAME}@${NODE_IP}"
# prints Node: my_app@1.2.3.4

# this works
exec bin/start foreground

# this crashes

bin/start eval 'Elixir.MyApp.Tasks.Release.migrate()'
# Unable to load vm.args and NAME is not exported, unable to configure node!
bitwalker commented 5 years ago

This should no longer be an issue, but feel free to reopen if you are still seeing this with the latest releases