proposal: custom default `exec` command for tasks

[Thunderbottom]

Proposal

It would be cool to have a custom exec command predefined in the job file itself, under the task specification. This would allow users to run the said command immediately on exec either on the web UI or the CLI instead of storing and using the previously known exec command, i.e. /bin/bash or /bin/sh.

For the web UI, this command can be pre-rendered and the user just needs to execute it. For the CLI and default use case, we can set the alloc exec trigger to execute the default command, so accessing the task would automatically run the said command.

Use-cases

Sometimes exec'ing into a job requires a task-specific command, which will be useful in cases where you need to quickly exec into a job for debugging and other purposes. Currently, there's no such default exec command that can be configured and run by the user, and the user requires to manually type down the command, which is cumbersome with the web UI where you repeatedly need to type in the same command every time. This is particularly useful for cases where you need to drop into, for example, a redis-cli shell that connects to the server so that queries can be run directly without having to drop into the shell first and run the command manually. If the user needs to drop into a bash or a POSIX shell, they can override the default command by appending the said command, which currently is the default behavior.

While I know this can easily be scripted for the CLI, some people also prefer using the web UI for accessing a task, and having a default exec command will certainly reduce the effort required to remember/copy-paste the said command.

Attempted Solutions

As an example, the following task block will have the default exec command:

task "app" {
  driver = "docker"

  config {
    image   = "redis:7"
    command = "redis-server"
    nomad {
      default_exec_command = ["redis-cli", "-c", "-h", "$JOB_HOST"]
    }  
  }
}

and running nomad alloc exec <ID> would by default run the above command. I'm open to suggestions on how this can be improved further, or if there's any way to achieve this currently (I apologize!), please do let me know.

[jrasell]

Hi @Thunderbottom and thanks for raising this feature request.

Nomad recently added task actions which allows you to define custom commands that can be execute on running allocations when necessary. In this situation defined above, you could therefore configure custom actions to run redis CLI commands are you desire. I don't believe this gets us exactly what you want though, but I am hesitant to add another field to the jobspec specifically to control alloc exec behaviour. If you have thoughts on the tradeoffs and using the actions block, please let me know.

Whilst looking into this I did find that the actions block could benefit from supporting interpolation which would allow actions to interpolate network variables for example. I'll raise this as a separate feature request.

[Thunderbottom]

Hi, thanks for the link to the actions block! That's certainly helpful. But from what I can see, this is for running one-off commands on the task and it does not drop you into a shell for further access/debugging. For our use case, we would like to have a command specifically for alloc exec, so that it's easier/quicker to drop right into an interactive session and run things from there.

As an example, there are use cases where one needs to connect to a database, which currently would require alloc exec into a shell like /bin/sh and then running a possibly long command to get an interactive session for, let's say, psql or redis-cli. The workaround right now is to mount scripts to the job that do this for you, and then run that script when you get on the shell, but we would like to avoid that totally if that's possible. I hope that makes sense.

Again, this does not have to be an additional field in the jobspec and can be accomodated in the existing actions block itself. I'm up for further discussion on the possibility of implementing this and would gladly help in any way possible.

[tgross]

I'm going to post something from @philrenaud's internal design doc on Actions describing why this wasn't implemented originally:

Interactive actions The only commands we want to support with Task Actions are “one-shot”; in other words, if there is anything like a “Are you sure?” confirmation to a command, it is not a good candidate for a Task Action. The reasoning behind this is similar to why we’d avoid passing arguments: the management operator should feel confident that all possible results of a user-run action can be known at definition time, and this leaves open the possibility of an operator performing a command out of expected bounds.

But I'll also note that we have "pass arguments to actions" as an open item in https://github.com/hashicorp/nomad/issues/18627, so it seems like there's definitely room to consider this. We may want to make sure Actions Governance has been implemented (or at least designed-out) first. Actions does feel like the place this wants to live, in any case!

[Thunderbottom]

That does makes sense, thanks! Although adding this feature would make things better for us, and I believe for a lot other people as well. Do let me know if you would like to have further discussions on the implementation of this feature. We use nomad extensively and it's a huge part of our infrastructure, so we can definitely try to help out with this.

[okbrown]

This would be really useful, for example the default on exec alloc is /bin/bash but what if this is not the case and you use /bin/sh? There is no way to change this default?

Unless you know you can just change it, but as a irregular user I have to know this ahead of time then rely on the cookie to remember this.