[Enhancement] Improve experience for apps that only execute tasks.

[zrob]

There is a category of apps that never have a long-running process. Instead they house code to execute a task periodically. The needs of these apps are different than apps that house long-running processes in a number of ways.

• No need for a route
• No need to be started

There are also needs similar to long running apps that are not fulfilled by the existing task experience.

• Use a buildpack provided start command
• Stage new application source on push

An ideal workflow for handling task-only apps should mirror, as much as possible, the experience of pushing an app that has long-running processes. It’s useful to illustrate the differences in UX.

Deploy an app with long-running processes

cf push my-app

The app source is uploaded, staged, start command automatically stored, and started.

Deploy and execute a task only app

cf push my-app --no-route --instances 0
cf run-task my-app “some-command”

Points of interest:

• The deployer must know the trick of using --instances 0. Looking at this it’s not obvious even why this is needed. The answer is that the app bits must be staged, but the app should not be started and health-checked like a long running app. So instances 0 prevents the app from actually running anything. Not at all intuitive.
• The command must be entered at the time of invoking the task. How does a user decide what command to use? Ideally the command can be specified by a buildpack. This means a user must inspect the API to uncover the start command to then copy and paste for every task invocation.

An improved UX

There is a definite opportunity to improve the UX here. I believe there are a handful of use cases to consider.

1. The app is task-only and wants to use a buildpack defined command to perform a single job
2. The app is task-only and has multiple jobs it can perform that it specifies in a manifest
3. The app has both long-running processes and tasks

In all cases it is beneficial to encode the start command into the app definition to be used later. This feature is now enabled in the CF API using template processes when creating a task http://v3-apidocs.cloudfoundry.org/version/3.70.0/index.html#create-a-task.

This leads to these proposed UXs for the above use cases.

1. The app is task-only and wants to use a buildpack defined command to perform a single job

   cf push my-app --task
   cf run-task my-app

   This behavior relies on an agreed upon convention between buildpacks and CF. If a buildpack detects a task based app, it will provide a process type of task with the desired start command.

   If there is no process type task then the above run-task invocation would error.

2. The app is task-only and has multiple jobs it can perform that it specifies in a manifest

cf push my-app --task
cf run-task --process batchjob1
cf run-task --process batchjob2

This behavior assumes the app specified processes named batchjob1 and batchjob2 via either a Procfile or manifest, with valid start commands.

3. The app has both long-running processes and tasks

cf push my-app
cf run-task --process batchjob1

This behavior assumes the app has web and batchjob1 process types defined.

Wrapping up

An alternative UX could be cf push my-app --type task. This might be more flexible if there were other types of push in the future, perhaps cf push my-app --type function for example.

Another interesting question that is raised looking at the existing command is the required COMMAND field. cf run-task APP_NAME COMMAND [-k DISK] [-m MEMORY] [--name TASK_NAME]

In this proposal it could either become optional and remain as a positional arg to not break existing scripts. cf run-task APP_NAME [COMMAND] [-k DISK] [-m MEMORY] [--name TASK_NAME] [--process PROCESS_TYPE]

Or it could move into a named flag. cf run-task APP_NAME [--command COMMAND] [-k DISK] [-m MEMORY] [--name TASK_NAME] [--process PROCESS_TYPE]

I’d be happy to provide some PRs to enhance both the push and run-task commands, probably on the v7 side only. Any feedback greatly appreciated!

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/165868766

The labels on this github issue will be updated when the story is started.

[abbyachau]

Thanks @zrob sorry for the delay. I was hoping we would get feedback on this issue. I'll circle around again soon. Thanks.

[beatrichartz]

Thanks @zrob for putting together this summary of the pain points and suggestions, and @abbyachau for pointing me to this. I have recently come across this myself and had to figure out how to push an app for running tasks only.

I like the suggestions you outlined, and it would be great if some UX research could be done for this. I think one other usecase to cover would be pushing and running a one-off task. This is currently only possible by setting instances to 0 as well. So the list of scenarios becomes combinations of:

• Number of times I intend to run the task (once per update, repeatedly)
• There is a long-running process on the instance (yes, no)
• Number of possible jobs the instance can run (n)

The combination of cf push and cf run-task seems like the right way to approach this, with cf push covering the update concern, and run-task covering the execution concern.

I wonder whether an alternative could be to define tasks in the manifest, e.g.:

tasks:
  taskA: exec ./taskA
  taskB: exec ./taskB

This could then enable running these named tasks via run-task.

cf run-task my-app taskA

Also, in the absence of a command, this could either assume that it is a task instance, or require --task or something similar to be appended to cf push.

[abbyachau]

Hi @zrob we'll be happy to take a pull request for this, thank you. As usual, it would be good to lay out the workflow in a template (happy + error cases) before proceeding with the pr. I'll forward you the template.

[abbyachau]

Hi everyone, many thanks for commenting on this enhancement. The cf CLI are closing feedback about @zrob's proposal above and plan to move forward with the proposed changes. This is the summary of changes we plan to make:

Current workflow (truncated)

$ push my-app --no-route --instances 0
$ cf run-task my-app command

Proposed new workflow

$ cf push my-app --task
$ cf run-task my-app --process worker --start-command

Summary of changes of user-facing changes

• a new flag on cf push called --task
• a new flag on run-task called --process to run a specific job specified in the manifest

Feedback Required

• a new flag on run-task called --start-command.
• we need feedback on whether the command is something you would like to override? what are the use cases for doing so?
• we've received feedback on the requirements for appending a command

Many thanks and we hope to hear from you all.

[nebhale]

I'm partial to --type task over --task as it makes room for an obvious extension when the expected addition of functions comes to pass.

Overriding commands is very common in the Java world. The command that is provided in the task process type commonly has instance-specific arguments appended to it (e.g. partition parameters for parallel processing of a large data set). So a precedence that follows

1. cf run-task my-app --start-command <COMMAND> (exclusive with --process)
2. cf run-task my-app --process <PROCESS_TYPE> (exclusive with --start-command)
3. cf run-task my-app (implicit task process type)

is correct. Given that --start-command and --process are mutually exclusive, it seems odd that they'd be two different flags. A single flag that first attempts to match process types and then falls back to executing the value seems to be sufficient.

Related to the fact that the command is overridden in many Java use-cases is that the user must have a way to get the existing command for a given process type so that they can append values to it. The --start-command is not particularly useful to users without having that in place first as they can't simply run java foo, they have to start with the 300-character command line created by the buildpack.

[abbyachau]

Hi @nebhale thanks for your feedback, it's always appreciated. We will get back to you as soon as we can. The one issue we were trying to improve for you folks is the requirement to copy and paste the start command - it looks like this might be necessary for some workflows, according to your feedback. We could add the start command to the app display to help ease this pain point.

I think Zach had some additional questions and will get back to this thread perhaps after Summit.

Thanks and speak soon.

[abbyachau]

Hi @zrob many thanks for the pull request for this feature. We plan on releasing this enhancement for tasks in the next v7 beta release.

The changes users should expect on v7 cf CLI:

• a new --task flag on push: if the user has configured their manifest with a task process type, then they can run run-task app.
• run-task now no longer requires a COMMAND as an arg. Instead, users can optionally provide it as a flag option
• run-task now takes an optional --process flag where users can provide a process

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed.