elsony commented 4 years ago

Which functionality do you think we should add?

Add a new mechanism to provide event notification on build status and application status change to reflect the build and run status of the application:

The build and run status will be separated as two different types of notifications since those are two independent operations.
The main goal of providing the build and run status is to allow facilitate IDE integration with the CLI. In a typical IDE support, they may have custom views to display individual application build and run status of a given project. This allows the user to have a quick snapshot view on the projects that they are working on to provide a richer developer experience on application development.
Some examples of the build status are building, build success, build failed. Some examples of application status are starting, started, stopping, stopped.
Individual build and run events can have more detailed messages to provide more detailed notification event, e.g. "building" can provide a detailed message on "Compiling application" or "Packaging application", etc.
Consumer of the notification events can registry to listen to the events and those events will be sent out during the build process.

Why is this needed?

The existing odo CLI relying on the user to take a look at the output of the build in text format to find out the progress of the build and application. This works well in a pure CLI scenario but when used in an IDE integration, it does not provide enough fine grained details to provide a richer IDE developer experience, e.g. IDE won't know when the application build is completed and application is ready for serving that it can start opening the homepage of the application to the user (if there is one) or when they can start shelling into containers if the user needs to, etc.

Checklist for the items to be implement can be found in here: https://github.com/openshift/odo/issues/2550#issuecomment-635556448

elsony commented 4 years ago

/area devfile

elsony commented 4 years ago

/kind feature

jgwest commented 4 years ago

/assign @jgwest

kadel commented 4 years ago

CLI user and also IDE plugins can already know when the build is finished as the build is triggered by odo push (or odo watch) and both commands are synchronous. They report the status and progress. When odo push command is finished the build is finished.

We currently don't have a command that would report if the application that was built already started or not but that could be easily resolved implementing something like odo component status.

elsony commented 4 years ago

For the push status and progress, I think it is currently feeding to the user as log style output to the user on the odo command. If IDEs wants to provider richer integration, e.g. have some kind of custom view to show the list of running application as well as detailed status of the build, e.g. compiling application, starting container, etc., they'll need to default back to text styled output. Providing a proper notification event system that allows tools provider to listen to individual events and display notifications in those custom views will allow them to provide better developer experience on their application.

If we rely on the odo component status, the tools provider will need to periodically calling the command to find out status on applications. Having an event notification system will allow them to listen for changes to update the view as the app status changes.

jgwest commented 4 years ago

Note: The latest version of this proposal is currently available here

ODO build and application status notification

~This comment is a "living document" in that it has been updated based on @kadel's comments plus my own further work on the code; for the original contents see the revision history in the GitHub web UI.(Updated April 22nd, 2020)~

Feedback welcome! I've written up a fair bit of detail, but that is primarily for informative purposes... don't let that stop you from suggesting changes, providing feedback, etc.

Summary

With this issue I looked at how we can consume output from ODO to allow external tools (such as IDEs) to determine the application/build status of an ODO-managed application.

In short, I propose:

New flag for push command, odo push -o json: Outputs JSON events that correspond to what devfile actions/commands are being executed by push.
New ODO command odo component status -o json: Pings the application URLs, and checks the container status every X seconds. The result of both is output as JSON to be used by external tools to determine if the application is running.
A standardized markup format for communicating detailed build status, something like: #devfile-status# {"buildStatus":"Compiling application"}, to be optionally included in devfile scripts that wish to provide detail build status to consuming tools.

Details below.

Terminology

With this issue, I am looking at adding ODO support for allowing external tools to gather build status and application status. We further divide both statuses into detailed and non-detailed, with detailed principally related to statuses that can be determine by looking at container logs.

Build status: A simple build status indicating whether the build is running (y/n), and whether the last build succeeded or failed. This can be determined based on whether ODO is running a dev file action/command, and the process error code (0 = success, non-0 = failed) of that action/command.

Detailed build status: An indication of which step in the build process the build is in. For example: are we 'Compiling application', or are we 'Running unit tests'?

App status: Determine if an application is running using container status (for both local and Kubernetes, various status: containercreating, started, restarting, etc), or whether an HTTP request to the root application URL returns an HTTP response with any status code.

Detailed application status:

While app status works well for standalone application frameworks (Go, Node Express, Spring Boot), it works less well for full server runtimes such as Java EE application servers like OpenLiberty/WildFly that may begin responding to Web requests before the user's deployed WAR/EAR application has finished startup.
Since these application servers are built to serve multiple concurrently-deployed applications, it is more difficult to determine the status of any specific application running on them. The lifecycle of the application server differs from the lifecycle of the application running inside the application server.
Fortunately, in these cases we can use the console logs (from odo log) from the runtime container to determine a more detailed application status.
For example, OpenLiberty (the applications server I am most familiar with) prints a specific code when an application is starting CWWKZ0018I: Starting application {0}., and another when it has started. CWWKZ0001I: Application {0} started in {1} seconds.

Devfile writer: A devfile writer may be a runtime developer (for example, a Red-Hatter working on WildFly or an IBMer working on OpenLibery) creating a devfile for their organization's runtime (for example 'OpenLiberty w/ Maven' dev file), or an application developer creating/customizing a dev file for use with their own application. In either case, the devfile writer must be familiar with the semantics of both ODO and the devfile.

JSON-based ODO command behaviours to detect app and build status

New odo commands and flags:

odo push -o json
odo component status -o json

With these two additions, an IDE or similar external tool can detect build running/succeeded/failed, application starting/started/stopping/stopped, and (in many cases) get a 'detailed app status' and/or 'detailed build status'.

Build status notification via `odo push -o json`

odo push -o json is like standard odo push, but instead it outputs JSON events (including action console output) instead of text. This allows the internal state of the ODO push process to be more easily consumed by external tools.

Several different types of JSON-formatted events would be output, which correspond to ODO container command/action executions:

Dev file command execution begun (command name, start timestamp)
Dev file command execution completed (error code, end timestamp)
Dev file action execution begun (action name, parent command name, start timestamp)
Dev file action execution completed (action name, error code, end timestamp)
Log/console stdout from the actions, one line at a time (for example, mvn build output). (timestamp)

(Exact details for which fields are included with events are TBD, feedback welcome!)

In addition, odo push -o json should return a non-zero error code if one of the actions returned a non-zero error code, other zero is returned.

`odo push -o json` example output

This is what an odo push -o json command invocation would look like:

odo push (component name) --events (other flags)
{ "devFileCommandExecutionBegun": { "commandName" : "build", "timestamp" : "(TBD: unix epoch or ISO standard)" } }
{ "devFileActionExecutionBegun" : { "commandName" : "build", "actionName" : "run-build-script", "timestamp" : "(...)" } }
{ "logText" : { "text:" "one line of text received\\n another line of text received", "timestamp" : "(...)" } }
{ "devFileActionExecutionComplete" : { "errorCode" : 1, ( same as above )} }
{ "logText" : { "text": (... ), "timestamp" : "(...)" } } # Log text is interleaved with events 
{ "devFileCommandExecutionComplete": { "success" : false, (same as above) } }

(Exact details on event name, and JSON format are TBD; feedback welcome!)

These events allow an external ODO-consuming tool to determine the build status of an application (build succeeded, build failed, build not running).

Detailed build status via JSON+custom markup

For detailed build status, I propose that devfile writers optionally include custom markup in their devfile actions which indicate a detailed build status:

If a dev file writer wanted to communicate that the current command/action were compiling the application, they would insert a specific markup string (#devfile-status#) at the beginning of a console-outputted line, and then between those two fields would be a JSON object with a single field buildStatus:
- For example: #devfile-status# {"buildStatus":"Compiling application"} would then communicate that the detailed build status should be set to Compiling application.
Since this line would be output as container stdout, it would be included as a logText JSON event, and the consuming tool can look for this markup string and parse the simple JSON to extract the detailed build status.
Feedback welcome around exact markup text format.

The build step (running as a bash script, for example, invoked via an action) of a devfile might then look like this:

#!/bin/bash
(...)
echo "#devfile-status# {'buildStatus':'Compiling application'}
mvn compile
echo "#devfile-status# {'buildStatus':'Running unit tests'}
mvn test

This 'detailed build status' markup text is entirely optional: if this markup is not present, the ODO tool can still determine build succeeded/failed and build running/not-running using the other odo push -o json JSON events.

App status notification via `odo component status -o json` and `odo logs --follow`

In general, within the execution context that ODO operates, there are a few ways for us to determine the application status: 1) Can the application be pinged at its exposed URL? 2) What state is the container in? (running/container creating/restarting/etc -- different statuses between local and Kube but same general idea) 3) In the application log, specific hardcoded text strings can be searched for (for example, OpenLiberty outputs defined status codes to its log to indicate that an app started.)

Ideally, we like for ODO to provide consuming tools with all 3 sets of data. Thus, as proposed:

1 and 2 are handled by a new odo component status -o json command, described here.
3 is handled by the existing unmodified odo logs --follow command.

The new proposed odo component status -o json command will:

Every X seconds, ping the routes of the application as they existed when the command was first executed. Output the result as a JSON string.
Every X seconds, check the container status for the application, based on the application data that was present when the command was first issued. Output the result as a JSON string.
Note: This command will, by design, NOT respond to route/application changes that occurs during or after it is invoked. It is up to consuming tools
Wait for parent process to terminate it.

This is an example an odo component status -o json command invocation look like:

odo component status -o json

{ "applicationPing" : { "url" : "https://(...)", "response" : "true", "responseCode" : 200, "timestamp" : (unix epoch or iso string) } }
{ "applicationPing" : { "url" : "https://(...)", "response" : "false", error: "host unreachable", "timestamp" : (...) } }
{ "containerStatus" : { "status" : "containercreating", "timestamp" : (...)} }
{ "containerStatus" : { "status" : "running", "timestamp" : (...)} }
(...)

(Exact details on event name, and JSON format are TBD; feedback welcome!)

To keep from overwhelming the output, only state changes would be printed (after an initial state output), rather than every specific event.

Consumption of ODO via external tools, such as IDEs

Based on the existing knowledge from previously building similar application/build-status tracking systems in Eclipse Codewind, I believe the above described commands should allow any external tool to provide a detailed status for ODO-managed applications.

The proposed changes ensure that the the high-level logic around tracking application changes across time can be managed by external tools (such as IDEs) as desired, without the need to leak/pollute ODO with any of these details. These changes give consuming tools all the data they need ensure fast, reliable, up-to-date and (where possible) detailed build/application status.

What happens if the network connection is lost while executing these commands?

One potential challenge is how to handle network connection instability when the push/log/status commands are actively running. Both ODO, and any external consuming tools, should be able to ensure that the ODO-managed application can be returned to a stable state once the connection is re-established.

As I see it:

If network connection is dropped during push: consuming tool can restart the push command from scratch. Well-written dev files should be nuking any existing build processes (for example, when running a 'build' action, that build action should look for any old maven processes and kill them)
If connection is dropped during logs: start a new tail, and then do a full 'get logs' to make sure we didn't miss anything; match up the two (the full log and the tail) as best as possible, to prevent duplicates.
If connection is dropped during status: no special behaviour is needed here.

kadel commented 4 years ago

New flag for push command, odo push --events: Outputs JSON events that correspond to what devfile actions/commands are being executed by push.

+1

New ODO command odo appStatus --events --follow: Pings the application URLs, and checks the container status every X seconds. The result of both is output as JSON to be used by external tools to determine if the application is running.

I should be odo component status --events --follow. Application in odo terminology is a group of components (microservices). The component is where the source code is (component roughly maps to Pod). This can be a little bit confusing :-( But we need to keep it consistent. In the future, we should consider changing how odo sess the components, applications, projects.

Feedback welcome around exact markup text format.

I like what you are proposing. Maybe just one thing. Is it required to end with #devfile-status#? Wouldn't be just enough to have a line starting with #devfile-status# and assume that everything after that until the end of the line is JSON with the information?

Overall I like the proposal.

There already are existing IDE plugins (OpenShift Connector for VSCode, OpenShift Connector for IntelliJ) plugins using odo. @dgolovin and @jeffmaury can you please check the above proposal? Your feedback is important as you are already managing existing plugins on top of the odo.

jgwest commented 4 years ago

Updates since original proposal:

Adopted all @kadel's suggestions, including markup format and odo component status.

In the proposal I've replaced --events with -o json; I was not aware of ODO's use of -o json to trigger machine-readable codepaths at the original time of writing, but it works great here and I've fully adopted it for this.

Standardized on the timestamp format of (Unix epoch UTC seconds).(microseconds portion of time) which is the same log format as other Go-based loggers in this space (eg instead of standardizing on RFC 3339, which was the other option). This was primarily due to the (computational) ease of parsing this format for consuming tools.

kadel commented 4 years ago

There definitely should be a proposal document before this gets implemented!

kadel commented 4 years ago

New ODO command odo component status -o json: Pings the application URLs, and checks the container status every X seconds. The result of both is output as JSON to be used by external tools to determine if the application is running.

What if application doesn't communicate over HTTP(S) and uses some other tcp/udp protocol? Or if the application doesn't expose any ports at all?

jgwest commented 4 years ago

@kadel I thought this was a pretty good proposal.

jgwest commented 4 years ago

What if application doesn't communicate over HTTP(S) and uses some other tcp/udp protocol?

My assumption when initially writing the proposal was that URLs created with odo url create would only be HTTP/S URLs (eg exposed via an HTTP/S ingress); if not then we would need a way to differentiate between URLs which accept HTTP/S requests and those that don't.

Or if the application doesn't expose any ports at all?

This scenario is the easy one: if there are no ports exposed via URLs, then there would be not testing of the URLs and no { "applicationPing" : { "url" : "" } }-style messages would be emitted in this case.

elsony commented 4 years ago

What if application doesn't communicate over HTTP(S) and uses some other tcp/udp protocol?

My assumption when initially writing the proposal was that URLs created with odo url create would only be HTTP/S URLs (eg exposed via an HTTP/S ingress); if not then we would need a way to differentiate between URLs which accept HTTP/S requests and those that don't.

Do we expect to support exposing of URL anything other than HTTP/S? We are currently using ingress for exposing the URL and that is only supporting HTTP/S.

girishramnani commented 4 years ago

@kadel I thought this was a pretty good proposal.

A proposal is a document which is merged in the proposals folder that we have. Currently keeping track of what’s changed is difficult in an issue. So @jgwest please open a PR with a proposal document

jgwest commented 4 years ago

Proposal has been submitted as a PR: https://github.com/openshift/odo/pull/3177

(@girishramnani @kadel @elsony)

kadel commented 4 years ago

Proposal has been submitted as a PR: #3177

(@girishramnani @kadel @elsony)

The reason why I would prefer this to be a separate file and PR is that it makes it much easier to have a discussion there, as you can comment o specific lines. And the proposal document will be a good base for documentation. It is hard to find the comments on the issue especially once the issue is closed.

jgwest commented 4 years ago

Current status for this feature:

[X] Create design proposal and submit for team consideration: Approved and merged 🎉 .
[X] Implement odo push -o json for Kubernetes and Docker: PR: Approved and merged 🎉 .
[x] Implement odo component status -o json for Kubernetes and Docker: PR

openshift-bot commented 3 years ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot commented 3 years ago

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten /remove-lifecycle stale

openshift-bot commented 3 years ago

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen. Mark the issue as fresh by commenting /remove-lifecycle rotten. Exclude this issue from closing again by commenting /lifecycle frozen.

/close

openshift-ci-robot commented 3 years ago

@openshift-bot: Closing this issue.

In response to [this](https://github.com/openshift/odo/issues/2550#issuecomment-749930933): >Rotten issues close after 30d of inactivity. > >Reopen the issue by commenting `/reopen`. >Mark the issue as fresh by commenting `/remove-lifecycle rotten`. >Exclude this issue from closing again by commenting `/lifecycle frozen`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

redhat-developer / odo

Event notification support for build and application status for IDE integration for devfile scenarios #2550

Which functionality do you think we should add?

Why is this needed?

Note: The latest version of this proposal is currently available here

ODO build and application status notification

Summary

Terminology

JSON-based ODO command behaviours to detect app and build status

Build status notification via `odo push -o json`

`odo push -o json` example output

Detailed build status via JSON+custom markup

App status notification via `odo component status -o json` and `odo logs --follow`

Consumption of ODO via external tools, such as IDEs

What happens if the network connection is lost while executing these commands?

Updates since original proposal:

Current status for this feature:

redhat-developer / odo

Event notification support for build and application status for IDE integration for devfile scenarios #2550

Which functionality do you think we should add?

Why is this needed?

Note: The latest version of this proposal is currently available here

ODO build and application status notification

Summary

Terminology

JSON-based ODO command behaviours to detect app and build status

Build status notification via odo push -o json

odo push -o json example output

Detailed build status via JSON+custom markup

App status notification via odo component status -o json and odo logs --follow

Consumption of ODO via external tools, such as IDEs

What happens if the network connection is lost while executing these commands?

Updates since original proposal:

Current status for this feature:

Build status notification via `odo push -o json`

`odo push -o json` example output

App status notification via `odo component status -o json` and `odo logs --follow`