StackStorm / st2

StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, incident responses, troubleshooting, deployments, and more for DevOps and SREs. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html
https://stackstorm.com/
Apache License 2.0
5.97k stars 745 forks source link

[Feature] API endpoints for ST sync execution #5527

Open DavidMeu opened 2 years ago

DavidMeu commented 2 years ago

Saw that issue: https://forum.stackstorm.com/t/is-it-possible-to-execute-an-action-synchronously-with-the-api/99

So thought a good Async to sync ST execution can be in the following form. concatenating sync/ for the relevant new endpoints.

Asynchronous Pattern:

With the asynchronous pattern, if the remote server indicates that the request is accepted for processing with a 202 (Accepted) response, we will have to keep polling a URL specified in the response's location header until reaching a terminal state.

This pattern specifies that after an HTTP action calls or sends a request to an endpoint, service, system, or API, the receiver immediately returns a "202 ACCEPTED" response. This code confirms that the receiver accepted the request but hasn't finished processing. The response can include a location header that specifies the URI and a refresh ID that the caller can use to poll or check the status for the asynchronous request until the receiver stops processing and returns a "200 OK" success response or other non-202 response.

So Once we are creating an execution (Should return 202 status code with the location URL): curl -X POST -H 'Connection: keep-alive' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: python-requests/2.11.1' -H 'content-type: application/json' -H 'X-Auth-Token: da5ecf3b0ab841008d663052fe95cddd' -H 'Content-Length: 69' --data-binary '{"action": "core.local", "user": null, "parameters": {"cmd": "date"}}' http://127.0.0.1:9101/v1/sync/executions The response can include a location header that specifies the URI. I.E: <HOST>/api/v1/sync/executions/{id}

Full URL for polling will be like (Getting ST execution) (Should return 202 status code until it has a final state and will return a "200 OK" success response): curl -X GET -H 'Connection: keep-alive' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: python-requests/2.11.1' -H 'X-Auth-Token: da5ecf3b0ab841008d663052fe95cddd' http://127.0.0.1:9101/v1/sync/executions/58de117e49d4af083399181c

An HTTP 202 response should indicate the location and frequency that the client should poll for the response. It should have the following additional headers:

Header | Description | Notes -- | -- | -- Location | A URL the client should poll for a response status. | This URL could be a SAS token with the Valet Key Pattern being appropriate if this location needs access control. The valet key pattern is also valid when response polling needs offloading to another backend Retry-After | An estimate of when processing will complete | This header is designed to prevent polling clients from overwhelming the back-end with retries.
arm4b commented 2 years ago

Creating a new execution via POST https://api.stackstorm.com/api/v1/executions/#/action_executions_controller.post will return the execution ID for later polling so you can construct an URL, as you want. We don't want to hint or redirect to a new Location via HTTP header as a response to REST request.

For the GET https://api.stackstorm.com/api/v1/executions/#/action_executions_controller.get_all, it would be an indeed nice enhancement to return the 202 HTTP status code until the action is finished its execution (HTTP 200). And so someone could just check for a specific HTTP response code to understand if the action is still running or not instead of parsing the json body.

I don't see a use-case for Retry-After as there is no way to predict the execution time and clients will use their own opinonated retry anyway. It's not like 5xx status where the server can recommend coming later after the specified delay to reduce the overload and which clients might really respect.


And so this way adding HTTP response: 202 (running) vs 200 (done) as an enhancement to the st2 execution status would be indeed nice, and there is no need to add the new API endpoints.

DavidMeu commented 2 years ago

@armab Agreed. Very good prespective thanks! Just been thinking that users will probably want a single endpoint to call and get a sync response as stated here. I see what you mean with:

We don't want to hint or redirect to a new Location via HTTP header as a response to REST request.

So maybe add another single api endpoint creating the execution+returning the result (using 202 status code until we have a final state). POST http://127.0.0.1:9101/v1/executionProcessing (Probably the name can bebetter)

BTW I think you meant this endpoint : GET https://api.stackstorm.com/api/v1/executions/#/action_executions_controller.get_one Instead of:

GET https://api.stackstorm.com/api/v1/executions/#/action_executions_controller.get_all

arm4b commented 2 years ago

This would duplicate the existing API operations and might bring confusion as a result.

First, you need to create an execution:

Having the execution ID, you have to poll its status:

You have to do the same number of requests, you get the same results. The URL is just different. I think there is no need to have another API endpoint as the existing interface is sufficient and the new endpoint provides just the 202 status code which could be already applied to the existing API endpoint as an enhancement.