Short Circuit Processing

[mjallday]

Internal discussion

Goal: Allow processing to short circuit and return a response to the caller before the entire request is processed.

Method: Allow returning a Response object instead of the payload. When a Response object is returned the proxy would no longer continue processing and instead return the Response. This is similar to how Flask works.

Example

def validate(input):
  # do something

def process(input, ctx):
   if not validate(input):
    return abort(400, 'Input failed validation')
  return input

Conceptually it would allow this flow

client -> proxy (pre) -> upstream -> proxy (post) -> client
client -> proxy (pre) *abort* -> client

Where the upstream and post processing is no longer invoked.

[mjallday]

the implementation psuedo code will look like this

def execute(input):
  for operation in [operation1, operation2 ... operationN]:
     result = operation.process(input, ctx)
     if result instanceof Response:
        return result
  return input

method signature for abort would look like

def abort(status_code, message):
   return Response(...)

this is http specific (has a http status code) so we'd need to think about how this works with non-http protocols. simplest is probably just return a text message in stderr to the calling system and let it transform into a protocol specific response to the request.

[Agitolyev]

In terms of error handling and circuit braking, I still think that the most impactful will be the introduction of "ErrorCodeAware" Error object, and API will look like:

def process(input, ctx):
  if 1 == 1:
    return ExecutionResult.Ok(input)
  else:
    return ExecutionResult.Error(500, 'Error Message')

It is generic and maps on all protocols, which will allow us to give customers a way to gracefully communicate an error. In the context of an HTTP proxy, all error codes will be treated as HTTP error codes and mapped to HTTP response.

It is about like Spring Handles it:

Normally any unhandled exception thrown when processing a web request causes the server to return an HTTP 500 response. However, any exception that you write yourself can be annotated with the @ResponseStatus annotation (which supports all the HTTP status codes defined by the HTTP specification). When an annotated exception is thrown from a controller method, and not handled elsewhere, it will automatically cause the appropriate HTTP response to be returned with the specified status code.

After we will introduce this basic mechanism we can build more sophisticated and more protocol-specific ways to communicate errors.

[mjallday]

more internal discussion - https://getverygood.slack.com/archives/CE6TQLH7V/p1704987777291259