HTML Snippet should be able to talk with services using event bus

marcinczeczko commented 8 years ago

Knot.x snippets should be able to talk with services through Vert.x event bus only. It implies few design changes to Knot.x that will simplify understanding its architecture. Additionally, such decision does not limit to HTTP protocol of the target services, because event bus service that Knot.x is going to use (from now on) can be treated as extension point. Out of the box Knot.x will be shipped with service that's a bridge between template engine and HTTP based target service. Depending on needs a new event bus based sevice can be implemented to talk using any other communication method, e.g. TCP, ActiveMQ and so on...

In this Story a following aspects need to be implemented:

1. Services configuration - engine JSON configuration

services section of engine configuration

"services": [
    {
        "name" : "first-service",
        "address" : "knotx.core-adapter",
        "config": {
            "path": "/service/twitter/{twitter.params.user}/{uri.pathpart[2]}"
        }
    }
]

Where:

name - Human readable name of the service that will be used on snippet level
address - event bus address of the service
config - configuration data specific to the service. Config object structure is free. It means you can compose any form of json objects or fields inside config. The only requirement is the target service adapter should have contract what's the accepted config structure. Out of the box the Core Service Adapter will be available in Knot.x, see point 4 what config object it accepts. It will be send to the service (as JSON) together with original HTTP request data (path, method, query string params, headers, body), e.g.:

{
   "request" : {..},
   "config" : {..},
   ...
}

2. Template markup syntax updated

The template syntax don't need to carry uri or http method name with it, it should have form as follows:

data-service-<namespace>="<name>"  -- For service for reading data
data-action-<namespace>="<name>"  -- For services for POSTing to

Additionally, some parameters to the service might need to be provided on markup level (e.g. through CMS), those need to be provided using new elements

data-params-<namespace>="{<JSON>}"  -- For both GET or POST

Where namespace is the namespace of on which results will be accessible. E.g.

<script type="text/x-handlebars-template" data-api-type="templating" 
        data-service-tw="twitter" data-params-tw="{'user': 'cognifide'}"
        data-service-second="name2"
        data-action-newsletter="name3">
</script>

Params related to the service will be send to adapter service together with request and config

{
   "request" : {..},
   "config" : {..},
   "snippet.params" : {..}
}

3. Template engine logic

Template engine verticle need to be modified to address following aspects:

Reading new shape of services configuration
Simpler mapping of service name to the service event bus address
Build message payload to adapter service:
- original request + services.config (from services config Json) + service.params (from markup)
POST calls will be done only for action address (data-action-...)
4. Create configurable event bus service - To keep same set of functionality

A new verticle in core to be created, let's call it CoreServiceAdapter.

Core Adapter will accept config object sent from the templating engine. It should have a following syntax:

"snippet": {
   "user" : "cognifide"
},
"config": {
   "path" : "/service/twitter/{uri.pathpart[2]}"
}

Where: config.path - is the target path to the service. It supports syntax of parameterized values as described here Parametrized service calls. However it should be extended - see next point. snippet - is the json object with parameters from snippet (tag data-params-)

Will listen on address configured in config JSON of core adapter verticle
Will accept message consisting of original request, config object can be ignored - as it's going to be used by project specific implementations
Will have it's own configuration JSON - (or can be configured in master config JSON, e.g. in standalone or monolith example app) - this section will go away from engine section

"coreServiceAdapter" : {
    "config": {
      "address": "knotx.core-adapter",
      "client.options": {
        "maxPoolSize": 1000,
        "keepAlive": false
      },
      "services": [
        {
          "path": "/service/mock/.*",
          "domain": "localhost",
          "port": 3000
        },
        {
          "path": "/service/.*",
          "domain": "localhost",
          "port": 8080
        }
      ]
    }    
}

Will use origin request from message payload in order to obtain actual HTTP service endpoint
Will do actual HTTP call to the target service and return call result in the message reply.
5. Parametrized service calls

As stated above, parametrized service calls mechanism will be reflected in core service adapter. However it require extra feature that allows to form an URI using parameters from the snippet params tag. E.g., assuming the core adapter will get following data

payload = {
   "request" : {..},
   "snippet" : {
      "user" : "cognifide"
   }.
   "config" : { 
     "path" : "/service/twitter/{snippet.user}/{uri.pathpart[2]}"
  }
}

The functionality must support snippet to resolve any value inside that json object. Snippet above is self explainable.

6. Core standalone & example updated

Core Standalone need to be updated to start CoreServiceAdapter + standalone json config updated
Example monolith to be updated to use CoreServiceAdapter too.
7. Documentation updated

Wiki documentation need to reflect changes:

configuration of TE, Core Adapter Service
new snippet syntax
Core Adapter Service itself (configuration options, paramterized service calls should fall into this)
General concept of Adapter service and it's benefits (easy to replace without touching core processing, scalable, easy to add custom implementation very easily)
Architecture diagrams
See the conceptual diagram below how the message and processing flow might look

tomaszmichalak commented 7 years ago

Approved.

tomaszmichalak commented 7 years ago

POST error handling clarification

This is handlebars snippet in /userProfile.html

<script type="text/x-handlebars-template" data-api-type="templating" 
  data-service-profile="getprofile" data-params-profile="{'path': '/service/user/{_request.headers.userId}'}"
  data-action-modify="updateprofile" data-params-modify="{'path': '/service/user/{_request.headers.userId}'}">
  <form action="">
    {{#if modify.fail}}
      <intput type="text" name="firstname" value={{_request.params.firstname}}>
    {{else}}
      <intput type="text" name="firstname" value={{profile.firstname}}>
    {{/if}}
    {{#if modify.fail}}
      <intput type="text" name="lastname" value={{_request.params.lastname}}>
    {{else}}
      <intput type="text" name="lastname" value={{profile.lastname}}>
    {{/if}}
    <input type="submit" value="Submit">
  </form>
</script>

User submit form with incorrect firstname and lastname:

POST /userProfile.html
{
  firstname: firstnamevalue
  lastname: lastnamevalue
}

Service returned HTTP 400 with body

{
  "fail": "true"
}

Rendered page looks like:

<form action="">
  <intput type="text" name="firstname" value="firstnamevalue">
  <intput type="text" name="lastname" value="lastnamevalue">
  <input type="submit" value="Submit">
</form>

User updates values and submit form once again:

POST /userProfile.html
{
  firstname: Firstnamevalue
  lastname: Lastnamevalue
}

Validation ok, service returned HTTP 200 with body:

{
  "fail": "false"
}

Rendered page looks like:

<form action="">
  <intput type="text" name="firstname" value="Firstnamevalue">
  <intput type="text" name="lastname" value="Lastnamevalue">
  <input type="submit" value="Submit">
</form>

This solution assumes that POST request is handled before GET request (new approach for forms). As we have separate methods POST and GET requests we need to reflect it in handlebars with #if statements.

marcinczeczko commented 7 years ago

Small comoment. data-action and data-action-params attributes are enough. Don't need to have extra namespace for action. It can be just one on the snippet. In handlebars it can be available simply as {{action.....

marcinczeczko commented 7 years ago

Agree. POST must be always processed as first. But I'd not touch this in this ticket. It's enough, let's address it in forms ticket and here keep existing behavior. Simply one step at a time ;)

Knotx / knotx