Create a new action for multiple NGSI filter based updates/appends

[manucarrace]

Design an action that given an NGSI filter do the following:

• make the querycontext operation for the NGSI filter
• apply the NGSI action (update/append) to every result

[mrutid]

This action is mandatory to allow schedulling scenarios

[AlvaroVega]

Could you provide me an example of NGSI filter?

[fgalan]

A (non exahustive but pretty complete) example:

GET /v2/entities?q=temperature>40&mq=humidity.avg==80..90&georel=near;maxDistance:100000&geometry=point&coords=40.418889,-3.691944

Probably the best strategy is to store this in a field of the CEP rule in an opaque way, eg:

"filter": "q=temperature>40&mq=humidity.avg==80..90&georel=near;maxDistance:100000&geometry=point&coords=40.418889,-3.691944"

A side advantage is that any extension of Orion filter can be directly used without any extra modificaiton at CEP.

[AlvaroVega]

Example of this new plain rule

"action": {
      "type":"query_and_update",
      "parameters":{
          "filter": "q=temperature>40&mq=humidity.avg==80..90&georel=near;maxDistance:100000&geometry=point&coords=40.418889,-3.691944"
          "update": " ¿?"
          }
      }
}

Regarding with the example about NGSI filter, @fgalan could you provide an example about NGSI update action?

[AlvaroVega]

Instead a querystring parameter like: q=temperature>40&mq=humidity.avg==80..90&georel=near;maxDistance:100000&geometry=point&coords=40.418889,-3.691944 A well formed json options like https://conwetlab.github.io/ngsijs/stable/NGSI.Connection.html#.%22v2.listEntities%22__anchor would be desirable:

Object with extra options:

    attrs (String): Comma-separated list of attribute names whose data are to be included in the response. The attributes are retrieved in the order specified by this parameter. If this parameter is not included, the attributes are retrieved in arbitrary order.
    correlator (String): Transaction id
    count (Boolean; default: false): Request total count
    id (String): A comma-separated list of entity ids to retrieve. Incompatible with the idPattern option.
    idPattern (String): A correctly formated regular expression. Retrieve entities whose ID matches the regular expression. Incompatible with the id option
    limit (Number; default: 20): This option allow you to specify the maximum number of entities you want to receive from the server
    offset (Number; default: 0): Allows you to skip a given number of elements at the beginning
    metadata (String): A comma-separated list of metadata names to include in the response
    mq (String): A query expression for attribute metadata, composed of a list of statements separated by semicolons (;)
    orderBy (String): Criteria for ordering results
    q (String): A query expression, composed of a list of statements separated by semicolons (;)
    service (String): Service/tenant to use in this operation
    servicepath (String): Service path to use in this operation
    type (String): A comma-separated list of entity types to retrieve. Incompatible with the typePattern option.
    typePattern (String): A correctly formated regular expression. Retrieve entities whose type matches the regular expression. Incompatible with the type option.
    unique (Boolean): Represent entities as an array of non-repeated attribute values.
    values (Boolean): Represent entities as an array of attribute values

[fgalan]

We could use the "expressions" format used when the query is codified in subscriptions payloads (see http://telefonicaid.github.io/fiware-orion/api/v2/stable/, "Subscriptiosn" section, about the "expressions" field).

For instance, the above filter will be expressed as:

"filter": {
    "q": "temperature>40",
    "mq": "humidity.avg==80..90",
    "georel": "near;maxDistance:100000",
    "geometry": "point",
    "coords": "40.418889,-3.691944"
}

[fgalan]

Regarding with the example about NGSI filter, @fgalan could you provide an example about NGSI update action?

The idea behind this functionality is that once the rule is triggered (based in the EPL clause) then an update is applied in all entities matching the filter. For instance "update the status attribute to 'on' in all entities of type Lamp.

That would be something like this:

"action": {
      "type":"updateFiltered",
      "parameters":{
          "filter": {
              "type": "Lamp"
          }
          "attributes": [
              {
                  "name":"status",
                  "type":"Text",
                  "value":"on"
              }
          ],
          "actionType": "update"
        }
    }
}

The CEP will do the following:

1. Do a GET /v2/entities?type=Lamp query`
2. For every entity returned by the query, it will do a PATCH /v2/entities/{id}/attrs - {"status: "on"} (or equivalent update method currently used for the regular update action)

A final comment regarding the action name to use. We can either:

1. Use a different name to update (such as updateFiltered)
2. Use the same update action, making filter an optional parameter. If filter is not used, then the behaviour is as it is now ("conventional" update).

Personally I prefer the second one to have a more compact documentation (otherwise we need and extra section in the plain rules documentatio, etc.).

[fgalan]

With regards to ngsijs I have been doing a liltle investigation:

• It supports NGSIv2 batch update: http://conwetlab.github.io/ngsijs/stable/NGSI.Connection.html#.%22v2.batchUpdate%22__anchor. This is good, as it would allow to "pack" several entities in a single operation (the only thing to take care is about the 1MB limit in batch operations, so you need to split in several batches if you have more data than that).
• It seems it doesn't support geo-queries. I'm not fully sure, I have opened an issue about it in their repo: https://github.com/conwetlab/ngsijs/issues/27. In the worst case, we can leave that functionality out of CEP action filters in a first version of this feature.

[fgalan]

It seems it doesn't support geo-queries. I'm not fully sure, I have opened an issue about it in their repo: conwetlab/ngsijs#27. In the worst case, we can leave that functionality out of CEP action filters in a first version of this feature.

I have checked (see discussion on the cited issue) that it is actually supported and it just a problem of documentation.

[fgalan]

Current update action allows to chose between NGSIv1 and NGSIv2.

The new action should use NGSIv2 (note that filters will not work with NGSIv1). In other words, if the filter field is used, then the version field is ignored and the API selection is always NGSIv2.

[AlvaroVega]

@fgalan is another reason to keep a part updateAction from new queryUpdateAction

[fgalan]

Implemented by PR https://github.com/telefonicaid/perseo-fe/pull/360