search

mu-semtech / delta-notifier

Sends notifications about changes to interested mu.semte.ch microservices
0 stars 14 forks source link

readme

Delta Notifier

This component receives raw delta messages from mu-authorization and forwards them to interested entities.

Configuration

Delta's need to be sent from mu-authorization to the delta-notifier. The delta notifier needs to be configured to send the right information to your microservice.

Wiring mu-delta-notifier in a mu-semtech stack

The delta-notifier needs to receive messages from mu-authorization and it needs to send messages to other services. Ideally this would be expressed by links to make the communication paths clear. The link option in docker-compose creates an alternative name and places a dependency between the services. We cannot use this as it creates a loop some-service -> mu-authorization -> delta-notifier -> some-service. We do advise documenting which services consume the delta service in docker-compose comments to ensure the application flow is clear.

More information on wiring mu-authorization and the mu-delta-notifier can be found in the documentation of mu-authorization. At the time of writing, you can add a file in mu-authorization's config (most often at config/authorization/delta.ex) and include the following contents:

defmodule Delta.Config do
  def targets do
    [ "http://deltanotifier" ]
  end
end

Including the delta-notifier in your stack

Default inclusion of the delta-notifier looks like this:

  deltanotifier:
    image: semtech/mu-delta-notifier
    volumes:
      - ./config/delta:/config

Receiving delta notifications

Receiving services should be configured in config/delta/rules.js. The format of this file is in flux, yet it is the intention that services consuming these delta messages can consistently receive messages in a specific format. Use the resourceFormat key to select your preferred format.

We first present an example, next we explain each of the properties. The following is a connection to the resource service in config/delta/rules.js.

export default [
  {
    match: {
      // form of element is {subject,predicate,object}
      // predicate: { type: "uri", value: "http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#isPartOf" }
    },
    callback: {
      url: "http://resource/.mu/delta", method: "POST"
    },
    options: {
      resourceFormat: "v0.0.0-genesis",
      gracePeriod: 1000,
      ignoreFromSelf: true
    }
  }
]

The exported property contains an array of definitions, each linking a match to a callback.

Modifying quads

Normalize datetime

To enable normalization of datetime values, set the NORMALIZE_DATETIME_IN_QUAD to "true". This may reduce false effective changes being sent. E.g. timezone differences or "2024-02-22T15:04:37.000Z" being the same as "2024-02-22T15:04:37Z".

Custom quad normalization

Mount a custom function in /config/normalize-quad.js to implement your own quad normalization. See corresponding ./config/normalize-quad.js as example.

Delta formats

The delta may be offered in multiple formats. Versions should match the exact string. Specify options.resourceFormat to indicate the specific resourceformat.

v0.0.1

v0.0.1 is the latest format of the delta messages. It may be extended with authorization rights etc. in the future. The value encoding follows the json-sparql spec RDF term encoding. For example:

    [
      { "inserts": [{"subject": { "type": "uri", "value": "http://mu.semte.ch/" },
                     "predicate": { "type": "uri", "value": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" },
                     "object": { "type": "uri", "value": "https://schema.org/Project" }},
                     {"subject": { "type": "uri", "value": "http://mu.semte.ch/" },
                     "predicate": { "type": "uri", "value": "http://purl.org/dc/terms/modified" },
                     "object": { "type": "literal", "value": "https://schema.org/Project", "datatype": "http://www.w3.org/2001/XMLSchema#dateTime"}}],
        "deletes": [] }
    ]

v0.0.0-genesis

Genesis format as described by the initial Delta service PoC. It looks like:

    { 
      "delta": {
        "inserts": [{"s": "http://mu.semte.ch/",
                     "p": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type",
                     "o": "https://schema.org/Project"}],
        "deletes": [] }
    }

false or undefined

Any falsy value will currently not send the changed triples to the consuming service. Use this if you use this as a trigger for checking the new state in the database.

Debugging

Debugging can be enabled in the service by setting environment variables. The following may be handy:

Extending

You are encouraged to help figure out how to best extend this service. Fork this repository. Run an experiment. Open an issue or PR describing your experiment. Feel free to open up an issue if you would like to discuss a possible extension.