Implement NGSIv2 notification processing endpoint (federation scenarios)

[fgalan]

Implement a notification processing endpoint in the NGSIv2 API, similar to the one existing for NGSIv1 in POST /v1/notifyContext. This would allow "pure" NGSIv2 federation scenarios.

Note that current Orion version implements semi-NGSIv2 federation scenarios, using NGSIv2 subscriptions with "attrsFormat": "legacy" so notification can be sent using NGSIv1 format to POST /v1/notifyContext endpoint. However, there is a limitation: JSON native values (e.g. numbers) are string-fied when processed by the POST /v1/notifyContext operation. Implementing the pressent issue will solve this limitation.

[fgalan]

Upon completion of this issue the documentation on https://fiware-orion.readthedocs.io/en/master/user/federation/index.html should be update to decribe the pure NGSIv2 case (the old NGSIv1 could be moved to a separate document in a ending section on the same document).

[jmcanterafonseca]

do you mean replication scenarios?

[fgalan]

I mean an scenario in which the notifications from one CB are received by a second CB, which processes them as updates. Traditionally, this has been referred as "federation", but I guess that "replication" has a close meaning.

[jmcanterafonseca]

yes, better to name it as replication and not as federation, because federation has more to do with multiple context brokers each one serving data for different scopes, entity types, etc.

2017-10-02 13:37 GMT+02:00 Fermín Galán Márquez notifications@github.com:

I mean an scenario in which the notifications from one CB are received by a second CB, which processes them as updates. Traditionally, this has been referred as "federation", but I guess that "replication" has a close meaning.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/telefonicaid/fiware-orion/issues/2986#issuecomment-333509812, or mute the thread https://github.com/notifications/unsubscribe-auth/AAqZ_hfT-WSGD0fzcSH-U2bznIpuV-5rks5soMsTgaJpZM4PixfW .

[fgalan]

I'm afraid, I'm not getting your point...

The scenario I described ("the notifications from one CB are received by a second CB, which processes them as updates") is also used in the case you mention. You can have N CBs, each one speciallized in a given kind entity type / scope / whatever, all them notifying to an agregattion CB. This is also a federation scenario.

[jmcanterafonseca]

this is an important issue for implementing the IDS concepts so we need to deal with it very soon

[fgalan]

this is an important issue for implementing the IDS concepts so we need to deal with it very soon

Please have a look to the NGSIv1 autocast feature described in https://fiware-orion.readthedocs.io/en/master/user/ngsiv1autocast/index.html, recently added to Orion. Maybe it could suffice your federation case, based in in NGSIv1.

[jmcanterafonseca]

I can confirm the autocast option works properly. Just to note that the official docker runs the broker enabling the autocast mode so nothing has to be added to command line if using the official docker.

[kzangeli]

Documentation for this issue:

• modifications in devel manual ("duplicate" what is already done for NGSIv1 notifs)
• modifications in NGSIv2 .apib
• modification in user manual (I think there is a federation.md document or similar name that needs to be adjusted)

[jmcanterafonseca]

why the apib has to be modified?

2018-04-10 21:39 GMT+02:00 Ken Zangelin notifications@github.com:

Documentation for this issue:

• modifications in devel manual ("duplicate" what is already done for NGSIv1 notifs)
• modifications in NGSIv2 .apib
• modification in user manual (I think there is a federation.md document or similar name that needs to be adjusted)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/telefonicaid/fiware-orion/issues/2986#issuecomment-380222479, or mute the thread https://github.com/notifications/unsubscribe-auth/AAqZ_tXft_RH2-U0ijpAq5cpt9jKEW_Jks5tnQqHgaJpZM4PixfW .

[fgalan]

To add the new operation. If I remember correctly POST /v2/op/notify is not included in the specification (I may be wrong).

[kzangeli]

Some tests/test modifications still missing:

• Error Handling
• Possible beautification of cases/2986_apiv2_notifications/apiv2_notification_reception.test

[fgalan]

Implementation in PR https://github.com/telefonicaid/fiware-orion/pull/3157 (although there are yet pending bits, as commented above)

[fgalan]

Documentation in NGSIv2 spec done in PR https://github.com/telefonicaid/fiware-orion/pull/3185

[fgalan]

PR https://github.com/telefonicaid/fiware-orion/pull/3232 covers documentation mentionced above:

• modification in user manual (I think there is a federation.md document or similar name that needs to be adjusted)

In addition, it adds an implemetnation note about lack of keyValues support.

With regards to:

• modifications in devel manual ("duplicate" what is already done for NGSIv1 notifs)

I think the way this has been implemented in PR https://github.com/telefonicaid/fiware-orion/pull/3157 doesn't need any modification to the devel manual (appart from maybe an update to https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/devel/ServiceRoutines.txt), but I'd like to have a second opinion (@kzangeli )

[fgalan]

After merging PRs #3185 and https://github.com/telefonicaid/fiware-orion/pull/3232 the only pending think in this issue is to implement options=keyValues in notification. Once done, the note about the gap included in https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/user/ngsiv2_implementation_notes.md must be removed.