notifications api

[qwesda]

the clients should be able to register themselves for notifications

the workflow might be as follows:

• client establishes connection(s) to the server on _notifications/<resource> to be informed of changes of certain types, e.g.

var genresFeed  = EventSource("_notifications/genres");
var formatsFeed = EventSource("_notifications/formats");
var labelsFeed  = EventSource("_notifications/labels");

• client establishes a connection to the server on _notifications/<resource>/1,2,3,4,5,6,7,8), (102, "artists", [11,12])

var watchedArtistsFeed  = EventSource("_notifications/artists/234,435643,4534");
var watchedReleasesFeed = EventSource("_notifications/release/435,4325,6547567,232,432,4235");

• server checks if the user who is associated with the connection has the rights to see these resources and stores the notification-requests that pass this test

• a resource gets created

  • if any notification-requests exist for this resource-type the registered client receives a message like

  event:resourceCreated
  data:["releases", 24]

• a resource gets modified

  • if any notification-requests exist for this resource the registered client receives a message like

  event:resourceModified
  data:["releases", 24, { "title" : "new title" }]

  • if any notification-requests exist for a resource that is directly related to the modified the registered client receives a message like

  event:resourceModified
  data:["releases", 24, { "title" : "new title" }]

• a resource gets deleted

  • if any notification-requests exist for this resource or for this resource-type the registered client receives the message

  event:resourceDeleted
  data:["releases", 24]

  • if any notification-requests exist for a resource that is directly related to the modified the registered client receives the message

  event:relatedResourceDeleted
  data:["releases", 24]

• a relation gets created

  • if any notification-requests exist for any of the related resource-types the registered client(s) receive(s) the corresponding message

  event:relationCreated
  data:["releases", 24, "releses_artists", {artist_id: 34, release_id: 24, role: "main artist"}]

  or

  event:relationCreated
  data:["artists", 34, {artist_id: 34, release_id: 24, role: "main artist"}]

• a relation gets modified

  • if any notification-requests exist for any of the related resource-types the registered client(s) receive(s) the corresponding message

  event:relationModified
  data:["releases", 24, "releses_artists", {artist_id: 34, release_id: 24, role: "main artist"}]

  or

  event:relationModified
  data:["artists", 34, "releses_artists", 12, {artist_id: 34, release_id: 24, role: "main artist"}]

• a relation gets deleted

  • if any notification-requests exist for any of the related resource-types the registered client(s) receive(s) the corresponding message

  event:relationDeleted
  data:["releases", 24, "releses_artists", 102]

  or

  event:relationDeleted
  data:["artists", 34, "releses_artists", 12]

a change notification should be the following tuples:

• resourceCreated, resourceDeleted, relatedResourceDeleted (MESSAGE_TYPE, RESOURCE_NAME, RESOURCE_ID)
• resourceModified, relatedResourceModified (MESSAGE_TYPE, RESOURCE_NAME, RESOURCE_ID, DATA)
• relationCreated, relationModified (MESSAGE_TYPE, RESOURCE_NAME, RESOURCE_ID, RELATION_NAME, DATA)
• relationDeleted (MESSAGE_TYPE, RESOURCE_NAME, RESOURCE_ID, RELATION_NAME, RELATION_ID, DATA)

[ibotty]

let's evaluate server sent events. they might be what we really want. and might be faster as well.

https://developer.mozilla.org/en-US/docs/Server-sent_events/EventSource

(that was the tech i had in mind the day before yesterday but could not remember.)

[ibotty]

ah. there is a polyfill for internet explorer: https://github.com/remy/polyfills/blob/master/EventSource.js

[qwesda]

if I understand correctly this will only handle messages from the server to the client. how should the client subscribe to notifications?

[qwesda]

the only possibility i see with SSE would be for _register_fortypes new EventSource("_notifications/<resource_type>"); – closing the connection would be unsubscribing.

But how should the client subscribe ti individual ids? new EventSource("_notifications/<resource_type>/234,456,767,8678,769,43,54,534,645788,678"); or the same thing with ?ids= both are a tad ugly ... but ok with me

[ibotty]

this is a limitation indeed. in the end it's another ad-hoc registration. can we live with it or do we need the extra flexibility?

sse might be a little more expensive (more open connections) when listening to many different notifications.

with an ie polyfill browser support is identical (or in favor of sse, i did dont bother checking older ie) comparing to websockets.

sse are a little easier to implement server-side. client-side at least reconnection logic is handled by sse and not by websockets (of course that is not that big a deal). dispatching the different kind of notifications will be left to the client js, but that shouldn't be a big deal.

broken proxies between the server and the client might not support websockets but sse's. i don't consider that very relevant though.

(coloring the bike shed purple i favor "releases/id1,id2,id3" as channel name)

[ibotty]

in all: i guess i'll leave it for you to decide.

[qwesda]

EventSource("_notifications/<resource_type>/id1,id2,id3"); is fine by me. The server should then pass the messages like this:

event: relationCreated
data: ["releases", 24, "releses_artists", {"artist_id": 34, "release_id": 24, "role": "main artist"}]

[qwesda]

I wouldn't bother with the polyfill right now since no one has an IE to test it anyway

[ibotty]

so it is server sent event then?

i'd say for all resources of type res it should simply be

  _notifications/res

(with or without trailing slash.)

k?

[ibotty]

let's just disallow text primary keys with ,. they won't work with that protocol.

[qwesda]

primary keys should only be of type int or uuid - no text primary keys allowed

[ibotty]

disregarding cartesian product of primary keys which is very useful, i agree for database layout in general. but pg-crud should support strange primary keys as well. enums and even text might make sense for some applications as well.

that's a little besides the point though.

[qwesda]

updated spec

[ibotty]

we might think about how to handle event ids. the following scheme might work for connections losses as well as suspended (i.e. inactive) tabs.

• the event id is incremented in a opaque scheme (to be defined later);
• pg_crud will cache events up to N notifications together with its event id;
• after sse connection re-connect check whether the last id (Last-Event-ID) is still cached. if so, it will send only those change events. otherwise it will send a potentially all resources are invalid event with the last event id (and maybe the payload as array).

that way, the client does not need to re-get all resources of interest when nothing changed after resuming.

i suppose there is a way to specify last event id from the browser apis when opening a eventsource connection. i have not checked that. edit: but there is not.

[qwesda]

I don't think this will be necessary. If a tab gets deactivated the client will close the SSE-connections. If a tab gets reactivated i will request the visible data again from the server and reestablish the SSE-connection.

[ibotty]

i wanted to avoid the need to retrieve the information for every tab switching.

[ibotty]

a resume of our phone call is, that

• there should be some kind of rate-limiting. another type of message will be send in that case.
• subscribing to the whole collection won't send modification events.

there is still the question on how that much policy will work generically (i.e without hard-wiring up the notify channel name).