Allow 404s within a Subscription

[toomim]

In existing HTTP, if a resource doesn't exist, the server responds with a 404 not found and ends the connection. A resource that exists can be deleted with the DELETE method, and a non-existent resource can be created with a PUT or POST.

Braid adds subscription to HTTP, and it'd be nice to subscribe to a resource and be notified when it comes into existence, and when it's deleted, in addition to when it changes. However, subscriptions return a 209 status code, and we have no way to send a 404 within a subscription to reflect that the resource disappeared.

I see two high-level options here:

1. We find a way to express a "this resource does not exist" update over a subscription
2. We require that subscriptions only work on resources that exist, and end as soon as a resource is deleted

I'm in favor of (1), but it does require us to invent a new syntax for embedding "resource has been deleted" within a subscription response. Perhaps we can send a header like :status: 404, similar to how HTTP2 represents a status code as a pseudo-header.

[toomim]

This issue comes up in statebus here: https://github.com/invisible-college/statebus/blob/fbe9e7e059bce387a8205684d780725be50f77dc/server.js#L226

[mitar]

Maybe something in the transport for the subscription can be used? In SSE, different event types could be used, one for creation event, one for deletion, and one for change.

[toomim]

Yes, I believe that's what I was suggesting by (1) above. Let's make this more concrete with an example:

HTTP/1.1 209 Subscription
Subscribe: keep-alive

Version: "ej4lhb9z78"                      | This version is deleted
:status: 404                               | 
                                           |
Content-Type: application/json             | Now it exists again
Merge-Type: sync9                          |
Content-Length: 64                         |
                                           |
[{"text": "Hi, everyone!",                 | | Body
  "author": {"link": "/user/tommy"}}]      | |

The question is how to express that a version doesn't exist. We could either make the response status code something that can update over time, like in the above example using http/2 :status: pseudo-header format, or we could find some other way to express "deleted" or "does not exist", e.g.:

Version: "ej4lhb9z78"
Exists: false

Or perhaps by thinking of it as content that doesn't exist:

Version: "ej4lhb9z78"
Content-Length: null

Or that's like, even more non-existent:

Version: "ej4lhb9z78"
Content-Length: -1

Or where there is no content-length in the first place, just the end of headers:

Version: "ej4lhb9z78"

[mitar]

Hm, isn't deletion just a patch which removes everything? :-) So maybe just a patch-language should be rich enough to differentiate between change to "empty data" vs. "remove data itself".

[toomim]

Yes, let's clarify the two types of "deletion":

1. An entire resource can be deleted with the DELETE method.
2. The content within a resource can be deleted with a patch.

This issue is about (1) deleting the entire resource.

There's also an open question about whether any resource metadata is preserved after a DELETE. There's a good argument for at least preserving history, and perhaps a merge-type, so that you can resolve concurrent DELETE and PUT events using the existing time DAG and merge-type.

[mitar]

There might be also another operation, i.e., REDACT, where maybe you are unable to publish anymore the patch (e.g., for legal reasons) but you can still provide metadata so that users might know how the changes happen. This happens often in Wikipedia, where some older versions are redacted/deleted because they are illegal/vandalism/etc., but versions before or after are OK and available.

[toomim]

I presented a new proposal for this on Monday at https://braid.org/meeting-71.

The proposal is to add a status lines to each update, so that a sequence of updates will look like this:

HTTP/1.1 219 Subscription Update
Version: "csjwuxujowf-118"
Parents: "csjwuxujowf-71"
Content-Length: 47
Content-Range: json [1530:1530]

typing typing typing more.  Where is teh bottle

HTTP/1.1 219 Subscription Update
Version: "csjwuxujowf-124"
Parents: "csjwuxujowf-118"
Content-Length: 0
Content-Range: json [1571:1577]

HTTP/1.1 219 Subscription Update
Version: "csjwuxujowf-128"
Parents: "csjwuxujowf-124"
Content-Length: 0
Content-Range: json [1567:1571]

HTTP/1.1 219 Subscription Update
Version: "csjwuxujowf-143"
Parents: "csjwuxujowf-128"
Content-Length: 15
Content-Range: json [1567:1567]

the bottleneck?

HTTP/1.1 219 Subscription Update
Version: "csjwuxujowf-144"
Parents: "csjwuxujowf-143"
Content-Length: 1
Content-Range: json [1585:1585]

HTTP/1.1 219 Subscription Update
Version: "csjwuxujowf-174"
Parents: "csjwuxujowf-144"
Content-Length: 30
Content-Range: json [1583:1583]

Oh, it's on the SERVER!!!!!!! 

We could then have one of those updates return a 404, or invent a new code like 114 for "delete update."