WebSocket SPARQL Protocol

afs commented 5 years ago

Why?

HTTP 1/1 has limitations such as the status code required to be sent first, not allowing for errors or timeouts during results response (#51).

WebSockets is designed for building different interaction protocols. It has the potential to be faster as well.

Related Issues

Issue	Description
#7	Asynchronous SPARQL
#8	SPARQL Protocol support for structured error responses
#49	Database Cursors / keyset pagination
#51	standardized communication of partial results
#60	Relaxation of transaction requirements
#63	Reveal effects of update operations (DELETE, INSERT)
#82	Binary encodings for SPARQL results
#83	Protocol Transactions

Proposed solution

This needs a lot of requirements work to find an acceptable balance of features and practicality. Adding all features often results in a messy and hard to implement solution. It is better to identify fundamental concepts and primitives.

WebSockets provides a framed-message exchange layer. Work on this feature would define the message for SPARQL, and include "trailing error codes" to communicate partial results.

Considerations for backward compatibility

Problems if one transport and not the other is offered (server) or provided (client) similar to content negotiation failure.

Other

If structured as an abstraction, with implementation in plain HTTP and web sockets, then other transports could arise.

akuckartz commented 5 years ago

Do these issues persist with http/2 ?

afs commented 5 years ago

HTTP/2 does not change the HTTP request-response model (methods, status codes, header fields) including terminating proxies and response caching.

Partial results #51 and multi-request transactions #83 don't fit that model very well.

HTTP/2 itself does provide for exchange of frames and this is used to multiplex HTTP request-response on one connection per origin connection. The framing isn't exposed to HTTP level, at least not in APIs. (Even if it were, the question of what SPARQL-level messages to send/receive remains - which is the WebSockets situation.)

Websockets is low level bi-directional exchange of application frames where the setup uses HTTP and then HTTP interaction semantics does not apply.

If structured as an abstraction, i.e. the protocol is described as a series of messages, this could be fitted over use HTTP 1.1, HTTP/2, WebSockets or even plain TCP.

HTTP/2 -- RFC 7540. WebSockets -- RFC 6455.

lisp commented 5 years ago

yes. one approach is to "simply" wrap the http interaction in websocket frames. a number of things suggest this approach:

an http implementation is required in order to initiate the connection.
http provides a well-defined encoding for the necessary metadata.
one can piggyback on the http request/response semantics.
the implementation is small.
necessary extensions to the sparql/http protocol are minimal.
the approach provides a path to gsp/http+ws.

it is possible to implement in a short (800 line) extension (based on hunchensocket), to a variant of hunchentoot which houses a sparql+gsp implementation. the 1K includes the ws codec and state machine and the ws-specific response logic.

this implementation reiterates the response headers with each ws frame, which does have the disadvantage of a large envelope, but that could be mitigated by encoding just deltas. it also involves an extension to (sparql+gsp)/http to indicate streaming response windows and the requisite control flow, which is implemented elsewhere in the sparql+gsp processor's request processing.

VladimirAlexiev commented 2 years ago

Hi @afs, can you elaborate on how this is related to #49? To implement this, one also needs to address #49?

afs commented 2 years ago

To implement this, one also needs to address https://github.com/w3c/sparql-12/issues/49?

No, #49 is not needed. Different levels in the stack; #49 is "pull" (i.e. make cursor requests).

(and WebSockets is probable superseded by HTTP/2 and HTTP/3 - and still "no".)

w3c / sparql-dev