ndjson / ndjson-spec

Specification
680 stars 29 forks source link

How to handle incomplete streams? #43

Open corneliusroemer opened 1 month ago

corneliusroemer commented 1 month ago

Is there a way to detect if an ndjson stream is terminated prematurely by the server?

Assume a server tries to stream the following ndjson:

 {"some":"thing"}
 {"foo":17,"bar":false,"quux":true}
 {"may":{"include":"nested","objects":["and","arrays"]}}

but due to server error, aborts prematurely, after sending the first 2 lines.

In JSON, this is trivially detected:

[
 {"some":"thing"},
 {"foo":17,"bar":false,"quux":true},

was obviously aborted prematurely.

However, this is not at all clear for the equivalent ndjson:

 {"some":"thing"}
 {"foo":17,"bar":false,"quux":true}

Could the spec be updated to indicate how to handle this? How can one indicate that the stream is complete? A closing sentinel could be used for example, e.g. 2 empty lines?