A chunked parser that failed to consume the next token means either:
Insufficient tokens. The user must buffer more and try again.
Invalid token.
There's nothing polemic about the first case. As for the second one:
I. Either the stream is invalid and it'll abort parsing;
II. Or we're using a syntactic extension to the JSON stream and want to
trigger a fallback mini-parser to consume that part of the stream.
If a parser doesn't report invalid token the user will try to buffer the
stream forever. However if invalid token is a terminal state then the
user can't combine parsers to support an extended JSON syntax. A chunked
parser meant to be used combined with other parsers should report all 3
scenarios:
Valid token.
Insufficient tokens.
Invalid token (and invalid token cannot be a terminal state).
If we exclude the first event from the notification set the parser isn't
useful at all. If we exclude the second event then we don't have a
chunked parser. If we exclude the third event the user might buffer
forever on invalid tokens. Thus all three events must be part of the
notification set.
Not sure how to handle the requirement to reach a non-terminal invalid-next-token state. I mean the interface would have to change here. Which small interface change should be implemented then?
RATIONALE
A chunked parser that failed to consume the next token means either:
There's nothing polemic about the first case. As for the second one:
I. Either the stream is invalid and it'll abort parsing; II. Or we're using a syntactic extension to the JSON stream and want to trigger a fallback mini-parser to consume that part of the stream.
If a parser doesn't report invalid token the user will try to buffer the stream forever. However if invalid token is a terminal state then the user can't combine parsers to support an extended JSON syntax. A chunked parser meant to be used combined with other parsers should report all 3 scenarios:
If we exclude the first event from the notification set the parser isn't useful at all. If we exclude the second event then we don't have a chunked parser. If we exclude the third event the user might buffer forever on invalid tokens. Thus all three events must be part of the notification set.
Updates #53