Closed kevinresol closed 7 years ago
Another issue is about tink_streams I believe. I expect the stream stop looping after parsing 3 times, but on neko the stream loops indefinitely.
progress: 05,00,00,00,01,05,00,00,00,02,05,00,00,00,03
read : 05,00,00,00,01
parsed : 05,00,00,00,01
progress: 05,00,00,00,02,05,00,00,00,03
read : 05,00,00,00,02
parsed : 05,00,00,00,02
progress: 05,00,00,00,03
read : 05,00,00,00,03 (expected to stop right at this line)
parsed : 05,00,00,00,03
parsed : 05,00,00,00,03
parsed : 05,00,00,00,03
parsed : 05,00,00,00,03
parsed : 05,00,00,00,03
parsed : 05,00,00,00,03
parsed : 05,00,00,00,03
....(indefinitely)
This is great news (the mongodb driver). Will try to fix it later this day.
I hope I can finish it. The mongodb spec seems huge.
When working with websocket I hit this same wall again. Btw I think I can move the mongo protocol to the new protocol lib too, because the protocol part is already finish. The unfinish(able?) part is the server discovery/recovery part, which is huge.
Investigated a bit and found that the problem lies around how flush is repeated in a Pipe here
Ok, the problem is like so: Taking the example in the first post in this thread, the total incoming data has 15 bytes, which comes in a single batch.
flush(repeat = 1)
is called: 5 bytes is consumed by the parserflush(repeat - 1)
is called because repeat > 0
, 5 more bytes is consumed by the parserflush
anymore, so the parser never has the chance to consume the last 5 bytes.The logic is a bit complex here and I don't have a whole picture so I am afraid of attempting for a fix.
I am writing a parser (for mongodb driver) that will parse the incoming data from a connection but then I hit a strange thing (bug?) that it won't consume all the available data from the source.
Suppose the incoming data is
"05,00,00,00,01,05,00,00,00,02,05,00,00,00,03"
(bytes in hex) Actually, such data comprises of three individual message: -05,00,00,00,01
-05,00,00,00,02
-05,00,00,00,03
For each message the first four bytes is a little endian int32 indicating the length of the message, so all three of them are
05,00,00,00
which is 5.The remaining bytes are just the payload.
At the end of the post is a reproducible example on nodejs. The output such program is
but it is expected that it should parse the third message as well: (but it didn't)
I tried to use a Source of Bytes or BytesInput and in such cases all 3 messages are parsed correctly. So I am not sure if it is my problem not writing a "correct" parser or it should be a bug of the library.