use with ringbuffer

[jkarneges]

Hi,

First, thanks for this great library!

Is it possible to parse around the edge of a ring buffer boundary? E.g by providing two buffers as input to the parser? Currently I'm shifting all remaining bytes down after every successful parse but it would be nice to not have to do that. Any advice?

[seanmonstar]

I think it could be nice to have an API that takes a Buf. That'd allow ring buffers, and any other instance where you don't have the bytes in contiguous memory.

However, there'd definitely be a performance cost to that, so I don't think it could be the default API... Hm, it might actually be annoying to figure out how to get that to all work nicely...

[jkarneges]

The main times it would be useful are probably:

• Additional pipelined requests after current request. Each subsequent request will need to be memmove'd. If requests aren't pipelined, and the client waits for a response before sending another request, then aligning left immediately after parsing each request would have no cost as there would be no bytes to move.

• Chunked request body. To avoid a chunk header wrapping at the ring buffer boundary, after every read of body the remaining bytes need to be aligned left. However if the user always tries to read all bytes from the buffer, then aligning left would have no cost (the exception being if there is a subsequent request present after the end of the body, but that's the same as the earlier pipelining issue).

I believe pipelining and chunked request bodies are uncommon in practice, so maybe a single buf parser plus memmove (when needed) as we have today is ideal in the general case.

An optional multi-buf parser would be interesting though, that the app could invoke in the right situations, provided that the performance hit of such a parser would not be worse than a memmove.

[seanmonstar]

You don't necessarily need to memmove with pipelined requests. If you're using some sort of cursor, you can just pass the rest of the buffer to be parsed after having finished with the previous request. If you get a partial result back, you can then decide whether to consolidate or just keep reading into the end.

[jkarneges]

Good point. Consolidating when data has already wrapped around can be tricky though, which is why I was shifting after each read to ensure that couldn't happen. But perhaps I can optimize this.