zeromq / zbroker

Elastic pipes
Mozilla Public License 2.0
50 stars 11 forks source link

Problem: client potentially loses data #22

Closed rpedde closed 10 years ago

rpedde commented 10 years ago

If the buffer provided by a client is smaller than the size of the pending data chunk, the excess data in the chunk is thrown away.

The client read function really needs to keep an offset of the position of the current chunk, and continue to serve from it until the chunk is exhausted.

In a perfect world, we'd never return short reads, either. Reads shorter then requested size happen, but are surprising unless at end of file, I think. So a single read in excess of the pending chunk size should probably continue to pull chunks until the client supplied buffer is full.

This might be something better deferred to after the raw client re-write, though. Just noting this for posterity.

hintjens commented 10 years ago

Are we agreed that the API should deliver a stream, not chunks?

On Tue, Mar 25, 2014 at 11:53 PM, Ron Pedde notifications@github.comwrote:

If the buffer provided by a client is smaller than the size of the pending data chunk, the excess data in the chunk is thrown away.

The client read function really needs to keep an offset of the position of the current chunk, and continue to serve from it until the chunk is exhausted.

In a perfect world, we'd never return short reads, either. Reads shorter then requested size happen, but are surprising unless at end of file, I think. So a single read in excess of the pending chunk size should probably continue to pull chunks until the client supplied buffer is full.

This might be something better deferred to after the raw client re-write, though. Just noting this for posterity.

Reply to this email directly or view it on GitHubhttps://github.com/zeromq/zbroker/issues/22 .

rpedde commented 10 years ago

I think from the client perspective, it should be a stream, yes.

hintjens commented 10 years ago

OK, that's clear, thanks.

hintjens commented 10 years ago

Closing this and replacing with #24.