madler / zlib

A massively spiffy yet delicately unobtrusive compression library.
http://zlib.net/
Other
5.58k stars 2.43k forks source link

Non-blocking handling in gzread #992

Open drbitboy opened 1 month ago

drbitboy commented 1 month ago

Caveat

I am new to zlib development; pehraps this has been brought up before and found to be insoluble; if so, please forgive me and feel free to delete this issue.

Background

The functions gzread/gzwrite are a sweet pair of near-analogs for read(2)/write(2), respectively (I was trying to re-invent their wheels over the past several weeks until I stumbled onto them ;-).

I am hoping to compress a stream of multiple XML entities (INDI protocol; see github.com/magao-x/MagAOX) with zlib, and gzread/gzwrite look like a nice way to do that since they handle all the intermediate buffering. The INDI server handles multiple connections, using select(2) and non-blocking I/O.

Issue

Of course gzread works as designed, but it does depend on the compressed stream not being non-blocking, so the read(2) will block when expected data are not yet available, so a return value of 0 can be interpreted as the other end closing the file descriptor i.e. EOF.

If the compressed stream is non-blocking, then a read(2) could return a status code of -1* with errno equal to EAGAIN or EWOULDBLOCK, for the case where the next data are not yet available e.g. TCP socket, which is normal, which would be interpreted by the current code develop//ceadaf2/gzread.c/line27 as an abnormal problem and pass the -1 return up the call chain, which would stop the decompression.

* N.B. this should not be possible if the stream is blocking, so the proposed PR does not affect gzread in that case.

PR

I will make a pull request from drbitboy/zlib-drbitboy.git, branch develop-20240718-drbitboy to madler/zlib branch develop. It's a very small change at this point, but I think it is only the initial step in making what I want to do work, so even if it makes sense don't merge it yet.

The biggest concern here would be if my eventual PR breaks existing applications; the minor change to this point should not do that, but who knows what else will be necessary; in that case I think special-purpose routines in gzread.c to handle non-blocking I/O would be the best path.

Best regards,

Brian T. Carcich Latchmoor Services, INC.

drbitboy commented 1 month ago

PR is #993

drbitboy commented 1 month ago

PR is #993

Seems to work now (https://github.com/drbitboy/zlib-drbitboy/commit/6e8ddfd960d9d9c2657cc6be3d5f192b72c8cfba); needs a little cleaning up (comments), and I plan to write a test for it.

Let me know if there is anything else you want to see before starting a formal review.

drbitboy commented 1 month ago

Test added for the new code; PR should be ready to merge. lmk, thanks!

drbitboy commented 1 month ago

I have used these changes to add compression to the MagAOX repository; see PR https://github.com/drbitboy/MagAOX/pull/8.