blocking bug in get-bytevector-n!

Reported by: will on Thu Feb 7 10:23:28 2008 As reported by Ray Racine (so "I" means Ray).

I ran across this one when I did a custom io port for the http protocol which one can use to wrap a socket port.

In general, in the http protocol, one always knows how many bytes to read for the payload. But when I tried to read n bytes via get-bytevector-n! I was blocking until the peer closed their side of the connection. This would cause several seconds of delay.

I could never figure out why.

The answer is that in portio.sch, in get-bytevector-n! the loop which reads into the buffer looks like this.

(define (get-bytevector-n! p bv start count)
  (if (and (input-port? p)
           (binary-port? p)
           (bytevector? bv)
           (fixnum? start)
           (fx<= 0 start)
           (fixnum? count)
           (fx<= 0 count)
           (fx<= (fx+ start count) (bytevector-length bv)))
      (do ((n    (fx+ start count))
           (i    start      (fx+ i 1)))
          ((or (port-eof? p) (fx= i n))
           (- i start))
        (bytevector-set! bv i (get-u8 p)))
      (portio/illegal-arguments 'get-bytevector-n! p bv start count)))

It loops until eof is reached OR when the required n bytes have been. The checks occur in that order.

Lets say I do a http get on a socket to some server. The server sends back a http header plus a 10 byte payload response. After reading the HTTP header off of the connection, from the header's Content-Length value I know the payload is 10 bytes in size.

When the 10th byte has been read the ports internal mainbuf (the one in iosys.sch) is exactly empty. At this point I'd like get-bytevector-n! to return as I have no more bytes to read and (fx= i n) is true.

However, first the loop does a (port-eof? p) which causes a io/get-u8 to occur in peek mode. But socket port's buffer is empty, causing a fill buffer to occur, but there are no more bytes to read, i.e., there is nothing to peek, so we sit blocked until the peer decides to close the socket. When I am in tasking + socket non-block mode the task sleeps until poll tells the task there are more bytes available (never happens) or poll triggers on a the socket close. Since the 10 bytes have already been read there is no need to sit blocked.

The answer is for the 'do' loop exit check first see if n bytes have been read, and then check if eof has been reached on the port.

Specifically,

((or (port-eof? p) (fx= i n))

should be

((or (fx= i n) (port-eof? p))

given the left-to-right sequencing of the 'or'.

Similarly in get-bytevector-n.

larcenists / larceny

blocking bug in get-bytevector-n! #524