Program hangs after writing too much on an os/pipe

[yohannd1]

Consider this example:

(def [pipe-r pipe-w] (os/pipe))
(pp pipe-w)
(for i 0 16000
  (def str (string i " -- THIS IS A MILDLY SIZED STRING FOR DEBUGGING PURPOSES\n"))
  (pp str)
  (:write pipe-w str))

The last line it should show on the screen (because of pp) is a string beginning with 15999, right? Instead it only lasts till 1021, when it hangs.

using janet v1.30.0-baf4540 uname: Linux core 6.1.44-1-MANJARO #1 SMP PREEMPT_DYNAMIC Wed Aug 9 09:02:26 UTC 2023 x86_64 GNU/Linux

[bakpakin]

This is just how pipes work in general - pipes have an internal, fixed size buffer that is OS dependent. I think Posix specifies it must be at least 512 bytes, but the solution is to have something reading from the pipe.

https://www.linuxquestions.org/questions/linux-newbie-8/write-to-pipe-file-hangs-using-echo-command-4175542239/

[bakpakin]

So this "fix" for this demo is as follows:

(def [pipe-r pipe-w] (os/pipe))
(pp pipe-w)

# drain the pipe until end of pipe
(ev/spawn
  (forever
    (unless (:read pipe-r 4096)
      (break))))

(for i 0 16000
  (def str (string i " -- THIS IS A MILDLY SIZED STRING FOR DEBUGGING PURPOSES\n"))
  (pp str)
  (:write pipe-w str))

# need to signal that we are done writing to pipe
(:close pipe-w)

[bakpakin]

For more info on the original behavior on linux, read https://man7.org/linux/man-pages/man7/pipe.7.html.

See the section on "Pipe Capacity" - Linux has a capacity of 64kB by default for pipes, so you could write about 1024 * 64 byte strings to the pipe before it filled up and blocked further writes.

[yohannd1]

Thanks!