Why is the tutorial not updated?

[mbwgh]

As a beginner using pipes-concurrency, or more generally the pipes framework, following the examples in Pipes.Concurrent.Tutorial you quickly note that something is off.

The examples make use of deprecated functions, for which a pull request has been submitted almost two years ago.

Starting with the examples in the "Mailbox Sizes" section, you may encounter

thread blocked indefinitely in an STM transaction

errors, an issue which has been known for more than three years. As far as I understand, this is tentatively being regarded as fixed by the recently introduced withBuffer function. From what I have seen, code may still deadlock using withSpawn, which is not mentioned in the API docs either.

The same example, this time using Unbounded, seems to not always terminate. Should this be expected?

These things makes it really hard to learn how to use the library correctly. Should I always use withBuffer instead of "naked spawns"? Why is withBuffer considered a "more restrictive alternative"? Why do I need to pass two continuations (which means the type is kind of "in the way" all the time)? Am I right to assume that I don't need to call performGC then?

The way I see it, as of now the tutorial is not up to par, which regarding the quality of the other documentation and the core API goes against the spirit of pipes.

[mbwgh]

Also, if somebody could elaborate on what something like the "broadcast" example would look like using withBuffer, I would really appreciate it.

[Gabriella439]

Yeah, this is mainly because I haven't had time to update the tutorial yet. I probably won't get to this for another few months at least but I will try to get to this eventually.

[mitchellwrosen]

@Gabriel439 You mentioned here that Carter might know why this regression occurred. Do you happen to have any more information about that? A GHC ticket, or Twitter thread, or something?

As a refresher, this is a program which, if I understand correctly, hasn't worked since 7.8:

import Control.Concurrent.STM
import Data.IORef

main :: IO ()
main = do
  var <- newTVarIO False
  ref <- newIORef ()
  mkWeakIORef ref (atomically (writeTVar var True))
  atomically (readTVar var >>= check)

GHC will throw a BlockedIndefinitelyOnSTM exception to the main thread here, when it seems like it really shouldn't.

[Gabriella439]

@mitchellwrosen: No, I don't know any more beyond that

[mitchellwrosen]

@mbwgh Here you go:

import Pipes
import Pipes.Concurrent
import qualified Pipes.Prelude as P
import Data.Monoid

main = do
  withBuffer unbounded
    (\output1 ->
      withBuffer unbounded
        (\output2 -> runEffect (P.stdinLn >-> toOutput (output1 <> output)))
        (\input2 -> runEffect (fromInput input2 >-> P.take 2 >-> P.stdoutLn)))
    (\input1 -> runEffect (fromInput input1 >-> P.take 2 >-> P.stdoutLn))

[mitchellwrosen]

@Gabriel439 I found the GHC ticket explaining the change in behavior beginning in 7.8.1: https://ghc.haskell.org/trac/ghc/ticket/7970

[mitchellwrosen]

There's also this at the very bottom of Control.Concurrent:

There is a subtle interaction between deadlock detection and finalizers (as created by newForeignPtr or the functions in System.Mem.Weak): if a thread is blocked waiting for a finalizer to run, then the thread will be considered deadlocked and sent an exception. So preferably don't do this, but if you have no alternative then it is possible to prevent the thread from being considered deadlocked by making a StablePtr pointing to it. Don't forget to release the StablePtr later with freeStablePtr.