How do you traverse a directory (for example)?

[mbwgh]

Given that this library is all about using the "with idiom", the lack of an actual bracket function that works with Stream makes you wonder how you should use all this. From what I could gather by looking at the issue trackers, there were MonadResource and MonadMask instances in the past, but they have been removed. Since the documentation is not up-to-date everywhere, I have no idea how to proceed.

For instance, this is how I got a simple directory traversal to work. Trigger warning.

module Cancer where

import           Control.Exception
import           Data.IORef
import qualified Data.Streaming.Filesystem as FS -- streaming-commons
import           Streaming
import qualified Streaming.Prelude         as S

streamDir :: FilePath -> Stream (Of FilePath) IO ()
streamDir dir = do
  sRef <- lift $ newIORef (mempty :: Stream (Of FilePath) IO ())
  let go ds = do
        mfp <- FS.readDirStream ds
        case mfp of
          Nothing -> pure ()
          Just fp -> do
            modifyIORef' sRef (S.cons fp)
            go ds
  lift $ bracket (FS.openDirStream dir) FS.closeDirStream go
  str <- lift $ readIORef sRef
  str

This is obviously not how to do it. But since there is no MonadMask instance, the types of both the vanilla bracket function, and the one from exceptions, are kind of "in the way". Should I reconsider the type of this kind of function? Does this functionality already exist? Did I miss some important piece of documentation?

[ivan-m]

I've been bitten by this as well.

What I think you want is withFiles :: FilePath -> (FilePath -> IO ()) -> IO (), where you use an unfoldrM-style function to keep applying the continuation to the result of readDirStream.

The next layer up would then be to have the continuation function build a Stream and apply a new continuation function to that.

[danidiaz]

Would the following function be a suitable solution?

withStreamedDir :: FilePath -> (forall r. Stream (Of FilePath) IO r -> IO r) -> IO r

To support recursive traversals, withStreamedDir would have to keep something like a list of opened handles and ensure that they are always called on exit.

[danidiaz]

I have uploaded a small helper package to Hackage called streaming-bracketed which I believe can help with the directory traversal problem.

For example, here's a function (taken from the testsuite) that performs directory traversal:

traverseDirectory :: FilePath -> R.Bracketed FilePath ()
traverseDirectory dir = do
  let maybeToEither = maybe (Right ()) Left
      stream        = R.bracketed
        (FS.openDirStream dir)
        FS.closeDirStream
        (\dirStream ->
          S.untilRight $ maybeToEither <$> FS.readDirStream dirStream
        )
  R.for
    stream
    (\filename -> do
      let filepath = dir </> filename
      R.clear (S.yield filepath)
      filetype <- liftIO $ FS.getFileType filepath
      case filetype of
        FS.FTDirectory -> traverseDirectory filepath
        _              -> pure ()
    )

A value of type Bracketed can be run by supplying a continuation that consumes its inner Stream:

R.with (traverseDirectory testDir) S.toList