search

gregwebs / Shelly.hs

Haskell shell scripting
BSD 3-Clause "New" or "Revised" License
415 stars 88 forks source link

Efficient writing of stdout to a file. #18

Open gregwebs opened 12 years ago

gregwebs commented 12 years ago

The other Haskell shell libs do connect file descriptors directly.

User want to run code like this without eating up RAM:

shelly $ run "cat" ["myfile"] >>= writefile "outfile"

To implement this right now you can run a shell command directly right now. Is this approach lacking for you because you will have to append strings together?

shelly $ escaping False $ run "cat myfile > outfile" []

Another approach besides directly hooking up file descriptors would be to provide a convenience function over runFoldLines possibly including something that makes a Conduit.

gregwebs commented 12 years ago

Some explanation of efficient piping, probably similar to how HSH works: http://book.realworldhaskell.org/read/systems-programming-in-haskell.html

see the source of: http://hackage.haskell.org/packages/archive/hsshellscript/3.1.0/doc/html/HsShellScript.html#g:11

I personally do not yet need these more efficient capabilities. Anyone want to try these options, look at the source a bit, and recommend an implementation?

ghost commented 10 years ago

How about wrapping the following into a function?

System.IO.withFile "outfile" System.IO.WriteMode
  (\outHandle ->
    shelly $ runHandle "cat" ["myfile"]
      (\inHandle -> liftIO $ Data.ByteString.Lazy.hGetContents inHandle >>=
                             Data.ByteString.Lazy.hPut outHandle)
  )

Didn't test it yet, but I'm guessing it shouldn't eat up all your RAM.

gregwebs commented 10 years ago

yeah, such an approach is better than the current system. It is hard to wrap this up in an elegant way though because runt already implies not using runHandle. We would like for the user to just be able to use combinators: run "cat" ["myfile"]redirectTo"outfile" rather than have to create a new run function.