Modify shelly to support lifted operations

[jwiegley]

Hi Greg!

Here is the very first cut. I'm submitting this as a pull request simply to begin a conversation on the best direction(s) for the code to go in. I intentionally have not finished it yet, because at this point I just wanted to prove that the concept works, and to get your feedback.

Here is what I have working now:

• Using lifted-base and lifted-async, you can freely use async, try, catch, etc., within Sh blocks. This does not require importing Shelly.Lifted, but works with existing code. This eliminates the need for asyncSh and the other _sh variants of the exception handling functions.
• If you import Shelly.Lifted, all the higher-order functions (like sub, withTmpDir, etc.) can work within any of the standard transformers (MaybeT, StateT, etc).
• There is a liftSh function, for lifting functions written in Sh rather than abstracted over MonadSh.

What's left to do:

• Provide re-exports in Shelly.Lifted for every function in Shelly, abstracting them over MonadSh. The goal here is to avoid the need to use liftSh with any of Shelly's own functions. The real purpose of liftSh is to allow users to call their own functions written in Sh.
• Finish writing the MonadShControl instances. The need for these instances is truly a hideous thing, but users will never see it, and it gives us great flexibility. Basically, we are using MonadBaseControl IO to allow lifted-base and lifted-async to work, and MonadShControl to allow things like the newly abstracted sub to work. I had to fork my own Shelly-specific brand of monad-control to do this, because you can't have two bases (lifted-async needs to see IO as the base, and sub needs to see Sh as the base).
• Mark the _sh exception handling variants as deprecated.
• Write more tests.

There is only about 3 hours more work left to do, but I wanted you to have a chance to review what I've done so far, to see if the approach is even acceptable to you.

[gregwebs]

Thanks, this looks great! Although I can't say I understand all the control stuff. For testing we could try replacing import Shelly with import Shelly.Lifted in the existing tests.

[jwiegley]

@gregwebs Everything looks done for this first cut. I finished the remaining todo items:

• Shelly.Lifted now re-exports every function that Shelly exports, providing lifted variants wherever possible.
• Finished the MonadShControl instances. Sometime you and I should get on Skype or Hangouts, and I can go into depth about what these are doing.
• The _sh variants of the exception handling functions now just use lifted-base and enclosed-exceptions (for safety with regard to async exceptions), and are marked deprecated with a pointer to those functions.
• I changed all of the tests to import Shelly.Lifted instead of Shelly, and everything seems to be working great. I did not add this change to the pull request, however.

I actually want to use this functionality in a tool I'm writing, where I want to use a ReaderT r Sh a, so porting my code is the next step for me.

It's still unfortunate to have a doubly-nested ReaderT, just to carry around things like command-line option values. But I can't think of anything at the moment which doesn't introduce a lot more complexity. After all, most of the time spent in Shelly will be executing processes, rather than the extra overhead of a nested Reader.

[gregwebs]

After all, most of the time spent in Shelly will be executing processes, rather than the extra overhead of a nested Reader.

yep, I have never heard a complaint about speed or CPU usage, just memory.

[gregwebs]

Looks great, thank you! I will wait for word on how porting your code goes.

I think we should create a flag for the test suite that will change the import with CPP so that both can be tested.

[jwiegley]

Ok, I will both port my code and conditionalize the tests today, and then we'll see how it goes. I think the Lifted layer is so "zero logic" that unless I've gotten some of the instances wrong, it should be pretty solid.

[gregwebs]

yeah, that is why I don't think we need to automatically run it every time by default, but just have it as a flag.

[jwiegley]

@gregwebs I've ported my application to using this type:

type App a = ReaderT Options Sh a

It worked like a charm! I didn't have to change any code except to remove all the Options arguments in my functions. This did reveal one minor bug, which I've fixed above.

[gregwebs]

When compiling this, I get:

src/Shelly/Base.hs:77:10:
    Not in scope: type constructor or class `Catch.MonadThrow'

Is there a version bound missing?

[jwiegley]

Very possibly. I'll try to get that fixed tonight. In fact, I need to add lower and upper version bounds for all the new dependencies.