Ambiguous sendM / Parametric base monad

[zarybnicky]

I'm not really sure how to title this issue better, but anyway, I've been playing with polysemy and trying to make it work with Reflex, which is where I do most of my development nowadays.

I managed to succeed in writing a fixpoint effect (below), but writing an interpret requires either AllowAmbiguousTypes or a Proxy argument, which isn't the end of the world, though I expected more from the typechecker...

data Store s t (m :: * -> *) a where
  Get :: Store s t m (Dynamic t s)
  Put :: Event t s -> Store s t m ()

runStore ::
     forall m s t r a.
     ( Typeable s
     , Typeable t
     , Reflex t
     , MonadHold t m
     , Member Fixpoint r
     , Member (Lift m) r
     )
  => s
  -> Semantic (Store s t ': r) a
  -> Semantic r a

Where I ran into trouble was in the next step, when trying to simultaneously use the effect and the GUI elements from Reflex-DOM. Despite having only one (Lift m) effect member, I've needed to annotate all calls to sendM with @m. As before, this isn't completely unusable but it's pretty annoying.

widget :: forall t m. (Typeable t, MonadWidget t m) => m ()
widget = runM $ runFixpointM runM $ runStore @m @Text "" $ do
  x <- get @Text @t
  dynText x
  dText <- sendM @m $ textInput def
  eSave <- sendM @m $ button "Save"
  put (current (value dText) <@ eSave)

Trying a third time, I thought that I could write a Semantic instance for the necessary typeclasses so that I could use the functions directly, as all Reflex functions are parametric in the monad, but that attempt ended abruptly after running into:

instance (Reflex t, DomBuilder t m, Member (Lift m) r) => DomBuilder t (Semantic r)

-- Illegal instance declaration for ‘DomBuilder t (Semantic r)’
--         The liberal coverage condition fails in class ‘DomBuilder’
--           for functional dependency: ‘m -> t’
--         Reason: lhs type ‘Semantic r’ does not determine rhs type ‘t’
--         Un-determined variable: t

When writing up this issue, I thought of introducing a newtype wrapper for Semantic with another type parameter, say

newtype SemR t r a = SemR { unSemR :: forall m. Member (Lift m) r => Semantic r a }

but quickly trying that, the definition doesn't even typecheck due to Ambiguous use of effect 'Lift'.

By now I've long ran out of my type-level programming knowledge and, more importantly, the time I've set aside for exploring this, so while I'd love to continue, it'll have to wait some two weeks.

This issue is not meant to report a problem with the library, it is more of a request for advice, but I'd say it's closely related to #15 with problems with the base monad...

[isovector]

This is a really interesting use case. I don't know a lot about Reflex, but I'd love to see how the two get on.

Potential warning: I've found that my fixpoint implementation diverges a little harder than others'. My understanding is that Reflex is pretty mdo dependent, and if so, this might be the first thing to bite you when running the code. Any advice here would be appreciated.

Despite having only one (Lift m) effect member, I've needed to annotate all calls to sendM with @m

Yeah, this sucks. Unfortunately Haskell realizes there's nothing preventing you from wanting to run two reflexes in the same Semantic block, and that maybe you just forgot to add the other one to your context. This is definitely a pain point that I have some ideas to solve, but it's not going to happen in the next month. Maybe you can instead write a custom sendReflex = sendM with a type signature more amenable to type inference.

As for your instance

newtype SemR t r a = SemR { unSemR :: forall m. Member (Lift m) r => Semantic r a }

try instead moving the m out, as it is indeed completely ambiguous:

newtype SemR t m r a = SemR { unSemR :: Member (Lift m) r => Semantic r a }

This thing will definitely pass the fundep coverage condition. I'm not sure it will be more usable, but I guess it's worth a shot!

If there is indeed a canonical reflex monad you could lift a la #15, that seems to me like the best solution.

[zarybnicky]

Thanks for the response. Yes, fixpoint divergence was the first problem that I've encountered, but that 'somehow' disappeared once I've noticed the MonadFix instance and used RecursiveDo instead of manually sending a Fixpoint effect, so I forgot to mention that in the initial post. (I think it was because of a non-lazy destructuring, but I didn't check.)

As for a canonical Reflex monad, that's not really possible as there are at least two, sometimes more base monads (one for the browser and one for the server), plus Reflex is very MTL-heavy, so any pre-made library effect would change the monad. I'm pretty sure that there are at least twenty 'base' Reflex typeclasses...

I made some time for another batch of exploration and I did manage to get half a step forward. I'm not sure why I didn't try a newtype with an exposed m which is the obvious next step in retrospective, but that somewhat worked. I only needed to ditch makeSemantic and write my own effect functions as wrapping bare effects in the newtype had the same inference problems as above (SemR @m $ get @Text @t).

get :: forall s t r m. Member (Store s t) r => SemR t r m (Dynamic t s)
get = SemR $ send Get
{-# INLINABLE get #-}

Once I did that, I finally got to the desired state of being able to write Reflex instances for the newtype wrapper and the demo code looks pretty much the way I'd want it:

app :: forall t m r. (Member (Store Text t) r, MonadWidget t m) => SemR t r m ()
app = do
  dynText =<< get @Text
  dText <- inputElement def
  eSave <- button "Save"
  put (current (value dText) <@ eSave)

This is where I encountered the next blocker. The newtype I used in the end 1) doesn't play well with GeneralizedNewTypeDeriving (DerivingFunctor works though, but that's the only one), which is quite OK as I'd only need to write out the instances only once, but I also discovered that I need an instance of MonadTransControl which is where I got stuck this time.

newtype SemR t r m a = SemR
  { unSemR :: (Member Fixpoint r, Member (Lift m) r) => Semantic r a
  } deriving (Functor)

Can't make a derived instance of ‘Applicative (SemR t r m)’
         (even with cunning GeneralizedNewtypeDeriving):
         cannot eta-reduce the representation type enough
     • In the newtype declaration for ‘SemR’

I'm pretty sure that there is some clever combination of Tactical and unLift that would enable me to encode type StT t a :: *, the "The monadic state of a monad transformer [which] is the result type of its run function", but this is where I ran out of time for this week...

I'd love to continue, the results so far are quite pleasant, but I expect I'll be able to get to this only next weekend.

P.S.: Oh, and while searching for freer and MonadTransControl, I found https://mail.haskell.org/pipermail/haskell-cafe/2018-May.txt, where they mention a Members '[...] r => Effects r a synonym. I think that it might be worth adding that, esp. for bigger effect stacks...

[zarybnicky]

Well, nevermind. After sleeping on it, I removed the constraints from the newtype which enabled GeneralizedNewTypeDeriving to do some of it magic. I also discovered that the MonadTrans(Control) constraints are only on the default implementations of class methods which means that I was able to a bunch of instances like this one:

instance (Member (Lift m) r, PerformEvent t m) => PerformEvent t (SemR t r m) where
  type Performable (SemR t r m) = Performable m
  performEvent e = SemR $ sendM @m (performEvent e)
  performEvent_ e = SemR $ sendM @m (performEvent_ e)

Where I got stuck this time was when writing two instances that require an effect 'runner', where the signature is e.g. SemR t r m a -> Event t (SemR t r m b) -> SemR t r m (a, Event t b) (an effect and an event of effect updates). This requires unwrapping the effect and event (using fmap) and sending the underlying m a to the underlying monad's method. Below is what that looks like in a trivial effect implemented using the MTL method Reflex uses. (Also, wow, those type signatures in the rest of the file look terrible...)

https://github.com/reflex-frp/reflex/blob/767faa134dc57ac83445e89fc03f4a42d699cd53/src/Reflex/EventWriter/Base.hs#L149-L161

The magic underlying this specific function is not trivial...:

https://github.com/reflex-frp/reflex-dom/blob/3383d8628d43001d153d7a79dc6ca52dc2021682/reflex-dom-core/src/Reflex/Dom/Builder/Immediate.hs#L1448-L1542

A simpler function to start with might be element, which just takes a config object and an effect producing children of the element: Text -> ElementConfig er t (...) -> SemR t r m a -> SemR t r m (Element er (...) t, a). This one also requires a runner, which makes me think I need to start with a new effect, something closer to Resource rather than just adding instances for a newtype...

[zarybnicky]

@isovector Hmm, I have the uncomfortable feeling that I've finally reached the limits of polysemy capabilities...

I did manage to figure out element which contains just a single nested action - by creating a separate effect and running the effect using Tactics. The resulting code is ugly, but I have no reason to suspect it doesn't work.

Where I've had to stop is when handling the effect RunWithReplaceE :: r a -> Event t (r b) -> DomBuilderEff t m r (a, Event t b) = an action and an event stream of actions, where I need to return the result of the first action and the event stream of results of the actions.

The way this is done in Reflex internals is by manually threading the state from the first action through the stream of actions. An example from the class EventWriter looks like this (the state is returned as the second part of the tuple returned by runEventWriterT, from https://github.com/reflex-frp/reflex/blob/master/src/Reflex/EventWriter/Base.hs#L153-L161):

runWithReplaceEventWriterTWith :: forall m t w a b. (Reflex t, MonadHold t m, Semigroup w)
                               => (forall a' b'. m a' -> Event t (m b') -> EventWriterT t w m (a', Event t b'))
                               -> EventWriterT t w m a
                               -> Event t (EventWriterT t w m b)
                               -> EventWriterT t w m (a, Event t b)
runWithReplaceEventWriterTWith f a0 a' = do
  (result0, result') <- f (runEventWriterT a0) $ fmapCheap runEventWriterT a'
  tellEvent =<< switchHoldPromptOnly (snd result0) (fmapCheap snd result')
  return (fst result0, fmapCheap fst result')

And that's only the simplest of the three methods of the typeclass Adjustable, the other two thread state through an IntMap and through a DMap...

An event is just a Functor with a special push method where I'm able 'hold' an f a (MonadHold m => (a -> m b) -> Event a -> Event b), so in theory I think I know how to thread an f b through an event, but that doesn't help me to get from a (f a, Event t (f b)) to an f (a, Event t b), which is what I need to return in an interpretH.

I'm including the snippet where I've finished, I've tried to annotate it with some types so that it's at least somewhat readable. This doesn't compile due to two reasons - one is the f (...)/(f a, f ...) mismatch, the other is the fact that an Event is not a Traversable

runDomBuilder ::
     forall t m r a. (DomBuilder t m, Member (Lift m) r)
  => (forall x. Semantic r x -> m x)
  -> Semantic (DomBuilderEff t m ': r) a
  -> Semantic r a
runDomBuilder runner = interpretH $ \case
  ElementE e cfg child -> do
    c <- runT child
    sendM $ do
      (out, fa) <- element e cfg (runner .@ runDomBuilder $ c)
      pure $ (out,) <$> fa
  RunWithReplaceE (a :: m1 a1) (eb :: Event t (m1 b)) -> (do
    let runner' = runner .@ runDomBuilder :: Semantic (DomBuilderEff t m : r) z -> m z
    ra :: m (f a1) <- runner' <$> runT a
    reb :: Event t (m (f b1)) <- traverse (fmap runner' . runT) eb
    sendM @m $ (runWithReplace ra reb :: m (f a1, Event t (f b1)))
    ) :: Semantic (WithTactics (DomBuilderEff t m) f1 m1 r) (f1 (a1, Event t b1))

[zarybnicky]

Hmm, one option that just occurred to me is to use the MonadFix instance of the underlying monad, to use the first element of the tuple returned from runWithReplace, which would solve the problem with Event not being a Traversable - this would give me an f a and an Event t (f b), but I still don't have a way to combine them into f (a, Event t b).

[isovector]

Reflex is surprisingly hard to find documentation for --- I wanted to track down the definition of Event but after a few minutes gave up.

That being said, I'm reasonably sure this is doable. If you can find an instance of the Effect typeclass below for your type, it's definitely possible to connect into polysemy.

class (∀ m. Functor m => Functor (e m)) => Effect e where
  weave
      :: (Functor s, Functor m, Functor n)
      => s ()
      -> (∀ x. s (m x) -> n (s x))
      -> e m a
      -> e n (s a)

[zarybnicky]

Event is a newtype that not supposed to be destructured anywhere outside of the timeline implementation itself (the t parameter). There were even some attempts to backpack-ify the t parameter (https://github.com/ezyang/reflex-backpack), but right now, there are three possible definitions:

• SpiderTimeline, the default one: https://github.com/reflex-frp/reflex/blob/376cded87b6de2b883844ff7a6b720c3f9c408eb/src/Reflex/Spider/Internal.hs
• ProfiledTimeline wrapping Spider, for profiling: https://github.com/reflex-frp/reflex/blob/376cded87b6de2b883844ff7a6b720c3f9c408eb/src/Reflex/Profiled.hs
• Pure, 'given a(ny) moment in time, get the value': https://github.com/reflex-frp/reflex/blob/376cded87b6de2b883844ff7a6b720c3f9c408eb/src/Reflex/Pure.hs

I'll give the manual instantiation of Effect a try later, but I'm in the last week of my thesis (where I wanted to include a demonstration of effects as an alternative to mtl), so I'm very much out of time for this side-task.

[zarybnicky]

After two more hours at it, I failed again with the approach I outlined in my previous post (threading the state through Reflex-provided state capabilities). Reading through some more polysemy code though, I'm pretty confident that it is possible to interpret the DomBuilderEff effect using weave without any extra instances of Effect in a manner similar to Polysemy.Error or Polysemy.NonDet. I'm slightly apprehensive about diving into Yo and the like, but squinting at the types in runError, I think the types will lead me.

(I needed a break from spewing my thesis' text, so I took another stab at it. The things I do to procrastinate...)

[isovector]

If you want to set up a call and hash this out in real time, let me know!

[zarybnicky]

Thanks for the offer! I'll start by watching that talk of yours, even just the observation from Reddit comments that Sem ~ ReaderT of interpreter is quite useful, and just in general try to wrap my head around all the types - but only after I finish my thesis, no more procrastination doing useful things! :)

[isovector]

Despite having only one (Lift m) effect member, I've needed to annotate all calls to sendM with @m.

Give this a spin: polysemy-plugin --- it should automatically disambiguate most ambiguous uses of effects, aka dramatically improve type inference.