Generalize "apply" to allow heterogeneous dependencies

[sheaf]

This is my first attempt at using shake, but I'd like to be able to declare dependencies on multiple different types in apply. Something like the following (assuming ShakeValue is rewritten to be a class newtype instead of a type synonym so that it can be partially applied):

infixr 5 `Cons`
data HList ( c :: Type -> Constraint ) ( as :: [ Type ] ) where
  Nil  :: HList c '[]
  Cons :: c a => a -> HList c as -> HList c ( a ': as )

mapNewKey :: HList ShakeValue keys -> [ Key ]
mapNewKey Nil             = []
mapNewKey ( k `Cons` ks ) = newKey k : mapNewKey ks

class MapFromValue ( vs :: [ Type ] ) where
  mapFromValue :: [ Value ] -> HList Typeable vs
instance MapFromValue '[] where
  mapFromValue _ = Nil
instance ( Typeable v, MapFromValue vs ) => MapFromValue ( v ': vs ) where
  mapFromValue ( k : ks ) = fromValue @v k `Cons` mapFromValue @vs ks
  mapFromValue []         = error "mapFromValue: unexpected empty list"

type family MapRuleResult ( ks :: [ Type ] ) :: [ Type ] where
  MapRuleResult '[]         = '[]
  MapRuleResult ( k ': ks ) = RuleResult k ': MapRuleResult ks

applyHetero
  :: forall ( ks :: [ Type ] )
  .  ( Partial, MapFromValue ( MapRuleResult ks ) )
  => HList ShakeValue ks
  -> Action ( HList Typeable ( MapRuleResult ks ) )
applyHetero Nil = pure Nil
applyHetero ks
  = fmap ( mapFromValue @( MapRuleResult ks ) )
  . Action
  $ stepRAW ( callStackFull, mapNewKey ks )

Does this make sense?

[ndmitchell]

Thanks for the code. Before figuring out if that's a good way of encoding it, can you explain why you want to call apply on multiple distinct dependencies? There is parallel which lets you run many dependencies in parallel (same type or different) and apply operations combined with Applicative (e.g liftA2 (,) (apply as) (apply bs)) will get applied in parallel. My experience is that wanting heterogeneous apply happens somewhat rarely, and when it does, one of the two solutions is usually good enough. But it's certainly possible to do more if there is a need.

[sheaf]

OK, I guess you meant par :: Action a -> Action b -> Action (a, b) to allow different types.

I was trying to figure out if I could use shake to handle the dependency management of a Vulkan application. In that case, I have various types (from the Vulkan library) like Swapchain, Framebuffer, Vector CommandBuffer, Vector DescriptorSet with various dependencies between them. Then, for instance, when the swapchain goes out of date, everything that depends on it gets rebuilt. That was my use case, which involves handling things of many different types.
When I looked at the internal implementation of apply I saw that it was already handling heterogeneous types so I thought it might be worth exposing that, but as I said I have no real experience using the library.

[ndmitchell]

Using Shake for dependencies of Vulkan might be a very interesting use case - and I suspect it's relatively latency sensitive? That might make approaches like par less appealing, since they do spin up (cheap) threads to do their work. Using applicative notation gives you the parallelism with a single direct call to the underlying apply, which would be the route I would suggest, as its gives you most type safety. Defining some more specific vector/tuple application on top of the applicative notation shouldn't be too hard, although I'm not sure what would work best for your application.

The final option would be to define a type along the lines:

newtype ApplyKey = forall key value . RuleResult key ~ value, ShakeValue key, Apply key

And then you can call apply with the type ApplyKey, and provided you defined all the right rules, and probably an ApplyValue, could extract everything out at the end. But it would be somewhat un-type-safe - it wouldn't catch many errors at compile time.