mitchellwrosen / otp

[WIP] Pico-OTP-like supervisors in Haskell
3 stars 0 forks source link

Add usage example #5

Open chshersh opened 5 years ago

chshersh commented 5 years ago

You probably remember my post about haskell library with workers:

Currently we're using capataz but I keep looking for alternatives 🙂 It would be nice to use more documentation and usage examples with the otp library. Your approach with slave-thread looks promising!

mitchellwrosen commented 5 years ago

Absolutely... First step would be to continue building it out! It doesn't work yet :)

mitchellwrosen commented 5 years ago

@ChShersh I've been thinking about this today, it might be possible to get a decent experience using only slave-thread. The hierarchy of threads is actually quite nice, the more I think about it.

One problem with "supervisor" abstractions is how heavyweight they are, and how they force you to specify a bunch of configuration up front. Another problem is Haskell's shared memory tends to make porting Erlang idioms awkward and un-idiomatic because Haskell threads can communicate directly via MVar/TVar/etc, rather than by each others' mailboxes.

I'm curious what you are actually using capataz for, and if it can be approximated with just SlaveThread.fork.

Just as an example, from your Reddit thread,

Several additional worker threads that do some jobs on various time periods (cleanup old user sessions, cleanup old cache, do some DB synchronisation work, calculate and aggregate some monitoring metrics, etc.).

Could this be as simple as this?

fork cleanupUserSessionsWorker
fork cleanupCacheWorker
fork dbSyncWorker
fork aggregateMetricsWorker

And say the dbSyncWorker wants to fork workers of its own, but they often crash, and we just want to restart them instead of crash the whole thread hierarchy all the way up to main. It might become something like this instead:

fork $ do
  fork dbSyncWorker
  forever (threadDelay maxBound)
    `catch` \ex ->
      ... react to crash, possibly restart it all ...

This is essentially a supervisor - it forks a thread, then sits around waiting for it to die, and reacts to its death. Lots more functionality might be desirable, but the right API has been kind of tricky for me to discover. I wonder if the answer is to just keep it simple.

chshersh commented 5 years ago

@mitchellwrosen Thanks for working on this and making usage of libraries more convenient 🙂

Could this be as simple as this?

Yeah, that's basically it: I have main thread which is servant server and several additional workers that run in parallel like cleaning up old sessions, old cache and spawning asynchronous workers. My desire is to have an interface that allows to spawn such workers that won't cause main thread to crash in case of any exceptions in them and the workers themselves will be restarted on exception after logging this particular exception.