jepst / CloudHaskell

A distributed computing framework for Haskell
http://hackage.haskell.org/package/remote
BSD 3-Clause "New" or "Revised" License
347 stars 22 forks source link

Spawned Processes can't spawn another process. #1

Closed molysgaard closed 13 years ago

molysgaard commented 13 years ago

Right now spawned processes can't spawn new proceses. Is this something that will be possible when the Static is implemented or is it something I'm doing wrong? Here's an example:

{-# LANGUAGE TemplateHaskell,BangPatterns,PatternGuards,DeriveDataTypeable #-}
module RecSpawn where

import Remote.Call
import Remote.Channel
import Remote.Peer
import Remote.Process
import Remote.Init
import Remote.Encoding

import System.Random(randomRIO)
import Control.Monad.Trans(liftIO)

spawner :: ProcessM ()
spawner = do
  thisNode <- getSelfNode -- This could also be a remote node
  flag <- liftIO $ randomRIO (0,1) :: ProcessM Int
  case flag of
    0 -> spawn thisNode (spawner__closure)
    1 -> say "Got one"

remotable ['spawner]

initial _ = do spawnLocal spawner
               return ()

main = remoteInit Nothing [RecSpawn.__remoteCallMetaData] initial
molysgaard commented 13 years ago

Updated the example to be more clear.

jepst commented 13 years ago

This is a really good question. Spawned processes can definitely spawn other processes, but not like this. The problem is that you can only use mkClosure after the corresponding remotable, which means that a function cannot directly spawn a function defined in the same module. The usual solution is to break the program into several modules, but that clearly doesn't work when a function needs to spawn itself.

My first answer was to try to pass spawner its own closure, like this:

spawner :: Closure (ProcessM ()) -> ProcessM ()
spawner selfClosure = do ... 
                         spawn thisNode selfClosure 
                         ...

initial _ = do let theclo = $(mkClosure 'spawner) theclo
               spawnLocal (spawner theclo)

This compiles, but unfortunately relies on serializing an infinite data structure. (It should be possible to serialize infinite structure using StableNames, but Data.Binary doesn't currently do this and I don't know of another library that does.)

Ultimately, the only way I know of for a function to spawn itself is to bypass the automatic TH closure-generator mechanism and construct its own closure from a string, like this:

{-# LANGUAGE TemplateHaskell,BangPatterns,PatternGuards,DeriveDataTypeable #-}
module Main where

import Remote
import Remote.Call

import System.Random(randomRIO)
import Control.Monad.Trans(liftIO)

spawner :: ProcessM ()
spawner = do
  thisNode <- getSelfNode -- This could also be a remote node
  flag <- liftIO $ randomRIO (0,1) :: ProcessM Int
  case flag of
    0 -> do selfClosure <- makeClosure "Main.spawner__impl" ()                           
            say "Get zero" >> spawn thisNode (selfClosure) >> return ()
    1 -> say "Got one"

remotable ['spawner]

initial _ = do spawnLocal spawner
               return ()

main = remoteInit Nothing [Main.__remoteCallMetaData] initial

I'm aware that this is ugly and inflexible: you lose type checking on the closure, and you're stuck referring to an opaque and implementation-dependent function name as a string.

jepst commented 13 years ago

Actually, just thought of a more elegant solution to this problem, but it will be a few weeks before I have to implement it. Email me if you'd like details.

jepst commented 13 years ago

The new push contains a function named mkClosureRec that does what you need. You can now call remote functions recursively through closures like this:

{-# LANGUAGE TemplateHaskell,BangPatterns,PatternGuards,DeriveDataTypeable #-}
module Main where

import Remote
import Remote.Call

import System.Random(randomRIO)
import Control.Monad.Trans(liftIO)

spawner :: ProcessM ()
spawner = do
  thisNode <- getSelfNode
  flag <- liftIO $ randomRIO (0,1) :: ProcessM Int
  case flag of
    0 -> say "Get zero" >> spawn thisNode  $( mkClosureRec 'spawner )  >> return ()
    1 -> say "Got one"

remotable ['spawner ]

initial _ = do spawnLocal spawner
               return ()

main = remoteInit Nothing [Main.__remoteCallMetaData] initial