Closure Serialization Sub-Package?

[danielwaterworth]

The ability to serialize closures has use beyond remote. Would it be possible to move this functionality into a sub-package?

[jepst]

That's a good idea. What further uses you have in mind? CH assumes that you'll be executing closures in different runtimes, on different systems, with separate memories. If that's not the case, then the serialization mechanism CH uses is overkill.

Serializing closures is only useful if you can then un-serialize (i.e. run) them. The mechanism for doing this is carried in the ProcessM monad. So if I separated this aspect, you'd still need some data structure, probably a monad, to let you run the closures. And for that you can just use ProcessM, even if you aren't running distributed programs per se.

[danielwaterworth]

The use case I have in mind is a distributed database that is able to store thunks. So all of the assumptions that you make hold, but I don't require any of the other functionality from remote.

EDIT: Have you considered pulling de-serialization out of a monad as it should be referentially transparent.

[jepst]

So your distributed database would not use messages to communicate?

Closure execution needs to be monadic if the underlying closure is monadic; since CH functions are usually in the ProcessM monad, their deserialization must also be in the ProcessM monad. Pure deserialization would work only if you want to call only pure closures.

[danielwaterworth]

No, there's no direct messaging between nodes. All communication happens between nodes and key-value stores.

The execution of the closure may need to be monadic, but the de-serialization of it doesn't.

[jepst]

That's a good point. Maybe that functionality should be separated. I will see what I can do, time permitting.

[danielwaterworth]

Thanks