Closed emlyn closed 7 years ago
The :eval
backend is significantly faster than the :base
backend. In practice, I'd suggest that you'd want to serialize the non-compiled FSMs, and then compile them locally using :eval
. Is there a reason this wouldn't work for you?
It's simpler to do the compilation once and serialize that, but if it's at a significant cost to performance I can also do it on the executors. I just have to memoize the compilation to avoid recompiling for every task the executor runs. In that case I'd just need the first of the two commits in this PR.
I've made the change to compile the FSM on the executors (with memoisation), and that seems to work fine. So I've removed the unneeded commit from this PR, and left just the one making uncompiled FSMs serializable. Would you be willing to merge this?
Merged, thanks for the contribution.
I'm using automat with Apache Spark. In order to avoid repeatedly recompiling my FSM, I compile it on the driver then send the compiled FSM to the executors, which requires them to be serializable.
It would be great if you are willing to merge this, so that I (and anyone else in a similar situation) don't have to maintain a separate fork.
I was only able to get
:base
compiled automatons to work, but not:eval
, because deserialization requires the class to be available in the jar, and the eval one doesn't generate a class file when aot compiled.This works as long as I specify
:backend :base
when compiling the FSM, but I was wondering what are the consequences of this? Presumably the the:eval
backend is some sort of optimisation? What are the differences between the backends?