Netflix / PigPen

Map-Reduce for Clojure
Apache License 2.0
565 stars 55 forks source link

Change UDF initialization to not rely on memoization #23

Closed mbossenbroek closed 10 years ago

mbossenbroek commented 10 years ago

Fixes #19

Prior to this change I was memoizing user functions based on what you pass to the pigpen operator. This has the unfortunate side effect of using stale versions of named functions in the REPL. This means that if you load foo, load the join, and then modify foo, it'll use the first version.

The reason for this is historical and for performance. I never want to re-eval the same code on the cluster and on the cluster you never change the code, hence the memoization. In the past, defining a function not-inline wasn't supported so this wasn't a problem.

This changes the user code to be evaluated only when the UDF is constructed.