Attempt 1 memoized every UDF creation, but this led to a sticky REPL situation. For example: define function foo, use it in a pig/map command, and you can never redefine it because the memoization looks for the symbol foo - not the contents.
Attempt 2 didn't do any memoization, but this led to poor performance (every piece of data re-eval'd the UDF code) and eventually ran out of permgen.
Hopefully, attempt 3 will be just right. I added a udf-scope symbol that is passed to the memoized function that creates the UDF and evals user code. This is re-created every time we evaluate a pigpen expression. Unfortunately, I couldn't thread it through graph->local as that would break a lot of existing code that doesn't live here, so I created a dynamic var instead. This should be cleaned up when other breaking changes are made to this code.
Third try is the charm :)
Attempt 1 memoized every UDF creation, but this led to a sticky REPL situation. For example: define function
foo, use it in apig/mapcommand, and you can never redefine it because the memoization looks for the symbolfoo- not the contents.Attempt 2 didn't do any memoization, but this led to poor performance (every piece of data re-eval'd the UDF code) and eventually ran out of permgen.
Hopefully, attempt 3 will be just right. I added a udf-scope symbol that is passed to the memoized function that creates the UDF and evals user code. This is re-created every time we evaluate a pigpen expression. Unfortunately, I couldn't thread it through graph->local as that would break a lot of existing code that doesn't live here, so I created a dynamic var instead. This should be cleaned up when other breaking changes are made to this code.