Our code generation for calls with a closure as argument is wrong. We don't check whether the enclosed values changed, but just return the memoized result regardless.
To Reproduce
package bug
pipeline bug {
val table = Table({
"a": [1, 2]
});
val threshold = table.numberOfRows - 2; // Change this, observe result
val result = table.filterRows((row) ->
row.getValue("a") as Int > threshold
);
}
Expected behavior
If an enclosed value changes, we should recalculate instead of looking up the result in the memoization table.
A quick fix would be that we don't memoize calls that have a lambda as an argument that references values outside of it.
Long term, we should just add those values as hidden arguments to the call, however. If the partial evaluator can already evaluate the reference, there's no need to add it as a hidden argument. We still need to add them to keep the order the same. Together with the generated Python code of the lambda, we must get something unique.
Describe the bug
Our code generation for calls with a closure as argument is wrong. We don't check whether the enclosed values changed, but just return the memoized result regardless.
To Reproduce
Expected behavior
If an enclosed value changes, we should recalculate instead of looking up the result in the memoization table.
A quick fix would be that we don't memoize calls that have a lambda as an argument that references values outside of it.
Long term, we should just add those values as hidden arguments to the call, however.
If the partial evaluator can already evaluate the reference, there's no need to add it as a hidden argument.We still need to add them to keep the order the same. Together with the generated Python code of the lambda, we must get something unique.Screenshots (optional)
No response
Additional Context (optional)
No response