Open pdeffebach opened 3 years ago
Yeah, it would just hash as the same function.
But do you want to compile so much? It seems like it would make it a Symbolics expression and then use substitute
, you could get away without ever compiling just by interpreting the expression. If the runtime cost isn't a major factor then using interpreting could be the right approach.
This is great news. I will file an issue at DataFramesMeta to remind myself to add this eventually.
We definitely need to compile, unfortunately. DataFramesMeta is really just an easy way of making calls like transform(df, :x => f => :y)
. So we need f
to exist and be a function.
And yeah, runtime cost is a major factor since DataFrames has untyped columns. So we need function barriers for performance.
I just finished a big PR in DataFramesMeta to try and reduce latency.
Given an expression of the form
we make an anonymous function
This carries a compilation cost of creating the anonymous function. So my PR made an optimization where if it saw
f(:x, :y)
it would just returnf
, since it's the same as the anonymous function above.But more other expressions are still problematic. Consider
This will always have to turn into
for purposes of the
src => fun => dest
syntax in DataFrames.It looks like this package will let me cash
:(x -> x + 1)
so that the anonymous function of that form is only compiled once, and later calls are taken from a lookup. Is that correct?If so, DataFramesMeta seems like a good application of this package.