not working with DataFramesMeta @by

[Lincoln-Hannah]

using DataFrames, DataFramesMeta, Dierckx, Interpolations

d = DataFrame( G = [1,1,1,1,1,2,2,2,2,2],  x = 1:10,   y = 2*(1:10) )

@by d :G    :spline = Dierckx.Spline1D(:x,:y)                       #works
@by d :G    :spline = Interpolations.linear_interpolation(:x,:y)    #fails

[mkitti]

Does this do what you want?

julia> dierckx_out = @by d :G    :spline = Dierckx.Spline1D(:x,:y)
2×2 DataFrame
 Row │ G      spline
     │ Int64  Spline1D
─────┼──────────────────────────────────────────
   1 │     1  Spline1D(knots=[1.0, 3.0, 5.0], …
   2 │     2  Spline1D(knots=[6.0, 8.0, 10.0],…

julia> interpolations_out = @by d :G    :spline = [Interpolations.linear_interpolation(:x,:y)]    # works
2×2 DataFrame
 Row │ G      spline
     │ Int64  Extrapol…
─────┼──────────────────────────────────────────
   1 │     1  5-element extrapolate(interpolat…
   2 │     2  5-element extrapolate(interpolat…

julia> dierckx_out[!, :spline][1](3.5)
7.000000000000001

julia> interpolations_out[!, :spline][1](3.5)
7.0

The problem is that the Interpolations.Extrapolation type is an AbstractArray so it is dispatching append! as an iterable AbstractArray. My sense is that we probably want push! rather than append! here, but I'm not really sure about other potential applications.

julia> interpolations_out[!, :spline][1] isa AbstractArray
true

[Lincoln-Hannah]

ok, so put it in [] thanks. No sure re push! vs addend!

[mkitti]

Also, do you really need the extrapolation here? Extrapolation is quite costly as I explained on Discourse because it introduces a branch that makes SIMD vectorization difficult. If you insist on using the convenience constructor, consider grabbing just the interpolation part in the following way.

julia> interpolations_out = @by d :G    :spline = [Interpolations.linear_interpolation(:x,:y).itp] 
2×2 DataFrame
 Row │ G      spline
     │ Int64  GriddedI…
─────┼──────────────────────────────────────────
   1 │     1  5-element interpolate((view(::Ve…
   2 │     2  5-element interpolate((view(::Ve…