JuliaData / DataFramesMeta.jl

Metaprogramming tools for DataFrames
https://juliadata.github.io/DataFramesMeta.jl/stable/
Other
481 stars 55 forks source link

inplace macros #215

Closed FuZhiyu closed 3 years ago

FuZhiyu commented 3 years ago

Probably it has been discussed somewhere but didn't find it in issues. Is there any plan to implement the macro counterpart for transform! and select!?

It seems pretty straightforward, simply replacing transform with transform!:

function transform!_helper(x, args...)
    t = (fun_to_vec(arg) for arg in args)
    quote
        $DataFrames.transform!($x, $(t...))
    end
end

macro transform!(x, args...)
    esc(transform!_helper(x, args...))
end

It would come in pretty handy especially when chaining operations on large datasets, but I'm not sure whether it fits in the design of this package.

pdeffebach commented 3 years ago

yes! Very straightforward, and planned!

pdeffebach commented 3 years ago

A PR would be welcome!

FuZhiyu commented 3 years ago

After making the PR I found out the old discussion #110 and your PR #119 .

It seems that PR was made with old infrastructure? With the current API the code above should do the trick, if I didn't miss anything.

FuZhiyu commented 3 years ago

@pdeffebach In addition to these, what's people's opinion on @where! and @orderby!? I personally find them very handy but don't know whether there are concerns of too many verbs confusing users.

@where! can be implemented pretty easily using delete!:

function where!(df::AbstractDataFrame, @nospecialize(args...))
    res = DataFrames.select(df, args...; copycols = false)
    notkeep = .!df_to_bool(res)
    delete!(df, notkeep)
end

function where!(gd::GroupedDataFrame, @nospecialize(args...))
    res = DataFrames.select(gd, args...; copycols = false, keepkeys = false)
    notkeep = .!df_to_bool(res)
    delete!(parent(gd), notkeep)
end 

I guess we could also do the same using sort! for @orderby!.

I'm happy to start another PR if needed.

bkamins commented 3 years ago

What you propose will be soon added to DataFrames.jl in this PR https://github.com/JuliaData/DataFrames.jl/pull/2496 (also there you can check out the implementation and how it differs from your implementation).

pdeffebach commented 3 years ago

Closed via #216 (we will add @subset!) as a replacement for @where! later. @orderby! can be left for a later PR, possibly with a different name if DataFrames adds something similar.