For #2, the story is a bit more complicated: There are a number of operator functions in tabelcoth.column.api.operators that return scalars. For example a function like reduce-* or sum. We decided to lift these because they may be useful within aggregation expressions.
However, we realized that lifting them in that way doesn't help make the aggregation expressions more terse. For example, as we've lifted the functions in this PR, we can do something like this:
This expression could be more terse if we didn't need to supply the anonymous function to tc/aggregate. What we are considering doing, therefore, following @genmeblog 's suggestion is to re-lift these fns in another PR so that they can function as aggregator functions themselves. We won't however, do this here, but I wanted to provide this context to make it clear that the second signature here is still a WIP.
Other Changes in this PR
This PR also improves the utility functions that we use to do the lifting. In this PR, we add a new utility file: src/tablecloth/utils/codegen.clj that holds the key functions that we are using here to do lifting and code generation. The key fn there is do-lift that has now been changed to take a lift plan that looks like:
A key part of this lift plan is the lift-fn-lookup where we specify which lift functions to use for which fn symbols. That map now maps a set of function symbols to a map containing a :lift-fn key and :optional-args that will be passed to that fn:
This change came from a suggestion by @daslu pointing out that doing it this way is a bit more transparent. Before we were packing the :optional-args inside of functions so they were less available for the kind of analysis @daslu did to look at how we lift fns here.
Goal
This PR builds on the new column API by lifting most of the function in
tablecloth.column.api
so that they can now be called on thedataset
.There are two signatures that have been lifted here:
(dataset target-column columns-selector) => dataset
(shown above)(datasest columns-selector) => scalar
For
#1
, you can do this kind of thingFor
#2
, the story is a bit more complicated: There are a number of operator functions intabelcoth.column.api.operators
that return scalars. For example a function likereduce-*
orsum
. We decided to lift these because they may be useful within aggregation expressions.However, we realized that lifting them in that way doesn't help make the aggregation expressions more terse. For example, as we've lifted the functions in this PR, we can do something like this:
This expression could be more terse if we didn't need to supply the anonymous function to
tc/aggregate
. What we are considering doing, therefore, following @genmeblog 's suggestion is to re-lift these fns in another PR so that they can function as aggregator functions themselves. We won't however, do this here, but I wanted to provide this context to make it clear that the second signature here is still a WIP.Other Changes in this PR
This PR also improves the utility functions that we use to do the lifting. In this PR, we add a new utility file:
src/tablecloth/utils/codegen.clj
that holds the key functions that we are using here to do lifting and code generation. The key fn there isdo-lift
that has now been changed to take a lift plan that looks like:A key part of this lift plan is the
lift-fn-lookup
where we specify which lift functions to use for which fn symbols. That map now maps a set of function symbols to a map containing a:lift-fn
key and:optional-args
that will be passed to that fn:This change came from a suggestion by @daslu pointing out that doing it this way is a bit more transparent. Before we were packing the
:optional-args
inside of functions so they were less available for the kind of analysis @daslu did to look at how we lift fns here.