Closed kzollove closed 1 year ago
While standardizeStaged()
does need to change a bit, the simplest solution turns out to be:
1) Adding optional field to the attr_spec, stage_transform
. This field can be used to pass R Code that alters the entire staged table. Its ideal use case is for filtering on one or more columns.
2) Handling the stage_transform
commands in the standardizeStaged()
function
Newest iteration of this solution is:
Newest iteration of this solution is:
depends on #104 and #103
Ready to be finished.
Operations that standardize R code, (included in the attr_spec JSON entry in a record of backbone.variable_source), may employ a filtering operation to extract an attribute from table:
"value_source_value": {"type": "code", "value": "dplyr::pull(dplyr::filter(staged, `Defining Parameter` =='CO'),'AQI')"}
The outer function
dplyr::pull(x, 'AQI')
is pulling the values from the AQI column. The inner functiondplyr::filter(staged, `Defining Parameter` =='CO')
defines the subset of the staged table to be used (i.e. only rows where Defining Parameter column has the value CO). The result of this function ends up in the attr_X columnvalue_source_value
The issue is in the way that
standardizeStaged()
is currently written:where the second line of the function defines number of rows in the resulting
stagedResult
table. Trying to "assign" a new column as is done instagedResult[rCodeRules$t_name[i]] <- eval(parse(text=rCodeRules$t_value[i]))
, simply doesn't work if its not the same number of rows/entries asstagedResult
.There are two possible solutions I see:
standardizeStaged()
so that instead of "assigning" columns that were sourced using R Code (i.e. "type": "code"), they are effectively "Right Joined" so that the number of rows in thestagedResult
table matches the number of entries in the subset of the staged table."type": "filter"
is used to signify a piece of filtering R code that would require the "Right join" as highlighted above. This seems to be changing to mechanisms to fix one problem. The only reason this would be preferable is if there is some instance where we specifically do not want the"type":"code"
option to cause a join.Do either of these solutions propose a foreseeable issue? Should a different approach be considered?