ENH/Design: support for spline-dummy interaction in penalized/GAM

[josef-pkt]

R mgcv has a by option for the interaction effect of penalized splines and a categorical variables.

I don't know yet how we will support this. patsy can create the interaction term using its splines.

two cases (that I can think of)

• spline has full support for each categorical level
• categorical level partitions the support of the explanatory variable in the spline, e.g. regression continuity design.

I think the former should be easy using standard interaction (like kronecker product for unbalanced dummies). The latter will be more difficult because the spline cannot be estimated in the unobserved part. I have a simple example without penalization in the causal impact, synthetic control PR #3647, or in one of the related experimental notebooks.

[josef-pkt]

(random thoughts)

using interaction effect with dummy directly, i.e. we create two variables z1 = dummy * x and z0 = (1 - dummy) * x

this means part of the new variable will be zeros. We cannot include those zeros when we create the spline basis, i.e. we would need to have a ignore_zeros option.

We can do it if we use the spline basis for the x, in that case we just compute a usual interaction variable on parts of exog.

The two cases above would need 2 options for knot selection when creating the spline basis

• common knots: impose similar spline structure or identical basis functions (although evaluated at different points) for all by categorical subsets.
• separate knots: choose knots separately as if the non-zero observations were the entire series represent different variables, e.g. each subset has separate restriction on boundary knots as for example the natural spline restrictions of zero second derivative at the end knots.
• variation on separate knots: we could choose separate spline basis but impose additional restrictions on the join or jump point, for example identical first and second derivatives.

The second two might be applicable to regression continuity design and structural change with jump points.

A corresponding option is on penalization or smoothing parameter choice

• common alpha/penalization weight: we impose equal penalization to avoid divergent smoothness across categorical variable interaction
• separate, independent alpha: subsets can have different smoothness, i.e. penalization adapts to subset.

mgcv allows the common alpha case in the options for GAM, i.e. add an index that maps from alpha list to the list of smooth terms.

related: varying coefficient models which are essentially also just interaction effects. I have not looked at those yet.

two possible example:

• growth curves by gender with common support in age variable
• structural break at known strict threshold value of the explanatory variable: e.g. intervention analysis (e.g. #3647 causal impact and synthetic control) or regression continuity design.

one alternative in the structural change, regression discontinuity case: We could add only a mean shift dummy. If we use a local spline basis like B-splines with drop one basis column for removing intercept, then the shape of the curves (including the mean shift variable) in the two segments would be mostly independent of each other, i.e. restricted to common penalization and spill over across knots in transition segment.

[josef-pkt]

just another example: cyclical time series, e.g.

• El Nino dataset with annual seasonal pattern, each year is a spline

• within day, e.g. hourly, data with weekly cyclical pattern, each day is a spline

• pooled model; every cycle is the same, i.e. every year or every day has same pattern this is currently possible

• separate model: e.g. estimate each year separately, or estimate data for all weeks for each weekday separately currently possible but no commonality across separate regression

• interaction model: each year or each weekday is a separate spline, and eg. parametric intercept or trend common to all sub splines. problems with zeros in dummy/period and spline interaction, we need spline bases that don't include the zero (not it interval) as a value, i.e. the spline basis at zero should be zero.

Our current implementation cannot construct spline bases ignoring some of the observations, e.g. the zero observation. Our current implementation cannot reuse and shift splines, e.g. with equal time interval the time variable is the same for all subunits and we could just shift and transpose it instead of constructing in each time separately (scipy.signal did and maybe still does that)

ideas

• add a mask to indicate which subset of observation the spline basis should be constructed for.
• reusable spline basis for common (affine transformed) exog:
  • construct canonical spline basis and project actual values like trend/period into that interval.