thomaswiemann / ddml

ddml: Double/Debiased Machine Learning in R
https://thomaswiemann.com/ddml/
GNU General Public License v3.0
15 stars 1 forks source link

Reformulate the stacking optimization problem #7

Closed thomaswiemann closed 3 years ago

thomaswiemann commented 3 years ago

The formulation of the stacking optimization problem for the MSPE minimizing ensemble weights in ensemble_weights is unlikely to be computationally efficient. Consider an alternative approach, for example, via non-negative least squares.

thomaswiemann commented 3 years ago

The package causalDML also implements an ensemble procedure (see here). Weights are solved for using the nnls package. This seems like a good approach.

thomaswiemann commented 3 years ago

To further simplify, one could compute the ensemble weights without any constraints (i.e., "weights" only in name then, as potentially negative and mustn't sum to one) via least squares.

As an alternative to the inital implementation for the fully constraint set of weights (i.e., all weights in [0, 1] and sum(weights) = 1), use a quadratic programming solver such as gurobi.