Closed dscolby closed 2 months ago
It actually should work pretty easily. Since our significance test is non-parametric (randomization inference), we don't have to worry about any distributional assumptions. The only thing we need to make work is not assuming linearity. The current uses OLS, which assumes linearity. We need to replace the OLS estimation with an ELM.
We should actually change our approach here. The current methods assume that there might be a violation of this assumption because there might be multiple treatments or controls. For example, if there is one treatment group in the dataset but there were actually multiple treatments administered then this would be a violation. The method tries to generate what could be alternative treatment statuses by using Jenks breaks and comparing the model fit with the Jenks break treatment groups vs the one in the dataset. Besides not being able to handle continuous treatments, it also only accounts for one potential violation of the counterfactual consistency assumption. A simpler, more general, and probably better approach is to simulate potential outcomes that differ from the outcomes in the dataset and reestimate models with that data to see how much this changes the results.
Specifically incorporate accuracy because MSE doesn't work for classification.