The article is of course methodology-focused. But as someone who's worked in multi-site intervention evaluation previously, I wonder if the results have the following policy implications (if measurability is not easily solvable by novel methodologies).:
It seems that achieving "provable" intervention results may mean "simplifying" or controlling them in a systematic way. For instance, one of the limitations discussed for the proposed procedure is a scenario where a site implements multiple but related interventions. E.g. indirect effects of educational attainment may confound indirect effects from using program's counseling. This may imply that, when designing our interventions, if the goal is to know what "works", we must resist potential political/financial pressures to do too much too fast and instead build up incrementally. Otherwise we may never know what works exactly and end up wasting resources on financing components that may or may not be helping participants. In other words, design programs whose effects are more readily measurable.
The article is of course methodology-focused. But as someone who's worked in multi-site intervention evaluation previously, I wonder if the results have the following policy implications (if measurability is not easily solvable by novel methodologies).:
It seems that achieving "provable" intervention results may mean "simplifying" or controlling them in a systematic way. For instance, one of the limitations discussed for the proposed procedure is a scenario where a site implements multiple but related interventions. E.g. indirect effects of educational attainment may confound indirect effects from using program's counseling. This may imply that, when designing our interventions, if the goal is to know what "works", we must resist potential political/financial pressures to do too much too fast and instead build up incrementally. Otherwise we may never know what works exactly and end up wasting resources on financing components that may or may not be helping participants. In other words, design programs whose effects are more readily measurable.