pat-alt
commented
8 months ago great package! I just went through the whole documentation :D.
Thank you, that's some heavy reading 😆
Bottom line, is there any progress on this direction? a working branch?
There isn't currently any progress in this direction, unfortunately. While I initially defined the likelihood field of the data struct with regression at the back of my mind, the package has since been tailored towards classification. This also largely reflects the state of the literature on this topic: I have seen very little work on regression and I personally haven't thoroughly explored counterfactual regression in my own research either.
That being said, this has always been on the bucket list for this package, so thanks for bringing this to our attention again (I'll link a branch to this issue).
or if not any advice on how to handle this scenario? I would really appreciate any advice on this regard.
As for potential solutions, I can see two directions.
1. Ad-hoc thresholding
The most ad-hoc approach in this context is thresholding, which essentially discretizes the counterfactual regression search problem: instead of defining a target label as in classification, you could define some upper/lower bound for the continuous outcome in regression. That approach is most compatible with the current state of this package, but I've never tried it and there are certain caveats such as high sensitivity to the choice of the threshold as this figure taken from Spooner et. al (2021) demonstrates (threshold $\varepsilon_1$ yields counterfactuals $x_1$ while $\varepsilon_2$ yields the vastly different counterfactual $x_2$):
2. Rigorous approach
The linked paper by Spooner et. al proposes a more rigorous approach, which would require considerable effort to implement here. I'm looping in @jiahao for info, who is not only an co-author of the paper, but also a Julia OG dev.
I personally will not have any capacity to explore this further in the near future due to other ongoing PhD commitments. Contributions are, of course, very much welcome.
I'll add a task list based on the above:
- [ ] #391
- [ ] #392
Hi, great package! I just went through the whole documentation :D. And here it kinda looks like making this package to work with linear regression models from flux would require to undo the
logitfunction, so that the output from the last layer is the raw output? However, in the pre-processing step while callingCounterfactualDatawill automatically one-hot encode the target variable.Bottom line, is there any progress on this direction? a working branch? or if not any advice on how to handle this scenario? I would really appreciate any advice on this regard.