Update: current version of the package

[jialul]

Hi @kosukeimai and @MichaelLLi, there is the current iteration of the package. I have cleaned up the original code Lily and I wrote and wrapped everything up into the evallITR package. The package now allows users to choose from a list of machine learning algorithms (Causal Forest, BART, Lasso, Boosted tree, Random Forest, Bagging, CART) and estimate PAPE, PAPEp, PAPD and AUPEC. It also incorporates a plot function and allows users to visualize AUPEC. Here is a demo of the new functions using the STAR dataset.

If your time allows, could you help to take a look at the current iteration and probably we can have a meeting with @xiaolong-y to discuss how to move forward with the next step?

[xiaolong-y]

Hi @jialul, I ran into a bug when running itr_run with bart that shows object 'training_data_elements_bart' not found, and it seems like the package will call function from bartMachine instead of bartCause even when we specify algorithms = c("bart"). Does this also happen on your end?

[jialul]

Hi @xiaolong-y, thanks for bringing this up. I have fixed this issue and also cleaned up the documentation (the "XXX" issue you mentioned) in a recent commit. If you pull from causal-ml, you can get the latest version of the code. Also, I wonder if you could create a new branch other than master and causal-ml and upload your code there? This would make it easier to compare across branches and merge them later.

[xiaolong-y]

Thank you for your response @jialul! Yes I am working on a new sub-branch causal-ml-gates that is based on causal-ml. https://github.com/MichaelLLi/evalITR/tree/causal-ml-gates

[xiaolong-y]

@jialul Kennedy 2020 seems to be an interesting paper that proposes a new meta-learner: DR-learner. The package that implements that is npcausal. Shall we look into this and see if we want to incorporate this to our package?

[kosukeimai]

Yeah, it would be good to incorporate these too!

[jialul]

Thanks for sharing it Xialong! Let's give it a try. Chernozhuko's team also has their R package for double-ML https://docs.doubleml.org/stable/index.html and a nice example of 401k https://docs.doubleml.org/stable/examples/R_double_ml_pension.html on financial wealth. I am checking whether we can use it as another example.

Best, Jialu

On Tue, Apr 25, 2023 at 9:34 PM Kosuke Imai @.***> wrote:

Yeah, it would be good to incorporate these too!

— Reply to this email directly, view it on GitHub https://github.com/MichaelLLi/evalITR/pull/6#issuecomment-1522636017, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKZHFPJNQAT2YVKN7SWEBLXDB3R7ANCNFSM6AAAAAAQFPL5B4 . You are receiving this because you were mentioned.Message ID: @.***>

[xiaolong-y]

Thanks for sharing it Xialong! Let's give it a try. Chernozhuko's team also has their R package for double-ML https://docs.doubleml.org/stable/index.html and a nice example of 401k https://docs.doubleml.org/stable/examples/R_double_ml_pension.html on financial wealth. I am checking whether we can use it as another example. Best, Jialu … On Tue, Apr 25, 2023 at 9:34 PM Kosuke Imai @.> wrote: Yeah, it would be good to incorporate these too! — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKZHFPJNQAT2YVKN7SWEBLXDB3R7ANCNFSM6AAAAAAQFPL5B4 . You are receiving this because you were mentioned.Message ID: @.>

Sounds great, and I will look into DR-learner today.

[jialul]

Hi @xiaolong-y I checked the documentation of Kennedy's package npcausal, sadly it does not implement the proposed CATE estimator in the paper. Instead, the package implements doubly robust estimator for ATE and ATT, and accommodates continuous treatments and incremental propensity score interventions. If you run into other packages that estimate CATE directly, happy to take a look.

@jialul Kennedy 2020 seems to be an interesting paper that proposes a new meta-learner: DR-learner. The package that implements that is npcausal. Shall we look into this and see if we want to incorporate this to our package?

[xiaolong-y]

Hi @xiaolong-y I checked the documentation of Kennedy's package npcausal, sadly it does not implement the proposed CATE estimator in the paper. Instead, the package implements doubly robust estimator for ATE and ATT, and accommodates continuous treatments and incremental propensity score interventions. If you run into other packages that estimate CATE directly, happy to take a look.

@jialul Kennedy 2020 seems to be an interesting paper that proposes a new meta-learner: DR-learner. The package that implements that is npcausal. Shall we look into this and see if we want to incorporate this to our package?

@jialul Thank you for checking. I think that is right. Perhaps another approach is to call into a python package to estimate the CATE, but I will share more here is that is the only way.