Create R package

[mollyow]

References

• I think this is a good start.
  • I'd like to use the instructions starting on section 3 p. 7
• Hadley Wickham and Jennifer Bryan have a much more detailed bookdown here; may be helpful for debugging

---

To-do to get started

• [x] Create a branch for package development.
• [x] Determine which functions we actually need in /code for running simulations and generating confidence intervals. Consolidate code and delete everything we don't need.
• [x] Create a minimal working version of the package. We'll fill in details and write up vignettes as we go.
  • [x] Move package up to main level of repository
• [x] Once the minimal working version is working, we'll merge the branch into main.
• [ ] Working on vignettes: See e.g., https://github.com/DeclareDesign/estimatr/tree/main/vignettes @mollyow
  • [x] Tidy up the contextual vignette
  • [x] Write non-contextual vignette
  • [ ] Add further notes; update 4x arms figure to label as groups that each arm is best for; add more notes on how policy is learned; add commentary on uniform/non-contextual two-point regarding relative performance
  • [ ] Check against Python simulated settings to make sure we get the same confidence intervals with both contextual and non-contextual data.
    • [ ] #15
    • [x] #14
    • [x] We still need to add in something like ridge_muhat_lfo_pai() to LFO estimation; should we do this as a function (not very generalizable) or just write up in the vignette?
  • [x] Add plotting function after estimating response / treatment effects and CIs, like what is in starting_script.R right now. @mollyow

[YinghuiZhouu]

What's needed for a good R package:

• Good user file: Functions are well commented. Inputs are clearly described.
• Vignette with illustrations.
  • For both contextual and non-contextual experiments
  • Figures and illustrations implemented in Base R
  • Nicely formatted output and tables
• Clear error messages.
• Considerations for computational efficiency? where are bottlenecks? (have undergrad RA test?)
• Show progress percent bar.
• Add random seeds.

To-do:

• [x] Start creating package
• [x] Test package and iterate (undergraduate RA can help with testing?)

[mollyow]

Continuing work:

• [x] Work on examples. Remaining failures in checks on my end are in the examples.
• [x] Add error checks so that the code is more robust, enforcing that each argument is the right shape and type, there aren’t NAs if that would break the code etc. We also need some objects to be the same shape, e.g. in output_estimates(), this is the case for the gammahats and contextual_probs arguments. We should start with these on higher level functions (generate_bandit_data(), output_estimates(), run_experiment(), calculate_mu_hat(), calculate_balwts()…). For lower-level functions that we only use inside other functions, the inherited data checks should keep anything from breaking. We may want to consider de-emphasizing documentation for these, and even making them internal only functions by putting a . in front of them (e.g., .check_A()).
• [x] Continue tidying, clarifying documentation. The error checks we put in place should be consistent with the documentation. E.g., if we require >1 observation data sets to conduct inference with output_estimates(), we impose the .check_A() function at the start of the function call, and we add to the documentation that it will only work if gammahats and contextual_probs are > length 1 and of the same dimensions. @mollyow
• [x] Keep thinking of ways to break the code. Add data checks, error codes, documentation to deal with this.
  • [x] Does it break if we try and run an experiment with K = 1?
  • [x] Or if we try to conduct inference with a gammahat with one column?
  • [x] What happens if rows of probs don’t sum to 1? We should write a line in that forces them to sum to 1.
• [x] Estimating the expected rewards of different arms using ridge regression, adding the functions into adaptive_utils. Calculate mu_hat in banditsCI line 70 with the new function.