mozilla / mozanalysis

A library for Mozilla experiments analysis
https://mozilla.github.io/mozanalysis/
Mozilla Public License 2.0
10 stars 13 forks source link

Improve readability of sizing functions #206

Closed jaredsnyder closed 6 months ago

jaredsnyder commented 7 months ago

This PR is adding wrapper classes for results returned from the objects in sample_size_calc.py. The idea is to add functionality to make it easier to work with results while maintaining backward compatibility

Fixes #204

jaredsnyder commented 7 months ago

For the simplest methods that just return a dict of dicts I created SampleSizeResultsHolder. It is just an extension of the dict class that has some extra methods and attributes, but is set up so that if you interact with it like a dict it works exactly the same as it did before. Added a nice graphing function where the output looks like this:

image

codecov-commenter commented 7 months ago

Codecov Report

Attention: Patch coverage is 93.04348% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 81.43%. Comparing base (2df08a3) to head (3a6524d).

Files Patch % Lines
src/mozanalysis/frequentist_stats/sample_size.py 93.04% 8 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #206 +/- ## ========================================== + Coverage 78.88% 81.43% +2.55% ========================================== Files 23 23 Lines 1037 1142 +105 ========================================== + Hits 818 930 +112 + Misses 219 212 -7 ``` | [Flag](https://app.codecov.io/gh/mozilla/mozanalysis/pull/206/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mozilla) | Coverage Δ | | |---|---|---| | [project](https://app.codecov.io/gh/mozilla/mozanalysis/pull/206/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mozilla) | `81.43% <93.04%> (+2.55%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mozilla#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

jaredsnyder commented 7 months ago

This is what the pretty_results function output looks like Screenshot 2024-02-26 at 11 31 09 AM

jaredsnyder commented 7 months ago

these are just some spelling corrections I came across while browsing. this isn't a code reveiw. if you would like an extra set of eyes, i can do so

Whoops, thanks for catching that! Pushed a commit to fix.