mozilla / mozanalysis

A library for Mozilla experiments analysis
https://mozilla.github.io/mozanalysis/
Mozilla Public License 2.0
9 stars 13 forks source link

Verify target_list and metric_list sources match #220

Closed jaredsnyder closed 3 months ago

jaredsnyder commented 3 months ago

Fixes #173

codecov-commenter commented 3 months ago

Codecov Report

Attention: Patch coverage is 94.73684% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 80.64%. Comparing base (2df08a3) to head (79ff51b).

Files Patch % Lines
src/mozanalysis/sizing.py 93.33% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #220 +/- ## ========================================== + Coverage 78.88% 80.64% +1.76% ========================================== Files 23 23 Lines 1037 1054 +17 ========================================== + Hits 818 850 +32 + Misses 219 204 -15 ``` | [Flag](https://app.codecov.io/gh/mozilla/mozanalysis/pull/220/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mozilla) | Coverage Δ | | |---|---|---| | [project](https://app.codecov.io/gh/mozilla/mozanalysis/pull/220/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mozilla) | `80.64% <94.73%> (+1.76%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mozilla#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

m-d-bowerman commented 3 months ago

The other thing that occurred to me is that using "source" in the warnings might be ambiguous -- users might take it to mean multiple DataSources are causing the warnings. I think using "app" might be more informative (e.g. segments and metric do not have a common app)

jaredsnyder commented 3 months ago

The other thing that occurred to me is that using "source" in the warnings might be ambiguous -- users might take it to mean multiple DataSources are causing the warnings. I think using "app" might be more informative (e.g. segments and metric do not have a common app)

Good catch, fixed it