Assumption of nan when computing weighted KPIs

zalando / expan

Open-source Python library for statistical analysis of randomised control trials (A/B tests)

MIT License

335 stars 50 forks source link

Assumption of nan when computing weighted KPIs #119

Closed shansfolder closed 7 years ago

shansfolder commented 7 years ago

In weighted KPIs, our implementation is based on the assumptions that reference KPIs are never nan.

We might have some miscomputation here. I saw from previous BQ tests that this column can have missing values. We may want to implement the weights in a way that does not rely on this assumption.

shansfolder commented 7 years ago

Here is my draft of implementation of weighting, which does not depend on non-nan assumption: https://github.com/shansfolder/ABTestingEarlyStoppingEvaluation/blob/master/analysis/real.py#L257-L264

shansfolder commented 7 years ago

solved by #121