search

jpvanhal / flask-split

A/B testing for your Flask application.
https://flask-split.readthedocs.io/
MIT License
120 stars 37 forks source link

ValueError: math domain error #2

Open arg0s opened 11 years ago

arg0s commented 11 years ago

Thanks for the excellent work on flask-split. At this point, the /split route is bailing out with this error. Let me know if you have any insights. I've got the latest version from pip.

2013-05-17T12:10:04.083015+00:00 app[web.1]: 2013-05-17T12:10:04.083146+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/jinja2/environment.py", line 372, in getattr 2013-05-17T12:10:04.083290+00:00 app[web.1]: return getattr(obj, attribute) 2013-05-17T12:10:04.083378+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_split/models.py", line 110, in z_score 2013-05-17T12:10:04.083477+00:00 app[web.1]: return mean / sqrt(var_cr + var_crc) 2013-05-17T12:10:04.083572+00:00 app[web.1]: ValueError: math domain error

jpvanhal commented 11 years ago

It seems that var_cr + var_crc is negative in your case, which causes sqrt to raise ValueError. I don't know how this is possible since both var_cr and var_crc should always be positive. Can you check what are the values of cr, crc, n and nc variables in z_score property, when this error is raised?

arg0s commented 11 years ago

Here you go. Let me know if you need more details.

>>> cr = 0.0
>>> crc = 10.3333333333
>>> n = 3
>>> nc = 3
>>> 
>>> mean = cr-crc
>>> var_cr = cr * (1 - cr) / float(n)
>>> var_crc = crc * (1 - crc) / float(nc)
>>> var_cr
0.0
>>> var_crc
-32.14814814792963
>>>
jpvanhal commented 11 years ago

The problem seems to be that your control alternative has more completions (31) than participations (3), which leads to crc not being between 0 and 1. I guess the reason for that is that finished() does not somehow mark the experiment as finished or the session is not saved after being modified.

I would appreciate if you could either debug finished() and check if the experiment is properly being saved to the 'split_finished' in session, or provide a minimal code that introduces this bug.

arg0s commented 11 years ago

Hmm OK. Does the module take care of ensuring repeat calls to finish don't inflate the counts or do these need to be managed so they match the ab calls? Users may be refreshing the finish page. On May 18, 2013 1:23 PM, "Janne Vanhala" notifications@github.com wrote:

The problem seems to be that your control alternative has more completions (31) than participations (3), which leads to crc not being between 0 and

  1. I guess the reason for that is that finished() does not somehow mark the experiment as finished or the session is not saved after being modified.

I would appreciate if you could either debug finished() and check if the experiment is properly being saved to the 'split_finished' in session, or provide a minimal code that introduces this bug.

— Reply to this email directly or view it on GitHubhttps://github.com/jpvanhal/flask-split/issues/2#issuecomment-18096807 .

jpvanhal commented 11 years ago

Yes, the module does take care that repeating calls to finish don't inflate the counts.

arg0s commented 11 years ago

Got it. If its fine can I have your IRC handle? I'll ping you when I'm home and we can figure this out.

Arvi On May 18, 2013 1:45 PM, "Janne Vanhala" notifications@github.com wrote:

Yes, the module does take care that repeating calls to finish don't inflate the counts.

— Reply to this email directly or view it on GitHubhttps://github.com/jpvanhal/flask-split/issues/2#issuecomment-18097049 .

NickWoodhams commented 9 years ago

Still have this error in some circumstances. Appears the user conversion tracking may be screwing up the math. Did you solve previously?