search

mozilla / webmaker.org

deprecated - https://blog.webmaker.org/whats-next-for-webmaker-tools
https://webmaker.org
Mozilla Public License 2.0
71 stars 99 forks source link

Splash page A/B tests only work for en-US #1085

Closed secretrobotron closed 9 years ago

secretrobotron commented 9 years ago

Try visiting any locale other than en-Us. You will always see the "The web is yours" variant.

This probably needs to be fixed in Optimizely. @simonwex: angular+optimizely pain-point :/ ?

/cc @HPaulJohnson

HPaulJohnson commented 9 years ago

Hmm. I just tweaked the config in optimizely hoping I'd fix. I originally had the experiment's "URL targeting" set for either webmaker.org/en-US or webmaker.org and I removed the /en-US part. So the experiment is now just running off the webmaker.org URL.

The variants are still serving as expected but unfortunately the non-en locales are all still defaulting to "the web is yours" variant. Others should verify in the morning but doesn't look like this tweak fixed the issue.

If there are other ideas about how to better configure for the localized versions let me know. Regardless, when evaluating results in GA we should just be comparing /en-US vs. /en-US/home-2, etc. as those results should be unaffected.

secretrobotron commented 9 years ago

Ah that was my fault. Changed it to work with both because https://webmaker.org/en-US never uses another variant.

AFAICT, like @HPaulJohnson said, non-en locales are still pointing at that single variant. That's still a problem.

I just tried adding a regex to optimizely (https://webmaker.org/[a-zA-Z][a-zA-Z]-[a-zA-Z][a-zA-Z]) which almost worked, but it still pointed back to the en-US version.

What might work is taking the en-US out of the destinations that the Optimizely router points to. @HPaulJohnson thoughts?

HPaulJohnson commented 9 years ago

@secretrobotron I stripped out en-US from all the redirects in optimizely. Looks like that may have fixed it. Double check and let me know?

secretrobotron commented 9 years ago

@HPaulJohnson Yeah. Definitely works for https://webmaker.org/, but not https://webmaker.org/en-US. Same with https://webmaker.org/pt-br/, https://webmaker.org/zh-CN etc.

Not sure how much of a problem is though. I bet some tiny % of people already have the locale string on the end of the url they use to get to webmaker.

thisandagain commented 9 years ago

I'd rather have this problem than the former. :wink:

The % of folks who hit the locale specific version of the splash pages directly is fairly low as a % of new users.

secretrobotron commented 9 years ago

Ah that's great. Thanks @thisandagain, @HPaulJohnson . My nonsense is noted.