search

marvelapp / react-ab-test

A/B testing React components and debug tools. Isomorphic with a simple, universal interface. Well documented and lightweight. Tested in popular browsers and Node.js. Includes helpers for Mixpanel and Segment.com.
MIT License
240 stars 47 forks source link

Showing only one variant, every-time refreshed #20

Closed khanharis87 closed 3 years ago

khanharis87 commented 3 years ago

Table of Contents

Expected Behavior

show 50, 50 of 2 variants as assigned in defineVariants

emitter.defineVariants(
  "themeExperiment",
  ["darkTheme", "lightTheme"],
  [50, 50]
);

const Pride: React.SFC = () => {
  return (
    <Experiment name="themeExperiment">
      <Variant name="darkTheme">
        <DarkTheme />
      </Variant>
      <Variant name="lightTheme">
        <LightTheme />
      </Variant>
    </Experiment>
  );
};

Current Behavior

Only one variant being shown

Possible Solution

My console cries about using old react lifecycle methods.

Steps to Reproduce (for bugs)

Use one the examples and it does the same, after refreshing few times, it still shows same variant

Your Environment

moretti commented 3 years ago

@khanharis87 It's because the active variant is saved on local storage (see calculateActiveVariant and store).

It is done on purpose, ideally a user should always get the same variant and this is best thing that we can do client-side.

If you delete PUSHTELL-${experimentName} from local storage and refresh the page, react-ab-test will pick another variant (see https://developers.google.com/web/tools/chrome-devtools/storage/localstorage).

khanharis87 commented 3 years ago

Thanks for the reply, It makes sense, but before( last week) it was showing different variants on refresh. So that is why I am confused

moretti commented 3 years ago

@khanharis87 it might be because you used a different browser, cleared the browsing data or, in case of local development used a different address or port number, (e.g. localhost:1234 vs localhost:5678 or 127.0.0.1 vs localhost).