assaf / vanity

Experiment Driven Development for Ruby
http://vanity.labnotes.org
MIT License
1.55k stars 269 forks source link

use_vanity not accepting identification method #303

Closed thomasjedstrom closed 7 years ago

thomasjedstrom commented 8 years ago

I initiated vanity using "use_vanity :current_user" in the action controller, but ended up nixing the devise gem, so it's been defaulting to creating it's own identities.

I recently was trying to force a new participant, so I changed the application controller to this:

  use_vanity :user_identity

  protected

  def user_identity
    (0...8).map { (65 + rand(26)).chr }.join
  end

The idea was then to create a link within the project where a new participant ID could be generated and to rerun the test.

The issue may be that since it's already cookied the identity of my computer, trying to force a new participant is failing. Is there a way around this?

thomasjedstrom commented 8 years ago

btw, I've never submitted an issue on github before and I'm semi-new to open source, couldn't seem to get the gist to show up up there.

phillbaker commented 8 years ago

Hey there! Thanks for using vanity!

Have you tried a private/incognito window?

On Wednesday, August 31, 2016, Thomas Edstrom notifications@github.com wrote:

btw, I've never submitted an issue on github before and I'm semi-new to open source, couldn't seem to get the gist to show up up there.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/assaf/vanity/issues/303#issuecomment-243831874, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFxKUlLoPKy3m1eUtBQqYOvtms8B8nDks5qlbRYgaJpZM4Jx1d0 .

daniel-nelson commented 7 years ago

I just ran into something similar to this today. It seems to me that the two conditions listed here should have their order reversed, so that cookies become a fallback. Otherwise, the cookies override whatever variant the user has already been opted into, so the user may see different variants on different devices / after clearing cookies when a user returns, is opted into an experiment, and then signs in. https://github.com/assaf/vanity/blob/8701dd5c0de2b273b14dd769fe15e95280c6b1e8/lib/vanity/frameworks/rails.rb#L75-L78

thomasjedstrom commented 7 years ago

Hey phil, Sorry I hadn't gotten back on this, but thanks for the response. I ended up having to put the project down for a while. Regardless, going incognito seems like a logical workaround. Seriously, kudos on this gem. I will be coming back to this project (by the end of this coming week (Sept. 30th)).

thomasjedstrom commented 7 years ago

Hey Phil,

Alright, finally was able to get everything back up and running, and going incognito worked. Thanks again for getting back to me on this and for what I'm doing that should work.

phillbaker commented 7 years ago

Hi @daniel-nelson - it's definitely a tricky spot. Looking at the note a few lines above that:

# This implementation prefers an initial vanity cookie id over a
# new user.id to avoid the flash of alternative B (FOAB).

it looks like there are disadvantages to either approach. Maybe we could make it a configuration setting?

daniel-nelson commented 7 years ago

@phillbaker A configuration would be excellent. We're using Vanity to determine what type of referral program a user is in (give $10 off, give $20 off, give $10 + get $10), so it's important that a user be shown consistent results. In our case, the user won't see FOAB because we only show information about our referral program to signed-in users. For now, I've simply forked and swapped those two conditions.