Vanity not respecting test choice in Rspec

[jennykortina]

I am trying to write a controller test in Rspec where I am forcing Vanity to choose a value, but the forced value is not respected

before do
    Vanity.context = controller
     Vanity.playground.experiment(key).chooses(value)
 end

[phillbaker]

Hi there! Thanks for using vanity, sorry to see you're having an issue.

There are some similar examples of using this helper in the vanity test suite:

https://github.com/assaf/vanity/blob/bc0205da9b1789c1f59e779b7085acb167b27a51/test/frameworks/rails/action_controller_test.rb#L56

(where experiment is just

https://github.com/assaf/vanity/blob/bc0205da9b1789c1f59e779b7085acb167b27a51/test/test_helper.rb#L132-L134

What are the value of the variables key and value you're passing in? Can you include how you're configure the experiment?

[jennykortina]

Hey,

Thanks for the quick reply!

One question I have based on your response is why are you using a symbol instead of a string to pass which experiment you are choosing:

experiment(:pie_or_cake).chooses(:pie)

In all the examples I've seen pass a string as the choice: https://github.com/assaf/vanity/issues/199

Does it need to be passed a symbol to work?

In any case this is what I did: the method I'm trying to test is (which is a private method in ApplicationController for Rails):

def _vanity_experiment_alternative(experiment)
  cookie = cookies[:experiments] && JSON.parse(cookies[:experiments])&.dig(experiment.to_s)

  if !cookie
    vanity_test_alternative = Vanity.ab_test(experiment)
    cookies[:experiments] = {:value => JSON.generate({experiment => vanity_test_alternative})}
    vanity_test_alternative
  else
    cookie
  end
end

I want two write two tests, one that the cookie gets set if not set, and the other that it does not get set if it's already present.

I don't have the exact test I had written anymore but it's something like the following:

describe '_vanity_experiment_alternative' do
  key = :jk_test
  value = 'i am a great value'

  before do
    Vanity.context = Struct.new(:vanity_identity).new(user.id)
    Vanity.playground.experiment(key).chooses(value)
  end

  it 'sets the cookie if cookie does not exist' do
    get :index

    expect(cookies['experiments']).to eq JSON.generate(key => value)
  end
end

The above test would not reliably return the value in the cookie that I was forcing Vanity to choose. I had to end up testing it like this which is not ideal

describe '_vanity_experiment_alternative' do
  it 'sets the cookie if cookie does not exist' do
    key = :jk_test
    get :index
    expect(['i am a great value', 'i am the worst value']).to include JSON.parse(cookies['experiments'])[key.to_s]
  end
end

Any help would be appreciated I was digging around the issues/source code for a while and couldn't figure out why it wasn't getting set.

[jennykortina]

@phillbaker I just tested again using a symbol instead of a string for the value and it still does not work.

[phillbaker]

@jennykortina sorry for the slow reply here, what version of rails were you using?