assaf / vanity

Experiment Driven Development for Ruby
http://vanity.labnotes.org
MIT License
1.55k stars 269 forks source link

Trying to use vanity on rails-api application. #302

Closed luiscarlos-gonzalez closed 6 years ago

luiscarlos-gonzalez commented 8 years ago

Hi there, I'm trying to use Vanity on an app that have a normal MVC and an API provide by rails-api. I use config.api_only = false that enable the sessions and the cookies for the API.

The experiment consist in show two ways of sign up forms, one use the regular MVC and the other use the API calls for sign up. The metric that we use is payment, we want to know how many of the people that signs up on the application pay for the service.

Showing the experiment alternatives and add the participants works perfect but add the conversions is not working, when I run the regular MVC experiment everything works well but when I run the version that make the API calls is not registering the conversions count.

I guess when the app make the API calls is not identifying the user and not adding the conversion for that reason.

My first problem is that I have no use_vanity method on my ApplicationApiController so I change the gem for include vanity on ActionBase::API.

On vanity/lib/vanity/frameworks/rails.rb

# Enhance ActionController with use_vanity, filters and helper methods.
ActiveSupport.on_load(:action_controller) do
  # Include in controller, add view helper methods.
  ActionController::Base.class_eval do
    extend Vanity::Rails::UseVanity
    include Vanity::Rails::Filters
    include Vanity::Rails::Identity
    helper Vanity::Rails::Helpers
  end

  ActionController::API.class_eval do
    extend Vanity::Rails::UseVanity
    include Vanity::Rails::Filters
    include Vanity::Rails::Identity
    helper Vanity::Rails::Helpers
  end
end

That make available the use_vanity method on my ApplicationApiController.

class ApplicationApiController < ActionController::API
  include ActionController::Serialization
  include Pundit

  use_vanity :current_user
end

Then in my payments controller I call track! method

def create
    if payment_form.save
      Vanity.track!(:fast_pay_metric)
    end
end

In the metrics section of vanity GUI I see how the metric is increasing but in the experiment info is not showing the conversions, any one have any idea why is not working well?

This is my experiment and metric

ab_test "Fast paid vs Classic Join Pool" do
  description "Fast Paid vs Join Pool"
  alternatives :join_pool, :fast_pay
  metrics :fast_pay_metric
  default :join_pool
end
metric "Fast pay payment metric" do
  description "Exclusive metric for payments using fast pay experiment"
end
phillbaker commented 8 years ago

Hi there, sorry for the slow response on this.

It sounds like the tracking isn't connecting the users' Vanity id with the experiment. Is vanity configured to add users via JavaScript?

On Thursday, August 4, 2016, Luis Carlos González Hernández < notifications@github.com> wrote:

Hi there, I'm trying to use Vanity on an app that have a normal MVC and an API provide by rails-api. I use config.api_only = false that enable the sessions and the cookies for the API.

The experiment consist in show two ways of sign up forms, one use the regular MVC and the other use the API calls for sign up. The metric that we use is payment, we want to know how many of the people that signs up on the application pay for the service.

Showing the experiment alternatives and add the participants works perfect but add the conversions is not working, when I run the regular MVC experiment everything works well but when I run the version that make the API calls is not registering the conversions count.

I guess when the app make the API calls is not identifying the user and not adding the conversion for that reason.

My first problem is that I have no use_vanity method on my ApplicationApiController so I change the gem for include vanity on ActionBase::API.

On vanity/lib/vanity/frameworks/rails.rb

Enhance ActionController with use_vanity, filters and helper methods.

ActiveSupport.on_load(:action_controller) do

Include in controller, add view helper methods.

ActionController::Base.class_eval do extend Vanity::Rails::UseVanity include Vanity::Rails::Filters include Vanity::Rails::Identity helper Vanity::Rails::Helpers end

ActionController::API.class_eval do extend Vanity::Rails::UseVanity include Vanity::Rails::Filters include Vanity::Rails::Identity helper Vanity::Rails::Helpers end end

That make available the use_vanity method on my ApplicationApiController.

class ApplicationApiController < ActionController::API include ActionController::Serialization include Pundit

use_vanity :current_user end

Then in my payments controller I call track! method

def create if payment_form.save Vanity.track!(:fast_pay_metric) end end

In the metrics section of vanity GUI I see how the metric is increasing but in the experiment info is not showing the conversions, any one have any idea why is not working well?

This is my experiment and metric

ab_test "Fast paid vs Classic Join Pool" do description "Fast Paid vs Join Pool" alternatives :join_pool, :fast_pay metrics :fast_pay_metric default :join_pool end

metric "Fast pay payment metric" do description "Exclusive metric for payments using fast pay experiment" end

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/assaf/vanity/issues/302, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFxKV8Rwjdjx1S-coKffDh4451DqOdfks5qchiqgaJpZM4Jc5sl .

luiscarlos-gonzalez commented 8 years ago

Is not configured to add users via JS.

Something more wired happend, I decided to use mixpanel to track conversions and alternatives, but I don't delete the vanity tracking functions and is receiving data.

Before the deployment to production I delete the use_vanity :current_user from my ApplicationApiController and I use the original gem (I clone the project to add the ActionController::API.class_eval part and used localy) and in production is measuring the expremient data.

The cookie with the user identity travel through ajax requests and it is used for the conversion rate calculation?

The payments controller that have the Vanity.track!(:fast_pay_metric) receive request form an mobile app and I am afraid that these request are interfering the experiment. Obviously when the request came form the mobile app theres is no cookie and no one alternative is shown.

phillbaker commented 6 years ago

Closing this since it seems like things ended up working!