nuxt-community / google-optimize-module

SSR friendly Google Optimize module for Nuxt.js
MIT License
215 stars 32 forks source link

Unsure of how to use #19

Open icehaunter opened 4 years ago

icehaunter commented 4 years ago

Good day!

I am unsure as to how to actually use the module with Google Optimize.

We have a SSR-ed Nuxt application, with Google Tag Manager installed and properly set up. We recieve statistics and added the Google Optimize bingings to the GTM on the GTM's side and it is working properly - segmentation occurs, cookie is set and all that.

Then, we've brought in your module, to actually react to the experiments on the server side. And this is where I get confused: your module sets the cookie with name exp and format { experimentID }.{ variant index }. GTM sets the cookie with name _gaexp and format GAX1.2.2{ experimentID }.?????.{ variant index }. Variant index actually differs between two cookies from time to time, and I don't really know which one actually applies.

This is separate (I think) from #16, as the $exp object is available.

Thank you for your assistance!

brandonaaskov commented 4 years ago

I'm going to jump in here as well: it's really not clear how this is supposed to be used. I also have GTM and GA properly set up (as far as I know), and I can retrieve the $exp object in my code. That said, nothing is getting reported back to Google Optimize and it's not clear why that's the case.

Could you publish a tiny sample app on how to use this on Codesandbox or something similar? That would go a long way toward understanding how to work with this module.

kklon commented 4 years ago

I'd also really appreciate some information as on how this plugin should be used. I've set up a simple experiment both in optimize, and in nuxt - and it seems like Nuxt is taking care of assigning users to experiment variations, rather than GO. So when I preview, say, an experiment variation no2, the plugin doesn't set no2 as active, but selects a weighted random one. making the reporting and stats from GO completely useless :|

ItsMeDelanoDev commented 4 years ago

I'm also extremely curious why we should define our own variant weights within the configuration. I agree with the comments above, it seems this library is doing it's own assigning based on randomness. This completely defies the purpose of Google Optimize, right?

Also the fact it seems this library is not maintained regularly prevents me a little bit from using it.

tschut commented 4 years ago

Personally I'm using this lib:

This is useful combined with 'server side' experiments in GO, i.e. where you use the tracking/data collection GO provides but handle experiment assignment yourself. Since some of my experiments are actually a/b-tests on the backend this is very useful.

Then again, since I don't use multivariate tests I made a local copy of this lib in /plugins and tweaked it myself, so strictly speaking I'm not using this lib I guess.

brandonaaskov commented 4 years ago

I've gotten this working on our project, and it's not that clear so I thought I'd share here. I'll open a PR soon to update the README.

First, here's the code sandbox: https://codesandbox.io/s/nuxt-google-optimize-example-43w4y

You'll either get three pictures of Keanu, or three pictures of Swayze depending on which bucket Google Optimize decides you fall into. You can test this by opening a variety of private browser windows. You can refresh over and over (within Code Sandbox's limits) and see that your list never changes: you've been properly bucketed.

The setup is in the nuxt.config.js file. The experiment file just has to match on the experiment ID. The nuxt config sets up Google Optimize alongside Google Analytics. What happens behind the scenes is that Google Optimize will bucket you (represented as this.$exp.variantIndexes) into one of the variants you created for your experiment. What this module does is automatically populate $activeVariants with that variant's data provided by your experiment file. This is demo'd in that code sandbox. You can edit the .env file if you want to try with your own setup.

Major thing to note: tracking doesn't show up in Google Analytics until about 24 hours later so you could easily be misled into thinking it's not working when it probably is.

Setting up the google optimize side of things is a little unclear too, so here's a pic of that: image

brandonaaskov commented 4 years ago

Apologies to anyone who tried to use that Codesandbox link ☝️and couldn't: I accidentally deleted the critical codesandbox.js file. It's working again now.

yaelp2 commented 4 years ago

I've gotten this working on our project, and it's not that clear so I thought I'd share here. I'll open a PR soon to update the README.

First, here's the code sandbox: https://codesandbox.io/s/nuxt-google-optimize-example-43w4y

You'll either get three pictures of Keanu, or three pictures of Swayze depending on which bucket Google Optimize decides you fall into. You can test this by opening a variety of private browser windows. You can refresh over and over (within Code Sandbox's limits) and see that your list never changes: you've been properly bucketed.

The setup is in the nuxt.config.js file. The experiment file just has to match on the experiment ID. The nuxt config sets up Google Optimize alongside Google Analytics. What happens behind the scenes is that Google Optimize will bucket you (represented as this.$exp.variantIndexes) into one of the variants you created for your experiment. What this module does is automatically populate $activeVariants with that variant's data provided by your experiment file. This is demo'd in that code sandbox. You can edit the .env file if you want to try with your own setup.

Major thing to note: tracking doesn't show up in Google Analytics until about 24 hours later so you could easily be misled into thinking it's not working when it probably is.

Setting up the google optimize side of things is a little unclear too, so here's a pic of that: image

In the code, you have experimentID: "Ch0MEC80TrqHroVtx3dHnw", Where did you take the experimentID from? there is a different experimentID in the picture of google optimize that you uploaded

vinialbano commented 4 years ago

As said by @icehaunter the cookie sent by the library is sometimes different than the cookie sent by Google Analytics. Shouldn't the library read the GA's cookie content to work propertly?

deepaksingh007 commented 4 years ago

I am also seeing two different cookies and each show different variation that I bucketed into. @brandonaaskov even after your sandbox the issue is same.

hecktarzuli commented 4 years ago

https://deltener.com/blog/how-to-setup-google-optimize-with-nuxt/

The purpose of this library is to facilitate server side experiments. In that scenario, the server (this module) splits the audience, assigns the experiments, and tells Optimize (really GA) about the experimentId/multivariant assigned. The only thing Optimize is used for is for reporting. (https://developers.google.com/optimize/devguides/experiments). I have a deep dive in my blog above.

@kklon @brandonaaskov @dneeleman @icehaunter @deepaksingh007

bogdan12893 commented 3 years ago

@hecktarzuli How can you can I create multiple experiments? I can only see the first experiment on $exp. Am i missing something?

import experiment1 from './experiment1'
import experiment2 from './experiment2'

export default [experiment1, experiment2]
hecktarzuli commented 3 years ago

@bogdan12893 have you tried using a new browser/incogneto to get assigned a new experiment? What are your weights per exp?

bogdan12893 commented 3 years ago

@hecktarzuli I did the incognito test and it works. Can you take a look if I approached the experiments ok? I feel like my code is not DRY at all. And my styles.scss files are 90% identical, I guess you need for every experiment a separate .scss file in order to apply styles for that specific experiment. In my case I have two buttons with the same styles but the only diference is the text inside of the button so i have to .scss files with same styles. Is there a better way to do this? Another question, how can i use experiments inside components and not on the entire body? Can we have multiple experiments the same time?

And thank you for the fast reply 🙌🏻

import BtnNavQuote from './btn-nav-quote'
import BtnNavContact from './btn-nav-contact'

export default [BtnNavQuote, BtnNavContact]
import './styles.scss'

const navBtnTxt = 'contact'

export default {
  name: 'btn-nav-contact',
  experimentID: '_kq_AiNERBWSyp-OsgaCwg',

  variants: [
    { weight: 25, btnText: navBtnTxt },
    { weight: 25, btnText: navBtnTxt },
    { weight: 25, btnText: navBtnTxt },
    { weight: 25, btnText: navBtnTxt }
  ]
}
import './styles.scss'

const navBtnTxt = 'request a quote'

export default {
  name: 'btn-nav-quote',
  experimentID: 'GhVRdryPTqmMgqY-vcG-wg',

  variants: [
    { weight: 25, btnText: navBtnTxt },
    { weight: 25, btnText: navBtnTxt },
    { weight: 25, btnText: navBtnTxt },
    { weight: 25, btnText: navBtnTxt }
  ]
}
hecktarzuli commented 3 years ago

@bogdan12893 If your CSS is 90% the same, you could take the common parts, put those into another file and have your experiments import that common css too. I should mention, you don't have to use the imported styles at all, you can use the properties of $exp in your components to just do whatever you want. $exp is the currently assigned experiment so you could check if $exp.name == "myAwesomeButtons" and if $exp.variants = contains whatever and inject your own style tags or Vue component that's already styled etc..

You can run multiple experiments at the same time, but no user will be assigned more than one at a time. This is to ensure tests don't poison each other's results.

DRY is a good principle, but for simple things it's not a huge deal to duplicate sometimes. AHA is the opposite (Avoid Hasty Abstractions) -- Kent C Dodds has a great post on this. Basically when it becomes a pain, use DRY, otherwise I usually use AHA until a pattern emerges.

Scino commented 3 years ago

How should it be used in order to automatically fetch the experiments from Google Optimize, without hardcoding experiments inside a Nuxt project?

I need it just as a replacement of the Optimize script, that seems to work at first, but then all of the changes get overwritten by Nuxt's hydration.

<script src="https://www.googleoptimize.com/optimize.js?id=OPT_CONTAINER_ID"></script>

hecktarzuli commented 3 years ago

@Scino the main purpose of this Module is for server side experiments. Mostly this is for a) complex experiments where your own tooling is more ideal or b) you are concerned about speed. If you aren't doing server-side experiments then you lose most of the value of the module, and I'd recommend just using Google's Client-Side script instead.

For the script you have, you could try injecting that as a plugin instead of their script tag. Another option (depending on what hydration problems you are fighting) is to use window.onNuxtReady to inject the script.