search

firebase / quickstart-unity

Firebase Quickstart Samples for Unity
https://firebase.google.com/games
Apache License 2.0
836 stars 430 forks source link

A/B Test Variant Targeting Criteria Not Working #585

Closed Handy-TN closed 4 years ago

Handy-TN commented 4 years ago

Please fill in the following fields:

Unity editor version: 2019.2.4f1 Firebase Unity SDK version: 6.6.0 Firebase plugins in use (Auth, Database, etc.): Analytics, Storage, RemoteConfig Additional SDKs you are using (Facebook, AdMob, etc.): Facebook, AppsFlyer Platform you are using the Unity editor on (Mac, Windows, or Linux): Mac Platform you are targeting (iOS, Android, and/or desktop): Android, iOS Scripting Runtime (Mono, and/or IL2CPP): IL2CPP

Please describe the issue here:

I have one A/B Testing running currently with 5 variants + 1 Control Group. I have set the base/main targetting to 100% users with only app id as the targeting The Control Group and 1 variant targets are app version matches exactly 0.0.12. And the other variants' target are app version matches exactly 0.1.18. image

And it seems the targeting variant is not working correctly. When I try to view the real data. It is more like the variant targeting doesn't work at all. Each of the variants have both version. image

Are there any additional configs or anything that I need to configure to fix this? Or it is actually a bug. Because this make the collected data not reliable.

Please answer the following, if applicable:

Have you been able to reproduce this issue with just the Firebase Unity quickstarts (this GitHub project)? Yes

What's the issue repro rate? 100%

paulinon commented 4 years ago

Hi @Handy-TN, it appears that your issue is related to your Firebase project setup rather than your Unity setup. Please file directly to Firebase Support and include screenshots indicating your experiment results and your method of gathering real data.