More AB testing features

[jorgemoralespou]

This is not an issue but a RFE/Request-for-documentation issue.

When you read Christian's Posta description of AB Testing in his blog:

A/B testing is NOT blue-green deployments. I’ve run into groups that mistake this. A/B testing is a way of testing features in your application for various reasons like usability, popularity, noticeability, etc, and how those factors influence the bottom line. It’s usually associated with UI parts of the app, but of course the backend services need to be available to do this. You can implement this with application-level switches (ie, smart logic that knows when to display certain UI controls), static switches (in the application), and also using Canary releases (as discussed below)

it's clear that the criteria to do AB testing usually will be not random, so you want to target specific users to version B of your application and them allow those same users to provide feedback about the experience they had with B. In this regard, application creators will typically introduce a way to identify those users that need to experience the B version of the app. While doing Level 7 at the routing layer is out of scope, and api managers could cope with this use case, a basic routing based on HTTP headers/cookies can be done in the current routing layer in a similar way as weighted balancing. If the consumer uses header/cookie XYZ direct them to version B of the application.

This is very frequent in mobile applications, where a new version of a mobile app will be handed to users (that will include this header/cookie) so they will interact with the B version of the backend for that application. The same technique with be used for web applications and many other clients.

This request is to allow to have make the condition to do AB testing not only based on weight, but also on cookie and/or headers.

Another request/question, that hasn't been discussed before, for this scenario (or any other BlueGreen or Canary) is that of being able to have application A and B in same or different projects. Currently AB needs to reside on the same project, as they are implemented as routes, but some applications will require that different versions live in different project if they need to be live at the same time (maybe it's not easy to change service names) so to segregate they need to reside in different projects. Would the main route be able to do route to service A in project A and to B in service B? Given of course there is network connectivity (if multitenant SDN plugin is used) would it be possible to do a FQDN to a different service, or one local B service that routed to a remote B service?

And third request/question is how to do AB testing of two services without using the routing layer. Would that be possible? Service A in Project A needs to talk to Service B in Project B. This communication is direct trough service layer. Could the application developer do AB testing of the service B without needing to communicate through routing layer?

cc/ @smarterclayton @rajatchopra @knobunc

[openshift-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten /remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen. Mark the issue as fresh by commenting /remove-lifecycle rotten. Exclude this issue from closing again by commenting /lifecycle frozen.

/close