233 Issue / Feature request (choose one and delete the paragraph for other)

Below is neither an issue nor a feature request. But just a request for missing information in the documentation.

Need A/B testing configuration help in documentation.

Feature

We are experimenting with Hydrosphere and wanted to check the A/B testing feature. Here is a sample case that we want to try out. We will have two versions of the model. Based on the client browser, we want to split the traffic say 50% between two different browser types to the different model versions .

For this, we plan to put the following snippet into the /etc/nginx/conf.d/default.conf file as a top level element.

split_clients "$http_user_agent" $model_version { 50% "1"; 50% "2"; }

But we are not sure how and where to use the $model_version value in the Hydrosphere nginx configuration, so that the traffic goes to the respective model versions which are running. On digging deeper through the logs, we could not find any direct HTTP POST URI to the specific model version resource like either /models/{model_version} (or) /api/v2/model/version/demo_regression/{model_version} similar to the below sets of logs shown:

172.18.0.1 - - [24/Jul/2019:11:17:03 +0000] "GET /api/v2/model HTTP/1.1" 200 35 "http://localhost/models/1" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0" "-" 172.18.0.1 - - [24/Jul/2019:11:17:03 +0000] "GET /api/v2/application HTTP/1.1" 200 2 "http://localhost/models/1" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0" "-" 172.18.0.1 - - [24/Jul/2019:11:17:03 +0000] "GET /api/v2/model/version HTTP/1.1" 200 1976 "http://localhost/models/1" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0" "-" 172.16.0.2 - - [24/Jul/2019:11:17:48 +0000] "POST /hydrosphere.discovery.ServingDiscovery/watch HTTP/2.0" 200 5 "-" "grpc-java-netty/1.18.0"

(or)

172.18.0.1 - - [24/Jul/2019:11:22:34 +0000] "GET /api/v2/model/version/demo_regression/4 HTTP/1.1" 200 674 "-" "python-requests/2.22.0" "-" 172.18.0.1 - - [24/Jul/2019:11:22:44 +0000] "GET /api/v2/model/version/demo_regression/4 HTTP/1.1" 200 674 "-" "python-requests/2.22.0" "-" 172.16.0.2 - - [24/Jul/2019:11:22:48 +0000] "POST /hydrosphere.discovery.ServingDiscovery/watch HTTP/2.0" 200 463 "-" "grpc-java-netty/1.18.0"

Further even if we consider that two separate applications need to exist if we want to use 2 different model versions, in that case also, it is not clear in which hydrosphere server configuration element the split_clients evaluated variable should be used so that the traffic is split between the two different applications.

Hence we are not sure if we can configure the A/B testing all by ourselves. And hence seeking your help.

Model name is "demo_regression" Application name is "demo_regression_app_1"

172.18.0.1 - - [24/Jul/2019:11:21:44 +0000] "GET /api/v2/model/version/demo_regression/4 HTTP/1.1" 200 674 "-" "python-requests/2.22.0" "-"

172.18.0.1 - - [24/Jul/2019:11:35:37 +0000] "POST /gateway/application/demo_regression_app_1 HTTP/1.1" 200 25 "http://localhost/applications/demo_regression_app_1" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"

172.18.0.1 - - [24/Jul/2019:11:33:55 +0000] "GET /api/v2/application/demo_regression_app_1 HTTP/1.1" 404 89 "-" "python-requests/2.22.0" "-" 172.18.0.1 - - [24/Jul/2019:11:33:55 +0000] "POST /api/v2/application HTTP/1.1" 200 1589 "-" "python-requests/2.22.0" "-" 172.16.0.2 - - [24/Jul/2019:11:34:03 +0000] "POST /hydrosphere.discovery.ServingDiscovery/watch HTTP/2.0" 200 932 "-" "grpc-java-netty/1.18.0" "-" 172.16.0.2 - - [24/Jul/2019:11:35:18 +0000] "POST /hydrosphere.discovery.ServingDiscovery/watch HTTP/2.0" 200 927 "-" "grpc-java-netty/1.18.0"

/etc/nginx/conf.d/default.conf structure :

split_clients "$http_user_agent" $model_version { 50% "1"; 50% "2"; }

upstreams

upstream manager_grpc {
should we use $model_version in some upstream element ? }

GRPC

server { should we use $model_version in some location element here ? }

HTTP