[hack] support Sail on vanilla Kubernetes now that OperatorHub.io has the Sail operator

[jmazzitelli]

These set of scripts will be able to install a full Istio-Kiali setup locally on a vanilla Kubernetes (not OpenShift).

You will get Sail operator and its Istio installation, Kiali operator and Kiali UI, along with Tempo operator and Tempo (for tracing backend and JaegerUI), Grafana, and Prometheus. The external services will have their UIs available assuming your k8s has a LoadBalancer (the steps below will give you that).

Here's how you can try it out.

For those that already have the kiali source repo git cloned locally and this PR branch checked out, here's a quick summary of the commands you can run to see everything... following this will be a more detailed set of instructions that even those that do not have the full source git cloned can run this stuff.

Quick summary of the test steps

1. hack/k8s-minikube.sh --load-balancer-addrs "70-84" start
2. hack/istio/sail/install-ossm-release.sh -c kubectl install-operators
3. sleep 30 # (wait a little bit for the operators to start installing - just wait 30 seconds or so for OLM to start installing things)
4. hack/istio/sail/install-ossm-release.sh -c kubectl install-istio
5. hack/istio/install-bookinfo-demo.sh -tg -c kubectl -ail istio.io/rev=default-v1-23-0
6. hack/kiali-port-forward.sh
7. Point your browser to Kiali UI at http://localhost:20001/kiali/console

Give it a minute or two for everything to start up and requests start flowing through the bookinfo traffic generator. Eventually you should see everything - bookinfo demo graph, you should be able to see traces, Jaeger UI , and Grafana UI links.

More details on the test steps

1. Start minikube. You can put a loadBalancer in it if you want to try to access the Jaeger UI (but I haven't been able to get that to work yet.) You can use our hack script for this (not part of this PR) or just start minikube on your own. You can use KinD if you want, rather than minikube.

   If you want minikube (which is suggested for testing this stuff because this gives you a load balancer) - call k8s-minikube.sh with the arg --load-balancer-addrs "70-84". If you do not already have the entire kiali source repo git cloned locally, you can run this command so you just run the script without the need to clone everything):

   bash <(curl -sk https://raw.githubusercontent.com/kiali/kiali/master/hack/k8s-minikube.sh) --load-balancer-addrs "70-84" start

   If you really want KinD (not sure how this works without a load balancer - might not be able to access grafana UI or jaeger UI, but other than that things should work)

   kind create cluster

2. You need this PR checked out, obviously. For those that don't have the Kiali source code checked out, just pull down the script files from the PR branch using this (after this runs, you'll have the scripts in ./kiali/hack/istio/sail):

   git clone --filter=tree:0 --no-checkout --depth 1 --sparse --single-branch --branch hack-sail https://github.com/jmazzitelli/kiali.git kiali && \
   cd kiali && \
   git sparse-checkout set --no-cone hack/istio/sail && \
   git checkout

3. Make the current working directory the location where the scripts are (cd to the hack/istio/sail directory)
4. Install all the necessary operators (sail, kiali, tempo). This will also install base OLM, too:

   ./install-ossm-release.sh -c kubectl install-operators

5. Install Istio and Kiali and all other addons:

   ./install-ossm-release.sh -c kubectl install-istio

6. You can look at the status of everything:

   ./install-ossm-release.sh -c kubectl status

7. Confirm Istio pods and Kiali pod is running via kubectl get pods -n istio-system (the same info can be seen in step above, too)
8. At this point the Kiali UI should be available. You can port-forward to it and point your browser to it at http://localhost:20001/kiali/console:

   kubectl port-forward -n istio-system $(kubectl get pods -n istio-system -l app.kubernetes.io/name=kiali -o jsonpath='{.items[0].metadata.name}') 20001:20001

9. Pass in --help to that ./install-ossm-release.sh script for more commands (you can delete istio, delete the operators, etc).

**NOTE: Because this is revision based, and istio tags are not supported by sail yet, you have to label your mesh namespaces with istio.io/rev=default-v1-23-0 ... you can see this work by installing bookinfo demo (using Kiali's hack script enhanced with -ail option that I added to this PR -- hack/istio/install-bookinfo-demo.sh -tg -c kubectl -ail istio.io/rev=default-v1-23-0).

[jmazzitelli]

This PR mainly touched the hack/istio/sail scripts, so doesn't affect anyone. There is one change to the hack script that installs bookinfo - it now allows you to tell it which auto-injection label to add to the bookinfo namespace. The old behavior is still the default behavior (it adds istio-injection=enabled label) so this should not be a breaking change.

[jmazzitelli]

You can see tracing is integrated and you can get to the Tempo UI:

[jmazzitelli]

Here you can see the Grafana integration:

[jmazzitelli]

The bookinfo graph also displays correctly:

[jmazzitelli]

Note: Sail does not install any gateways (which is why you'll see Kiali with some error/warnings in the masthead - its about missing ingress/egress gateways). Installations of those things are left up to the user. In the future we can add options to the scripts to install gateways but I left that out for now.

[jmazzitelli]

If you want to try this on OpenShift to see it work there, same basic commands, just spin up CRC and use "oc" for the client:

1. hack/crc-openshift.sh start
2. hack/istio/sail/install-ossm-release.sh -c oc install-operators
3. sleep 30 # (wait for OLM to start deploying things)
4. hack/istio/sail/install-ossm-release.sh -c oc install-istio
5. hack/istio/install-bookinfo-demo.sh -tg -c oc -ail istio.io/rev=default-v1-23-0

There should be a Kiali route in the istio-system namespace that you can point your browser to now. Bonus - you also get OSSMC:

Kiali UI route: https://kiali-istio-system.apps-crc.testing/