Service mesh sometimes stops responding for several seconds

[rfevang]

Is this a BUG or FEATURE REQUEST?:

BUG

Did you review existing epics or issues to identify if this already being worked on? (please try to add the correct labels and epics):

No relevant issues found, no idea where to look for epics.

Bug: Y

What Version of Istio and Kubernetes are you using, where did you get Istio from, Installation details

Istio 0.2.9 Kubernetes v1.7.6-gke.1

Is Istio Auth enabled or not ?

Enabled

What happened:

All services in the cluster stops responding for 20-40 seconds every few hours. This causes external jobs relying on the cluster to stop functioning.

Grafana graph of one such incident:

What you expected to happen:

The service mesh should work continuously, with no periods of downtime.

How to reproduce it:

• Modify Mixer's command line to remove zipkin logging (or the cluster goes down under load for other reasons: #100)
• Run continuous load on the cluster, wait for periods of downtime.

Let me know if there's anything I can do to help debug this, I have no idea where to even start.

[rfevang]

Requests in this case comes both from services in the cluster, as well as external services (through istio ingress). According to the graphs, both of these stopped functioning at the same time.

[louiscryan]

This issue is pretty stale. Please retest against 0.8