Add mysql-admin port to the headless service

[zoetrope]

fix #657

[vsliouniaev]

I think it may work in certain cases where there is no enforced mTLS in the mesh with the extra port from https://github.com/cybozu-go/moco/pull/658 Though it does not work in my case and I cannot turn off mTLS for the entire mesh.

There is a way to get this working end-to-end, which I describe at the bottom

Tests

Original Issue

mysql-admin port exists on service : FALSE
excludeInboundPorts                : FALSE

This is the original issue described. Fails - since there is no route to the endpoint through istio on the inbound side the request times out. mysql proxy error: "response_code_details":"filter_chain_not_found"

Treat mysql as external service

mysql-admin port exists on service : FALSE
excludeInboundPorts                : TRUE

This works in my setup but may not work in all setups. I think the route is handled as being outside the mesh.

Add admin port

This is with https://github.com/cybozu-go/moco/pull/658

mysql-admin port exists on service : TRUE
excludeInboundPorts                : FASE

Fails - the istio proxy being unable to reach mysqld on the pod's external IP address. mysql proxy error: "upstream_transport_failure_reason": "delayed_connect_error:_111"

Bypass istio on mysql side

mysql-admin port exists on service : TRUE
excludeInboundPorts                : TRUE

This fais with a TLS error on the operator side: "upstream_transport_failure_reason": "TLS_error:_268435703:SSL_routines:OPENSSL_internal:WRONG_VERSION_NUMBER" I can fix this by creating a destinationrule

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: disable-mtls-mysql-admin
  namespace: test-namespace
spec:
  host: *.test-namespace.svc.cluster.local
  trafficPolicy:
    portLevelSettings:
    - port:
        number: 33061
      tls:
        mode: DISABLE

End-to-end working mTLS

mysql-admin port exists on service : TRUE
excludeInboundPorts                : FALSE

Normally this would result in delayed_connect_error. But I tested that this can be fixed by getting the agent to use localhost instead of the pod's IP addres when communicating with the mysql admin port.

[ymmt2005]

Thank you for the detailed info. Are you interested in creating a pull request?

[vsliouniaev]

Sure - would be nice to make this generally applicable.

Do you mean the localhost implementation in https://github.com/cybozu-go/moco-agent? If so, then this will need to be made optional through some means as binding to localhost will break functionality if running inside istio. Since there are a variety of possible setups I think auto-detection of whether the pod is coming up in an istio mesh could also lead to problems. Would some additional field on the mysqlcluster CRD be the place to do this?

[ymmt2005]

I guess you mean this part of moco-agent, right?

func createConf() error {
        tmpl := template.Must(template.New("my.cnf").Parse(mycnfTmpl))

        v := struct {
                ServerID     uint32
                AdminAddress string
        }{
                ServerID:     config.baseID + config.podIndex,
                AdminAddress: config.podName,
        }

We supply this value with an environment variable, so yes, maybe we can have a switch to give a different environment variable or value.

I'm not sure either of the following would make more sense.

1. Extending MySQLCluster CRD to add the switch for Istio
2. Adding a moco-controller flag to turn on the switch for Istio

[vsliouniaev]

Thanks - I'll do some looking around at what the best way to support this might be.