Open gianniskt opened 3 years ago
Hi @gianniskt! Thanks for filing this issue!
The first error you mentioned is just a standard kubernetes message, you'll just need to specify a container for which you want to see logs: kubectl logs -f <your-pod-name> -c app1
.
The fact that you're seeing envoy-sidecar
in that log messages means that it is connect-injected.
For your second error, generally the reason the pod is unable to find Consul services is that you've not created a Kube Service for your application. This is required. I believe if you'd let that pod error for another minute or two it would've printed an error message suggesting that you do so, and to check the connect webhook deployment logs for additional info!
If you've created a Kube service for your connect application already, then the next place to look is the webhook deployment logs: ex: kubectl logs -f kyle-consul-consul-connect-injector-webhook-deployment-84d5dbq5
.. keeping in mind that there are 2 replicas so one will not be populated.
If you look at the logs of the webhook it may provide additional information.
Oh, to answer your last question, your service does not need to be Headless, but it does need to exist!
Hello @kschoche
Thanks for your answers. Yes my service for the deployment exists. This is the deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: app1
namespace: test
spec:
replicas: 1
selector:
matchLabels:
app: app1
template:
metadata:
namespace: test
labels:
app: app1
spec:
containers:
- name: app1
image: ${DOCKER-REGISTRY-}app1:v1
imagePullPolicy: Always
ports:
- containerPort: 80
And this is the service definition for the Deployment:
apiVersion: v1
kind: Service
metadata:
name: app1-service
namespace: test
spec:
ports:
- port: 80
selector:
app: app1
If I disable connect-inject, everything is fine and my service is accessible through "http://app1-service-test.service.consul"
and Ocelot route the appropriate request to that address. Also at the UI a service with name "app1-service-test"
is created, as I have "addK8SNamespaceSuffix: true"
at Consul Installation.
With Connect Inject enabled, the UI shows the same service but the Check id is "test/app1-697664bd65-5f29g-app1-service/kubernetes-health-check"
and health checks are failing with output: Pod "test/app1-697664bd65-5f29g" is not ready
The logs from connect-inject deployment are showing this:
INFO controller.endpoints retrieved {"name": "app1-service", "ns": "test"}
INFO controller.endpoints registering service with Consul {"name": "app1", "id": "app1-697664bd65-5f29g-app1-service", "agentIP": "10.23.45.44"}
INFO controller.endpoints registering proxy service with Consul {"name": "app1-service-sidecar-proxy"}
INFO controller.endpoints updating health check status for service {"name": "app1-service", "reason": "Pod \"test/app1-697664bd65-5f29g\" is not ready", "status": "critical"}
INFO controller.endpoints updating health check {"id": "test/app1-697664bd65-5f29g-app1-service/kubernetes-health-check"}
Furthermore the POD logs from all containers are showing this output:
[INFO] Registered service has been detected: service=app1
[INFO] Registered service has been detected: service=app1-sidecar-proxy
[INFO] Connect initialization completed
invalid character '\r' in string literal
Error from server (BadRequest): container "app1" in pod "app1-697664bd65-5f29g" is waiting to start: PodInitializing
Unable to find registered services; retrying
Check to ensure a Kubernetes service has been created for this application. If your pod is not starting also check the connect-inject deployment logs.
Just to mention that Consul is installed to different namespace (default) from my application & services (test), but I don't think that it's a problem.
I am not sure if the problem is at the configuration of Ocelot.json or the Service Definition, or maybe a bug of the Envoy Proxy.
Hello,
I have installed the helm chart successfully, and trying to use the connect-inject as clarified here: https://www.consul.io/docs/k8s/connect#installation-and-configuration Using these values:
I am using Ocelot as API Gateway with this configuration on ocelot.json:
Without connect-inject everything is fine, the services are registering on Consul, and the requests are forwarded as expected. By enabling connect-inject, I receive the error from POD Logs:
error: a container name must be specified for pod microservice1, choose one of: [app1 envoy-sidecar] or one of the init containers: [copy-consul-bin consul-connect-inject-init]
Also these are the logs from consul-connect-inject:
I haven' t tried to set annotations on Deployments like "consul.hashicorp.com/connect-service-upstreams: service1:80" as I suppose is not needed, due to "default: true". Is this a bug related to issue #764 , or something else that I am missing on the configuration?
I read somewhere that Kubernetes Services have to be Headless to integrate with Service Mesh (Consul Connect). Is this a possible solution?