search

zerda / kube-fluentd-cloudwatch

Collecting Docker Log Files with Fluentd and AWS CloudWatch
9 stars 7 forks source link

Debugging with AdmissionController #2

Open zanhsieh opened 6 years ago

zanhsieh commented 6 years ago

Hi,

I try to work on kube-fluentd-cloudwatch compatibility with AdmissionController. The K8S official document lists below:

https://kubernetes.io/docs/admin/admission-controllers/

My K8S cluster version is 1.8.7, and running on AWS with KOPS. The topology is very simple: 1x master + 1x node, due to it currently is a development cluster.

I tried to apply the best practice as document suggested above:

For Kubernetes >= 1.6.0, we strongly recommend running the following set of admission controllers (order matters): NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds

But it seems having problem. After pulling images, then it stuck there. Before I apply the best practice of AdmissionController, everything worked fine.

Just curious how I should troubleshoot this issue. Please advice and thanks.

zerda commented 6 years ago

What do you mean stuck, Is fluented stucked, and there is no logs ?

I am not familiar with AdmissionController, but I would try to add each controller one by one, and see what is going on.

zanhsieh commented 6 years ago

@zerda Yes, just as you described. I look for any method to speed up the trial-n-error process, due to the order of AdmissionControllers does also matter, which would be 5040 (7!) possibilities.

zerda commented 6 years ago

I am preferred not to change the order at all, as the controllers is executed one by one. If any of the controllers in either phase reject the request, the entire request is rejected immediately and an error is returned to the end-user.

Just following the current order, add each one, and check.

zanhsieh commented 6 years ago

Sorry for late reply. When I start only with NamespaceLifecycle, I got this log:

2018-04-04 08:41:28 +0000 [error]: config error file="/etc/fluent/fluent.conf" 
error="Invalid Kubernetes API v1 endpoint https://100.64.0.1:443/api: SSL_connect returned=1 errno=0 state=error: certificate verify failed"