search

fabric8io / openshift-elasticsearch-plugin

Apache License 2.0
27 stars 21 forks source link

OpenshiftAPIService use cache #172

Closed Gallardot closed 5 years ago

Gallardot commented 5 years ago

If openshiftAPIService use cache, then we can reduce access to openshift.

We access to openshift for three things:

1.get username by token 2.get projects by token 3.check the SAR is satisfied

We don't think they will change frequently, so we can cache it.

Entries in "the cache" should expire after O(minute)，so we can use the same as "ACLExpiresInMillis"

richm commented 5 years ago

If openshiftAPIService use cache, then we can reduce access to openshift.

Are you having problems with excessive openshift api usage, or Elasticsearch performance? If so, what do you use openshift-elasticsearch-plugin for?

We access to openshift for three things:

1.get username by token 2.get projects by token 3.check the SAR is satisfied

We don't think they will change frequently, so we can cache it.

Entries in "the cache" should expire after O(minute)，so we can use the same as "ACLExpiresInMillis"

Is this an openshift setting?

Gallardot commented 5 years ago

Are you having problems with excessive openshift api usage, or Elasticsearch performance? If so, what do you use openshift-elasticsearch-plugin for?

We use openshift-elasticsearch-plugin in the openshift-elasticsearch-plugin project on an openshift cluster.About 100 developer use kibana to search project logs in our cluster everyday.Access ES directly performance is OK.But use kibana to Access ES performance is not so good.So I want to optimize the searchguard or the openshift-elasticsearch-plugin every such possibility.

Entries in "the cache" should expire after O(minute)，so we can use the same as "ACLExpiresInMillis"

Is this an openshift setting?

I mean the openshift.acl.expire_in_millis an openshift-elasticsearch-plugin setting.

@richm

Our team had do some work to optimize the performance. we will pull requests later.

jcantrill commented 5 years ago

@Gallardot This plugin used to have a cache mechanism which caused numerous issues because of its design. We removed it under the assumption that use of Elasticsearch was significantly more write intensive then query. I have no issues re-implementing the cache to save the api hits you identify.

I am not certain you will be able to make use of the SG cache as identifed in https://github.com/fabric8io/openshift-elasticsearch-plugin/issues/173; we have some hackery where we perform work before and after the request is run through searchguard. I would suggest implementing our own. We have talked about moving all this logic outside of this plugin so I don't know what value there is to pursuing any large, major changes.

Gallardot commented 5 years ago

@jcantrill

This plugin used to have a cache mechanism which caused numerous issues because of its design.

I agree with you. But sometimes it depends on the situation.

some log from our cluster.

[2019-04-11T06:49:03,045][DEBUG][i.f.e.p.OpenshiftRequestContextFactory] Handling Request... /_msearch
[2019-04-11T06:49:03,045][DEBUG][i.f.e.p.OpenshiftRequestContextFactory] Evaluating request for user '' with a non-empty token
[2019-04-11T06:49:03,107][DEBUG][i.f.e.p.OpenshiftAPIService] Response: code '200' {"kind":"User","apiVersion":"user.openshift.io/v1","metadata":{"name":"tanhl","selfLink":"/apis/user.openshift.io/v1/users/tanhl","uid":"9f6e716f-5202-11e9-ae33-005056b07597","resourceVersion":"20046255","creationTimestamp":"2019-03-29T09:11:22Z"},"identities":["htpasswd_auth:tanhl"],"groups":["system:authenticated","system:authenticated:oauth"]}

[2019-04-11T06:49:03,112][DEBUG][i.f.e.p.OpenshiftAPIService] Request: Request{method=POST, url=https://172.31.0.1/apis/authorization.openshift.io/v1/subjectaccessreviews, tag=null}
[2019-04-11T06:49:03,113][DEBUG][i.f.e.p.OpenshiftAPIService] Request body: {"kind":"SubjectAccessReview","apiVersion":"authorization.openshift.io/v1","verb":"view","scopes":[],"resourceAPIGroup":"","resource":"pods/log","namespace":"default"}
[2019-04-11T06:49:03,164][DEBUG][i.f.e.p.OpenshiftAPIService] Response: Response{protocol=http/1.1, code=201, message=Created, url=https://172.31.0.1/apis/authorization.openshift.io/v1/subjectaccessreviews}
[2019-04-11T06:49:03,164][DEBUG][i.f.e.p.OpenshiftAPIService] Response body: {"kind":"SubjectAccessReviewResponse","apiVersion":"authorization.openshift.io/v1","namespace":"default","allowed":true,"reason":"RBAC: allowed by ClusterRoleBinding \"cluster-admin-1\" of ClusterRole \"cluster-admin\" to User \"tanhl\""}

[2019-04-11T06:49:03,164][DEBUG][i.f.e.u.RequestUtils     ] User '<UNKNOWN>' isOperationsUser: true
[2019-04-11T06:49:03,223][DEBUG][c.f.s.a.BackendRegistry  ] User 'tanhl' is in cache? false (cache size: 2)
[2019-04-11T06:49:03,224][DEBUG][c.f.s.a.BackendRegistry  ] tanhl not cached, return from io.fabric8.elasticsearch.plugin.auth.OpenShiftTokenAuthentication backend directly
[2019-04-11T06:49:03,229][DEBUG][i.f.e.p.OpenshiftAPIService] Request: Request{method=POST, url=https://172.31.0.1/apis/authorization.openshift.io/v1/subjectaccessreviews, tag=null}
[2019-04-11T06:49:03,229][DEBUG][i.f.e.p.OpenshiftAPIService] Request body: {"kind":"SubjectAccessReview","apiVersion":"authorization.openshift.io/v1","verb":"view","scopes":[],"resourceAPIGroup":"metrics.openshift.io","resource":"prometheus","namespace":"openshift-logging"}
[2019-04-11T06:49:03,280][DEBUG][i.f.e.p.OpenshiftAPIService] Response: Response{protocol=http/1.1, code=201, message=Created, url=https://172.31.0.1/apis/authorization.openshift.io/v1/subjectaccessreviews}
[2019-04-11T06:49:03,280][DEBUG][i.f.e.p.OpenshiftAPIService] Response body: {"kind":"SubjectAccessReviewResponse","apiVersion":"authorization.openshift.io/v1","namespace":"openshift-logging","allowed":true,"reason":"RBAC: allowed by ClusterRoleBinding \"cluster-admin-1\" of ClusterRole \"cluster-admin\" to User \"tanhl\""}

[2019-04-11T06:49:03,280][DEBUG][c.f.s.a.BackendRegistry  ] User 'User [name=tanhl, roles=[gen_user_b489efc3dbc1875886a37de5698d05b96fc08de7, gen_kibana_b489efc3dbc1875886a37de5698d05b96fc08de7, gen_project_operations, prometheus]]' is authenticated
[2019-04-11T06:49:03,280][DEBUG][c.f.s.a.BackendRegistry  ] sg_tenant 'null'
[2019-04-11T06:49:03,281][DEBUG][i.f.e.u.RequestUtils     ] Modifying header 'X-Proxy-Remote-User' to be 'tanhl'
[2019-04-11T06:49:03,281][DEBUG][i.f.e.u.RequestUtils     ] Handling Request... /_msearch
[2019-04-11T06:49:03,281][DEBUG][i.f.e.u.RequestUtils     ] Evaluating request for user 'tanhl' with a non-empty token

openshift-elasticsearch-plugin access to openshift for three things:

1.get username by token 2.get projects by token 3.check the SAR is satisfied

It takes about 200 milliseconds to do the above work. But we takes only 200 milliseconds to access Elasticsearch directly.

jcantrill commented 5 years ago

Closing this issue. I don't understand why we need three to track fundamentally the same thing.