streamnative / kop

Kafka-on-Pulsar - A protocol handler that brings native Kafka protocol to Apache Pulsar
https://streamnative.io/docs/kop
Apache License 2.0
450 stars 136 forks source link

Optimize authorization by caching authorization results #1999

Closed BewareMyPower closed 1 year ago

BewareMyPower commented 1 year ago

Motivation

To follow Kafka's behavior, KoP also performs authorization for each PRODUCE or FETCH request. If the custom authorization provider is slow to authorize produce or consume permissions, the performance will be impacted.

Modifications

Introduce caches for authorization:

Add SlowAuthorizationTest to verify the producer and consumer won't be affected significantly by slow authorization.

Introduce two configs to configure the cache policy so that revoke permission can work:

Documentation

Check the box below.

Need to update docs?

codecov[bot] commented 1 year ago

Codecov Report

Merging #1999 (9ba3925) into master (5193592) will increase coverage by 0.03%. The diff coverage is 25.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1999      +/-   ##
============================================
+ Coverage     17.03%   17.06%   +0.03%     
- Complexity      727      731       +4     
============================================
  Files           191      191              
  Lines         14251    14277      +26     
  Branches       1337     1340       +3     
============================================
+ Hits           2428     2437       +9     
- Misses        11646    11663      +17     
  Partials        177      177              
Files Changed Coverage Δ
...andlers/kop/security/auth/SimpleAclAuthorizer.java 0.00% <0.00%> (ø)
...pulsar/handlers/kop/KafkaServiceConfiguration.java 76.34% <70.00%> (-0.37%) :arrow_down:

... and 1 file with indirect coverage changes

BewareMyPower commented 1 year ago

@Demogorgon314 Done.