search

obsidiandynamics / kafdrop

Kafka Web UI
Apache License 2.0
5.53k stars 838 forks source link

Internal server error: Not authorized to access topics: [Topic authorization failed.] #115

Closed jrevillard closed 4 years ago

jrevillard commented 4 years ago

Hi,

We have a Kafka cluster with Kafka v1.0 protected by Apache Ranger. We use SASL_SSL with GSSAPI sasl mechanism. I see in the ranger audit logs that Kafdrop contact properly the cluster... all the connections are allowed (no denied at all) but in Kafdrop, I get this:

Internal Server Error
A 500 error has occurred: Request processing failed; nested exception is kafdrop.service.KafkaAdminClientException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [Topic authorization failed.]
Stack trace
kafdrop.service.KafkaAdminClientException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [Topic authorization failed.] at kafdrop.service.KafkaHighLevelAdminClient.describeTopicConfigs(KafkaHighLevelAdminClient.java:114) at kafdrop.service.KafkaMonitorImpl.getTopicMetadata(KafkaMonitorImpl.java:121) at kafdrop.service.KafkaMonitorImpl.getTopic(KafkaMonitorImpl.java:113) at kafdrop.controller.TopicController.topicDetails(TopicController.java:42) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:567) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897) at javax.servlet.http.HttpServlet.service(HttpServlet.java:645) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) at kafdrop.config.CorsConfiguration$1.doFilter(CorsConfiguration.java:88) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:835) Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [Topic authorization failed.] at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260) at kafdrop.service.KafkaHighLevelAdminClient.describeTopicConfigs(KafkaHighLevelAdminClient.java:103) ... 79 more Caused by: org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [Topic authorization failed.]

Do you have an idea ?

Best, Jerome

ekoutanov commented 4 years ago

SASL_SSL is an authentication mechanism, which means that it will let you successfully connect to a broker provided you have a valid set of credentials; it doesn't check (at the point of connection) whether you are authorised to perform any action on the broker(s).

The error you are seeing owes to a lack of privileges. You need to explicitly grant access to the Kafdrop user to access the topics that you want to browse, as well as consumer groups.

jrevillard commented 4 years ago

Hi @ekoutanov , thanks for your help. Let me describe more in detail. Here is my kafdrop configuration:

security.protocol=SASL_SSL
#ssl.endpoint.identification.algorithm=
ssl.truststore.location=/conf/certificate_authorities.jks
ssl.truststore.password=xxxxxx
sasl.mechanism=GSSAPI
sasl.jaas.config="com.sun.security.auth.module.Krb5LoginModule required \
     useTicketCache=true \
     renewTicket=true \
     serviceName=\"kafka\" \
     useKeyTab=true keyTab=\"/conf/jrevillard.keytab\" \
     principal=\"jrevillard@BIGEYS.PRIV\";"

In Ranger, no problem at all, Kafdrop was able to do everything apparently, I see plenty of "describe" operation which succeed without problem:

Capture d’écran du 2020-04-28 10-24-44

Therefore, I have no idea what is the issue. My user is admin of the kafka cluster and can do everything.

Best, Jerome

ekoutanov commented 4 years ago

Hi @jrevillard . I'm not sure what the specific issue is with authorization (and I'm no Kerberos expert), but I can confirm pretty definitively that a TopicAuthorizationException in Kafka means that the client attempted to execute an operation on the topic for which it does not have adequate permission. It might be a DESCRIBE, or a READ.

This has little to do with Kafdrop; more with how Kafka authorization works.

What I can suggest, is to enable fine-grained logging on your Kafka cluster to see where access is being denied. Alternatively, you may try a different authentication scheme just for experimentation, for example SASL with SCRAM and just to check that authorization has been set up correctly (to eliminate Kerberos as a potential issue).

jrevillard commented 4 years ago

I found exactly where the problem is by running in DEBUG mode In fact the problem is adminClient.describeConfigs(resources);

Kafdrop is able to properly retrieve the topic list in fact: 

kafdrop_1  | 2020-04-29 10:27:18.936 DEBUG 14 [  XNIO-1 task-1] o.a.k.c.NetworkClient                    : [Consumer clientId=kafdrop-consumer, groupId=null] Using older server API v5 to send METADATA {topics=[{name=topic1}],allow_auto_topic_creation=true} with correlation id 4 to node 1008
kafdrop_1  | 2020-04-29 10:27:18.996 DEBUG 14 [  XNIO-1 task-1] o.a.k.c.NetworkClient                    : [Consumer clientId=kafdrop-consumer, groupId=null] Using older server API v5 to send METADATA {topics=[{name=topic2}],allow_auto_topic_creation=true} with correlation id 7 to node 1008
kafdrop_1  | 2020-04-29 10:27:19.056 DEBUG 14 [  XNIO-1 task-1] o.a.k.c.NetworkClient                    : [Consumer clientId=kafdrop-consumer, groupId=null] Using older server API v5 to send METADATA {topics=[{name=topic3}],allow_auto_topic_creation=true} with correlation id 8 to node 1008
....

Best