Attempting to use token authentication in pulsar standalone causes it to exit

[rmbellovin]

Describe the bug I am trying to run pulsar standalone using token-based authentication. But when I add the line

authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken to conf/standalone.conf, running bin/pulsar standalone causes it to exit almost immediately.

To Reproduce Steps to reproduce the behavior:

1. In conf/standalone.conf, add the lines

   authenticationEnabled=true authorizationEnabled=true authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken tokenSecretKey=file:///path/to/secret.key (as directed by https://pulsar.apache.org/docs/en/security-jwt/)

2. Run bin/pulsar standalone.

Expected behavior I expected pulsar to start in standalone mode. Instead, it exited, and the log file contains errors such as

[main] ERROR org.apache.pulsar.functions.worker.WorkerService - Error Starting up in worker org.apache.pulsar.client.admin.PulsarAdminException$NotAuthorizedException : HTTP 401 Unauthorized and java.lang.NoClassDefFoundError: Could not initialize class io.netty.channel.epoll.EpollEventLoop

Desktop (please complete the following information): Ubuntu 18.04

Additional context I have attached a log file (from running bin/pulsar-daemon start standalone instead): pulsar-standalone-gideon.log

[david-streamlio]

You need to add the following properties to your conf/standalone.conf file

JWT Auth

authenticationEnabled=true authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken tokenPublicKey=file:///PATH TO/my-public.key

Authorization

superUserRoles=admin anonymousUserRole=anonymous

[david-streamlio]

FWIW, the error in the attached log is related to ZK.

13:58:24.305 [Curator-LeaderSelector-0] ERROR org.apache.curator.framework.recipes.leader.LeaderSelector - The leader threw an exception java.lang.InterruptedException: null at java.lang.Object.wait(Native Method) ~[?:1.8.0_242] at java.lang.Object.wait(Object.java:502) ~[?:1.8.0_242] at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1529) ~[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0] at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1512) ~[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0] at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:1791) ~[org.apache.pulsar-pulsar-zookeeper-2.5.0.jar:2.5.0] at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:274) ~[org.apache.curator-curator-framework-4.0.1.jar:4.0.1] at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:268) ~[org.apache.curator-curator-framework-4.0.1.jar:4.0.1] at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64) ~[org.apache.curator-curator-client-4.0.1.jar:?] at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100) ~[org.apache.curator-curator-client-4.0.1.jar:?] at org.apache.curator.framework.imps.DeleteBuilderImpl.pathInForeground(DeleteBuilderImpl.java:265) ~[org.apache.curator-curator-framework-4.0.1.jar:4.0.1] at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:249) ~[org.apache.curator-curator-framework-4.0.1.jar:4.0.1] at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:34) ~[org.apache.curator-curator-framework-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath(LockInternals.java:347) ~[org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(LockInternals.java:124) ~[org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(InterProcessMutex.java:154) ~[org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:449) [org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466) [org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65) [org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246) [org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240) [org.apache.curator-curator-recipes-4.0.1.jar:4.0.1] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_242] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_242] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]

[david-streamlio]

What changes did you make to the conf/client.conf file? It should be something similar to the following

JWT Authentication

authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken authParams=file:///PATH TO/admin-token.txt

[sijie]

@rmbellovin : I see. It failed related to function worker authentication. there are two options:

a) disable function worker in your standalone first. You can do it by running bin/pulsar standalone -nfw -nss.

b) you can configure conf/standalone.conf to enable authentication for broker client (https://github.com/apache/pulsar/blob/master/conf/broker.conf#L405). That is the one used by function worker contacting brokers.

[sijie]

brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=

You can configure these two settings just as how you will configure the normal client talking to a broker.

[rmbellovin]

Thanks very much, disabling function workers did the trick. That's fine for my purposes, but out of curiosity, if I wanted to enable authentication for broker clients, what role would I need to generate a token for?

[sijie]

@rmbellovin

if I wanted to enable authentication for broker clients, what role would I need to generate a token for?

The broker client is used for inter-broker communication, and function-worker and broker communication. So ideally the role should have a "super-user" permission if you enable authorization.

[rmbellovin]

Thanks very much for your help!

[sijie]

Closed this issue since the question has been answered.

[YouJiacheng]

@sijie

brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=

You can configure these two settings just as how you will configure the normal client talking to a broker.

In standalone mode, do we need to configure functions_worker.yml? I see these two setting in functions_worker.yml too.