search

apache / pulsar

Apache Pulsar - distributed pub-sub messaging system
https://pulsar.apache.org/
Apache License 2.0
14.23k stars 3.58k forks source link

Turning on default authorization provider prevents startup in standalone mode. #7209

Closed frankjkelly closed 4 years ago

frankjkelly commented 4 years ago

Describe the bug Turning on authorizationEnabled=true causes Pulsar to be unable to startup

To Reproduce Steps to reproduce the behavior:

  1. Taking a plain vanilla Apache Pulsar 2.5.2
  2. Edit /conf/standalone.conf and set authorizationEnabled=true
  3. Startup Pulsar via /bin/pulsar standalone

Expected behavior Pulsar starts up as normal OR the provided authorization documentation here https://pulsar.apache.org/docs/en/security-authorization/ provides details to work-around this

Screenshots Initially we start to see some warnings . . . 

12:38:37.703 [pulsar-io-50-7] WARN  org.apache.pulsar.broker.web.PulsarWebResource - [persistent://public/functions/assignments] Role null is not allowed to lookup topic
12:38:37.703 [pulsar-io-50-7] WARN  org.apache.pulsar.broker.lookup.TopicLookupBase - Failed to authorized null on cluster persistent://public/functions/assignments
12:38:37.703 [pulsar-client-io-78-1] WARN  org.apache.pulsar.client.impl.BinaryProtoLookupService - [persistent://public/functions/assignments] failed to send lookup request : org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Don't have permission to connect to this namespace

finally after some retries we see

11:13:51.379 [Thread-0] INFO  org.apache.distributedlog.impl.BKNamespaceDriver - Release external resources used by channel factory.
11:13:51.379 [Thread-0] INFO  org.apache.distributedlog.impl.BKNamespaceDriver - Stopped request timer
11:13:51.379 [Thread-0] INFO  org.apache.distributedlog.BKDistributedLogNamespace - Executor Service Stopped.
11:13:51.379 [Curator-Framework-0] INFO  org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11:13:51.492 [Thread-0] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x1000d8b4f810004 closed
11:13:51.492 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x1000d8b4f810004
11:13:51.492 [Thread-0] INFO  org.apache.bookkeeper.proto.BookieServer - Shutting down BookieServer
11:13:51.492 [Thread-0] INFO  org.apache.bookkeeper.proto.BookieNettyServer - Shutting down BookieNettyServer
11:13:51.508 [Thread-0] INFO  org.apache.bookkeeper.bookie.Bookie - Shutting down Bookie-3181 with exitCode 0
11:13:51.508 [Thread-0] INFO  org.apache.bookkeeper.bookie.Bookie - Turning bookie to read only during shut down
11:13:51.508 [Thread-0] INFO  org.apache.bookkeeper.bookie.SyncThread - Shutting down SyncThread
11:13:51.522 [SyncThread-7-1] INFO  org.apache.bookkeeper.bookie.SyncThread - Flush ledger storage at checkpoint CheckpointList{checkpoints=[LogMark: logFileId - 1591374076780 , logFileOffset - 3072]}.
11:13:51.530 [Thread-0] INFO  org.apache.bookkeeper.bookie.Journal - Shutting down Journal
11:13:51.531 [ForceWriteThread] INFO  org.apache.bookkeeper.bookie.Journal - ForceWrite thread interrupted
11:13:51.531 [BookieJournal-3181] INFO  org.apache.bookkeeper.bookie.Journal - Journal exits when shutting down
11:13:51.531 [BookieJournal-3181] INFO  org.apache.bookkeeper.bookie.Journal - Journal exited loop!
11:13:51.531 [Thread-0] INFO  org.apache.bookkeeper.bookie.Journal - Finished Shutting down Journal thread
11:13:51.531 [Bookie-3181] INFO  org.apache.bookkeeper.bookie.Bookie - Journal thread(s) quit.
11:13:51.541 [Thread-0] INFO  org.apache.bookkeeper.bookie.GarbageCollectorThread - Shutting down GarbageCollectorThread
11:13:51.541 [Thread-0] INFO  org.apache.bookkeeper.bookie.EntryLogger - Stopping EntryLogger
11:13:51.543 [Thread-0] INFO  org.apache.bookkeeper.bookie.EntryLoggerAllocator - Stopped entry logger preallocator.
11:13:51.704 [Thread-0] INFO  org.apache.bookkeeper.bookie.LedgerDirsMonitor - Shutting down LedgerDirsMonitor
11:13:51.816 [Thread-0] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x1000d8b4f810001 closed
11:13:51.816 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x1000d8b4f810001
11:13:51.931 [Thread-0] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x1000d8b4f810000 closed
11:13:51.931 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x1000d8b4f810000
11:13:51.931 [Thread-0] INFO  org.apache.zookeeper.server.ZooKeeperServer - shutting down
11:13:51.932 [Thread-0] INFO  org.apache.zookeeper.server.SessionTrackerImpl - Shutting down
11:13:51.932 [Thread-0] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Shutting down
11:13:51.932 [Thread-0] INFO  org.apache.zookeeper.server.SyncRequestProcessor - Shutting down
11:13:51.932 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - PrepRequestProcessor exited loop!
11:13:51.932 [SyncThread:0] INFO  org.apache.zookeeper.server.SyncRequestProcessor - SyncRequestProcessor exited!
11:13:51.933 [Thread-0] INFO  org.apache.zookeeper.server.FinalRequestProcessor - shutdown of request processor complete
11:13:51.938 [ConnnectionExpirer] INFO  org.apache.zookeeper.server.NIOServerCnxnFactory - ConnnectionExpirerThread interrupted
11:13:51.938 [NIOServerCxnFactory.SelectorThread-1] INFO  org.apache.zookeeper.server.NIOServerCnxnFactory - selector thread exitted run method
11:13:51.938 [main-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x1000d8b4f81000b, likely server has closed socket, closing socket connection and attempting reconnect
11:13:51.939 [NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxnFactory - accept thread exitted run method
11:13:51.939 [NIOServerCxnFactory.SelectorThread-0] INFO  org.apache.zookeeper.server.NIOServerCnxnFactory - selector thread exitted run method

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

sijie commented 4 years ago

@frankjkelly

We need to provide documentation for setting up authentication/authorization in standalone. That part is not well documented.

At the same time, you need to configure broker-client authentication parameters. These are used for function workers to talk to brokers for function related operations.

https://github.com/apache/pulsar/blob/master/conf/broker.conf#L535

Or you consider disabling functions in standalone by running standalone using bin/pulsar standalone -nfw.

You can take a look at this video to understand authentication/authorization for deploying a Pulsar cluster. https://www.youtube.com/watch?v=sTISVpyq73o&list=PLqRma1oIkcWhWAhKgImEeRiQi5vMlqTc-&index=10

frankjkelly commented 4 years ago

Awesome - thanks so much @sijie - will watch the video. BTW what does -nfw do I searched for that and could not find it.

jiazhai commented 4 years ago

@frankjkelly -nfw is short for "--no-functions-worker"

frankjkelly commented 4 years ago

Thanks @jiazhai - I saw mention of -nss is that related somehow also?

sijie commented 4 years ago

-nss is not related. -nfw is the one fixed the issue.