search

streamnative / pulsar-archived

Apache Pulsar - distributed pub-sub messaging system
https://pulsar.apache.org
Apache License 2.0
72 stars 26 forks source link

ISSUE-13302: PIP-117: Change Pulsar standalone defaults #3425

Open sijie opened 2 years ago

sijie commented 2 years ago

Original Issue: apache/pulsar#13302

Motivation

Pulsar standalone is the "Pulsar in a box" version of a Pulsar cluster, where all the components are started within the context of a single JVM process.

Users are using the standalone a as way to get quickly started with Pulsar or in all the cases where it makes sense to have a single node deployment.

Right now, the standalone is starting by default many components, several of which are quite complex, since they are designed to be deployed in a distributed fashion.

Goal

Simplify the components of Pulsar standalone to achieve:

  1. Reduce complexity
  2. Reduce startup time
  3. Reduce memory and CPU footprint of running standalone

Proposed changes

The proposal here is to change some of the default implementations that are used for the Pulsar standalone.

  1. Metadata Store implementation --> Change from ZooKeeper to RocksDB

  2. Pulsar functions package backend --> Change from using DistributedLog to using local filesystem, storing the jars directly in the data folder instead of uploading them into BK.

  3. Pulsar functions state store implementation --> Change the state store to be backed by a MetadataStore based backed, with the RocksDB implementation.

  4. Table Service --> Do not start BK table service by default

Compatibility considerations

In order to avoid compatibility issues where users have existing Pulsar standalone services that they want to upgrade without conflicts, we will follow the principle of keeping the old defaults where there is existing data on the disk.

We will add a file, serving the purpose as a flag, in the data/standalone directory, for example new-2.10-defaults.

If the file is present, or if the data directory is completely missing, we will adopt the new set of default configuration settings.

If the file is not there, we will continue to use existing defaults and we will not break the upgrade operation.

github-actions[bot] commented 2 years ago

The issue had no activity for 30 days, mark with Stale label.