search

apache / pulsar-helm-chart

Official Apache Pulsar Helm Chart
https://pulsar.apache.org/
Apache License 2.0
212 stars 224 forks source link

The throughput of the consumer side is only about 10M #522

Open JuddOslv opened 3 months ago

JuddOslv commented 3 months ago

Describe the bug The throughput of the consumer side is only about 10M. Whether it is to modify the queuesize of the consumer side or to add more consumer nodes, it is of no use.

To Reproduce

  1. Install pulsar to k8s cluster through helm and test normal production and consumption functions
  2. Modify some configuration parameters in helm chart and some initialization scripts, such as the authorization information of pulsar manager user
  3. It looks better after deployment. The pulsar manager background interface can be opened. Except for a few functions with problems, everything looks good
  4. Start developing consumer server code and prepare for testing
  5. The local environment runs in standalone mode, which is very fast. Each batch of 2,000 messages can be obtained, and the next batch can be obtained immediately
  6. But it doesn't work when deployed online. The production end can write more than 100MB, but the consumer end is only about 10M

Expected behavior The consumer end is expanded from 10 to more than 100, and the consumption throughput has not changed. Modify the consumer end code such as receiverQueueSize and maxNumMessages, and just print the number of messages obtained on the consumer end. Without any business logic, the throughput still cannot be increased

Screenshots image

Desktop (please complete the following information): broker: 8c16g 100g ssd proxy: 8c16g 100g ssd bookie: 8c32g 1.5T essd * 3

Additional context