search

AutoMQ / kafka-provider-comparison

Apache License 2.0
15 stars 2 forks source link

Kafka Provider Comparision Report #1

Open KaimingWan opened 5 months ago

github-actions[bot] commented 5 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-06-13 11:16:42

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 128000 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 15 warmupDurationMinutes: 10 logIntervalMillis: 1500

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.37:9092,10.0.1.168:9092,10.0.0.160:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.37:9092,10.0.1.168:9092,10.0.0.160:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3

Average Throughput

Average Throughput [AutoMQ]: 500.30 MB/s Average Throughput [Kafka]: 500.23 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost Usage Cost Total Cost
AutoMQ 3.4 15.19 6.54 15.91 34.16 1108.35 1265.0 2373.35
Apache Kafka 4.2 30.21 6.31 12.39 42.31 10647.35 0.0 10647.35
github-actions[bot] commented 5 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-06-17 01:20:56

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 128000 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 15 warmupDurationMinutes: 10 logIntervalMillis: 1500

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.60:9092,10.0.1.146:9092,10.0.0.158:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.60:9092,10.0.1.146:9092,10.0.0.158:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3

Average Throughput

Average Throughput [AutoMQ]: 500.33 MB/s Average Throughput [Kafka]: 500.25 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost Usage Cost Total Cost
AutoMQ 3.4 15.69 6.64 15.54 34.96 1107.35 1264.0 2371.35
Apache Kafka 4.92 37.65 6.88 16.40 50.34 10639.35 0.0 10639.35
github-actions[bot] commented 5 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-06-20 07:38:52

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 128000 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 15 warmupDurationMinutes: 10 logIntervalMillis: 1500

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.80:9092,10.0.1.244:9092,10.0.0.6:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.80:9092,10.0.1.244:9092,10.0.0.6:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3

Average Throughput

Average Throughput [AutoMQ]: 500.18 MB/s Average Throughput [Kafka]: 500.18 MB/s Average Throughput [AWS MSK]: 500.12 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost($) Usage Cost($) Total Cost($)
AutoMQ 3.36 14.6 6.65 15.53 34.93 1107.35 1264.00 2371.35
Apache Kafka 4.8 40.48 7.22 16.21 50.52 10639.35 0.00 10639.35
AWS MSK 164.16 794.8 151.26 439.94 757.91 2312.35 12657.00 14969.35
github-actions[bot] commented 5 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-06-24 04:22:22

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 100 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 1 warmupDurationMinutes: 1 logIntervalMillis: 100

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.91:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.91:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3

Average Throughput

Average Throughput [AutoMQ]: 0.00 MB/s Average Throughput [Kafka]: 0.00 MB/s Average Throughput [AWS MSK]: 0.00 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Elasticity

Take how many seconds to move 30 partitions from one broker to another broker after write 750GB data [AutoMQ]: 3 seconds [Kafka]: 3 seconds

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost($) Usage Cost($) Total Cost($)
AutoMQ 3.05 272.21 3.83 4.29 5.72 -94.65 1263.00 1168.35
Apache Kafka 2.54 314.71 3.05 3.57 7.23 1754.35 0.00 1754.35
AWS MSK 15.34 69.36 68.50 126.65 160.68 551.35 15.00 566.35
github-actions[bot] commented 5 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-06-24 06:31:21

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 100 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 1 warmupDurationMinutes: 1 logIntervalMillis: 100

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.91:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.91:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3

Average Throughput

Average Throughput [AutoMQ]: 0.00 MB/s Average Throughput [Kafka]: 0.00 MB/s Average Throughput [AWS MSK]: 0.00 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Elasticity

Take how many seconds to move 30 partitions from one broker to another broker after write 750GB data [AutoMQ]: 3 seconds [Kafka]: 3 seconds [Amazon MSK]: 3 seconds

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost($) Usage Cost($) Total Cost($)
AutoMQ 2.71 752.37 3.14 3.84 4.88 -94.65 1263.00 1168.35
Apache Kafka 2.28 297.82 2.37 2.86 6.32 1754.35 0.00 1754.35
AWS MSK 16.31 68 70.27 124.31 155.37 551.35 15.00 566.35
github-actions[bot] commented 5 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-06-24 10:21:20

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 25600 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 5 warmupDurationMinutes: 1 logIntervalMillis: 100

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.179:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.179:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3

Average Throughput

Average Throughput [AutoMQ]: 101.00 MB/s Average Throughput [Kafka]: 100.91 MB/s Average Throughput [AWS MSK]: 101.35 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Elasticity

Take how many seconds to move 30 partitions from one broker to another broker after write 36GB data [AutoMQ]: 4 seconds [Kafka]: 4 seconds [Amazon MSK]: 3 seconds

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost($) Usage Cost($) Total Cost($)
AutoMQ 8.08 30.62 12.08 26.96 45.22 277.07 1263.00 1540.07
Apache Kafka 17.98 75.13 16.30 39.97 95.52 2126.07 0.00 2126.07
AWS MSK 93.31 227.51 211.29 356.83 460.66 923.07 2531.00 3455.07
github-actions[bot] commented 4 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-07-01 00:56:32

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 25600 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 5 warmupDurationMinutes: 1 logIntervalMillis: 100

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.161:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.161:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3

Average Throughput

Average Throughput [AutoMQ]: 100.93 MB/s Average Throughput [Kafka]: 101.03 MB/s Average Throughput [AWS MSK]: 101.29 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Elasticity

Take how many seconds to move 30 partitions from one broker to another broker after write 36GB data [AutoMQ]: 4 seconds [Kafka]: 3 seconds [Amazon MSK]: 3 seconds

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost($) Usage Cost($) Total Cost($)
AutoMQ 8.51 30.92 12.13 26.54 44.28 277.07 1263.00 1540.07
Apache Kafka 17.31 69.83 17.04 42.30 96.59 2126.07 0.00 2126.07
AWS MSK 78.05 216.78 220.35 397.10 654.36 923.07 2531.00 3455.07
github-actions[bot] commented 4 months ago

AutoMQ Benchmark VS. Result 🚀

Benchmark Info

Report Generated: 2024-07-08 01:27:42

Workload Configuration

name: 1-topic-1000-partitions-4kb-4p4c-500m topics: 1 partitionsPerTopic: 1000 partitionsPerTopicList: null randomTopicNames: true keyDistributor: NO_KEY messageSize: 4096 useRandomizedPayloads: false randomBytesRatio: 0 randomizedPayloadPoolSize: 0 payloadFile: payload/payload-4Kb.data subscriptionsPerTopic: 1 producersPerTopic: 8 producersPerTopicList: null consumerPerSubscription: 4 producerRate: 25600 producerRateList: null consumerBacklogSizeGB: 0 backlogDrainRatio: 1 testDurationMinutes: 30 warmupDurationMinutes: 5 logIntervalMillis: 500

Producer Configuration

value.serializer: org.apache.kafka.common.serialization.ByteArraySerializer acks: all batch.size: 65536 bootstrap.servers: 10.0.0.128:9092 key.serializer: org.apache.kafka.common.serialization.StringSerializer linger.ms: 1

Consumer Configuration

key.deserializer: org.apache.kafka.common.serialization.StringDeserializer value.deserializer: org.apache.kafka.common.serialization.ByteArrayDeserializer enable.auto.commit: true bootstrap.servers: 10.0.0.128:9092 auto.offset.reset: earliest

Topic Configuration

min.insync.replicas: 1 retention.ms: 86400000 flush.messages: 1

replicationFactor

[AutoMQ] 1 [Kafka] 3 [MSK] 3

Average Throughput

Average Throughput [AutoMQ]: 100.08 MB/s Average Throughput [Kafka]: 100.10 MB/s Average Throughput [AWS MSK]: 100.10 MB/s

Cost Estimate Rule: Check explanation under cost-explanation directory of this repository

Elasticity

Take how many seconds to move 30 partitions from one broker to another broker after write 210 GB data (100 MB/s for 35 minutes) [AutoMQ]: 3 seconds [Kafka]: 4 seconds [Amazon MSK]: 3 seconds

Kafka Provider Pub Latency (ms) avg Pub Latency (ms) P99 E2E LatencyAvg(ms) E2E P95 Latency(ms) E2E P99 Latency(ms) Baseline Cost($) Usage Cost($) Total Cost($)
AutoMQ 6.81 25.36 10.22 19.37 35.26 276.07 1262.00 1539.07
Apache Kafka 12.3 72.54 14.82 37.17 99.12 2124.07 0.00 2124.07
AWS MSK 104.53 361.96 211.21 412.89 590.23 923.07 2531.00 3455.07