search

uber / uReplicator

Improvement of Apache Kafka Mirrormaker
Apache License 2.0
917 stars 199 forks source link

Federation Mode ERROR:java heap space #323

Closed soraka-hu closed 3 years ago

soraka-hu commented 3 years ago

hi: i have already published ureplicator federation mode online. but now, i get a log of error:

微信图片_20201208214800 微信图片_20201208214543

This is my configuration:

微信图片_20201208214627 微信图片_20201208214633

I start worker using -Xmx=20g -Xms=20g, I think it's enough to start that resource.but i don't no what happened?

@xhl1988 please help me.

xhl1988 commented 3 years ago

How many partition and how much traffic does your worker handles?

soraka-hu commented 3 years ago

I have 18 partitions and 500000 traffic per second. If the reason is that the throughput of the producer is relatively small and the throughput of the consumer is relatively large, the queue(queued.max.message.chunks=1000) is squeezing too much data, causing oom? @xhl1988

yangy0000 commented 3 years ago

queued.max.message.chunks=1000 is a bit too big. 5-10 is generally enough

soraka-hu commented 3 years ago

ok,l already modify this setting,this problem has been solved. thank you very much. l prepare close this issue.