apache / incubator-seata

:fire: Seata is an easy-to-use, high-performance, open source distributed transaction solution.
https://seata.apache.org/
Apache License 2.0
25.35k stars 8.79k forks source link

rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1的cpu使用率过高 #6041

Open gushuyong opened 1 year ago

gushuyong commented 1 year ago

Ⅰ. Issue Description

seata版本:1.4.2 服务配置:-Xms512m -Xmx1024m -XX:MaxNewSize=128m 因一直收到线上某服务cpu使用率过高的告警,就先nacos进行下线,准备重启大法,发现还是一直告警。然后arthas排查,最后发现无流量情况下, rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1这2线程的cpu使用率过高

Ⅱ. Describe what happened

"rpcMergeMessageSend_TMROLE_1" Id=31 cpuUsage=31% TIMED_WAITING on java.lang.Object@1c0c220d at java.lang.Object.wait(Native Method)

"rpcMergeMessageSend_RMROLE_1" Id=34 cpuUsage=30% TIMED_WAITING on java.lang.Object@25b70c1e at java.lang.Object.wait(Native Method)

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. xxx
  2. xxx
  3. xxx

Minimal yet complete reproducer code (or URL to code):

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

funky-eyes commented 1 year ago

我有三个想法: 1.不再每1ms 进行wait,直接以10ms或者更大的值进行wait,反正sendSyncRequest进来将消息放入队列的时候一定会唤醒线程 2.tm和rm的线程合并掉,不需要单独一个线程 3.直接改造,用批量write后再一次性flush达到批量的效果,不再设置单独的线程 I have three ideas:

  1. no longer wait every 1ms, directly wait with 10ms or greater, anyway, sendSyncRequest come in to put the message into the queue will wake up the thread
  2. tm and rm threads are merged off, do not need a separate thread
  3. directly remodeling, with a batch of write and then a one-time flush to achieve the effect of batch of effect, no longer set up a separate thread
PleaseGiveMeTheCoke commented 12 months ago

@funky-eyes 关于你的前两个想法,我提交了一个pr,能请你帮忙code review 一下吗

funky-eyes commented 12 months ago

@funky-eyes 关于你的前两个想法,我提交了一个pr,能请你帮忙code review 一下吗

非常欢迎你的贡献,在2.0发布完成后,我将进行review Thank you very much for your contribution. After the completion of version 2.0, I will review it

wt-better commented 11 months ago

Using seata version 1.4.2, empty traffic, I have not reproduced your problem, can you provide a demo code to replay.