[Bug] [Rest-API] /hazelcast/rest/maps/running-jobs very slowly When there are more than 50 CDC tasks running

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

https://seatunnel.apache.org/

Apache License 2.0

8.03k stars 1.82k forks source link

[Bug] [Rest-API] /hazelcast/rest/maps/running-jobs very slowly When there are more than 50 CDC tasks running #6875

Open W-dragan opened 5 months ago

W-dragan commented 5 months ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

When there are more than 50

CDC tasks running, using the/hazelcast/rest/maps/running jobs rest interface to query the number of running tasks results in a very slow response, requiring more than one minute

SeaTunnel Version

2.3.5

SeaTunnel Config

none

Running Command

none

Error Exception

none

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

W-dragan commented 5 months ago

@Hisoka-X @liugddx help cc，/hazelcast/rest/maps/finished-jobs There is also a similar problem, but it is not as obvious

Hisoka-X commented 5 months ago

Have you checked jvm heap usage?

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

W-dragan commented 4 months ago

Have you checked jvm heap usage?

After investigation, it was found that the first few minutes after rolling restart would be very slow. It should be loading the corresponding class jar package, and then it became faster. However, during capacity testing, it was found that when running around 200 CDC tasks, querying the running tasks is not very fast, taking an average of 5-8 seconds, occasionally 20 seconds