search

StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.35k stars 1.69k forks source link

[Enhancement] FE high IO Usage #45263

Open richzhu369 opened 2 months ago

richzhu369 commented 2 months ago

Enhancement

Hello , i got a problem when i using StarRocks, why FE node's I/O Usage always being high load , i try upgrade SSD IOPS and W/R speed , but no effect

image

Can you tell me what should i do for this situation, upgrade Memory or Cpu ? or Change type of aws Ec2

richzhu369 commented 2 months ago

i had upgrade Memory to 64G, but no useful

SakshiArora05 commented 1 week ago

We have developed a web application using PHP for coding, MySQL for the database, and hosting it on AWS Cloud. Recently, after optimizing our database queries, we have encountered high IOPS (Input/Output Operations Per Second) issues. It has become challenging to identify which processes are causing these high IOPS.

We attempted to use iotop to pinpoint the processes responsible for high IOPS and identified some potential culprits. However, despite our efforts, the issue persists. When IOPS are consumed rapidly, our burst balance is quickly depleted, leading to server downtime.

Could you please suggest how we can effectively determine which processes are causing high IOPS? Additionally, could you recommend tools that can help us identify these high IOPS processes?

Furthermore, when our server goes down, we are currently restarting it to bring our application back online. Are there any alternative solutions we should consider to mitigate these issues?