Open zuston opened 1 year ago
PTAL @jerqi @xianjingfeng @leixm @smallzhongfeng @kaijchen
Introduce the pluggable disk selection strategy. Currently the disk will be selected based on the hash. Free-capacity based strategy should be supported.
Agreed. Currently the hash based strategy may cause unbalanced disk I/Os among different disks as app's shuffle patterns may vary dramatically. Capacity and disk-stats based strategy is very nice to have.
Introduce the free space & total space metrics of every local disk
@zuston how do you plan to collect these metrics? By using df
, or any other fancy ways?
Interesting feature
Code of Conduct
Search before asking
What would you like to be improved?
I want to raise this issue to improve stability when using MEMORY_LOCALFILE storage type. Maybe some issues will be as sub-tasks in this improvement.
The first improvement is to avoid all apps fail when single disk capacity reaches high-watermark. We could do below optimizations.
How should we improve?
No response
Are you willing to submit PR?