sodafoundation / delfin

delfin is the SODA Infrastructure Manager project which provides unified, intelligent and scalable resource management, alert and performance monitoring
https://sodafoundation.io/
Apache License 2.0
202 stars 355 forks source link

Performance collection optimization. #917

Closed JiuyunZhao closed 2 years ago

JiuyunZhao commented 2 years ago

What this PR does / why we need it:

  1. metric constants optimization.
  2. add parameter collect_interval and failed_job_collect_interval to the return value of method get_capabilities, so we use custom interval time for different drivers.
  3. add metric CPU usage for the controller.
  4. change HostAddressOpt to HostnameOpt for the hostname.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

Release note:

codecov[bot] commented 2 years ago

Codecov Report

Merging #917 (2e4f163) into master (505078a) will increase coverage by 0.01%. The diff coverage is 97.53%.

@@            Coverage Diff             @@
##           master     #917      +/-   ##
==========================================
+ Coverage   71.11%   71.13%   +0.01%     
==========================================
  Files         182      182              
  Lines       21904    21931      +27     
  Branches     3340     3343       +3     
==========================================
+ Hits        15578    15600      +22     
- Misses       5268     5270       +2     
- Partials     1058     1061       +3     
Impacted Files Coverage Δ
delfin/common/config.py 95.00% <ø> (ø)
delfin/drivers/driver.py 62.50% <ø> (ø)
...dulers/telemetry/performance_collection_handler.py 90.27% <0.00%> (-2.58%) :arrow_down:
delfin/common/constants.py 100.00% <100.00%> (ø)
delfin/task_manager/perf_job_controller.py 61.29% <100.00%> (ø)
...ager/scheduler/schedulers/telemetry/job_handler.py 68.38% <100.00%> (ø)
delfin/drivers/fake_storage/__init__.py 93.65% <0.00%> (-0.77%) :arrow_down: