yahoo / storm-yarn

Storm-yarn enables Storm clusters to be deployed into machines managed by Hadoop YARN.
Other
417 stars 161 forks source link

Support Apache Storm 1.0.1 #85

Closed fuzhengjia closed 5 years ago

fuzhengjia commented 7 years ago

The Apache Storm has leased its 1.0.x version for a while, however the corresponding storm-yarn project has not been updated for a long time. Therefore, we believe that it is necessary to make some updates. In this pull reqeust, we have mainly done the following changes:

  1. Updated the version of Apache Storm from 0.9.0 to 1.0.1;
  2. Added a new StormClusterChecker class, in order to monitor the storm cluster. It can adjust the number of supervisors based on the usage of system resources;
  3. Based on (2), a new storm command "removeSupervisors" for removing idle/useless supervisors was added;
  4. Updated the logging framework from logback to log4j2.

There are also some minor changes not listed here. Please feel free to let us know your comments and feedbacks.

Best regards and thanks, Data analytics team at ADSC