pooling: query YARN resource manager for available memory and CPU

Yelp / mrjob

Run MapReduce jobs on Hadoop or Amazon Web Services

http://packages.python.org/mrjob/

Other

2.62k stars 586 forks source link

pooling: query YARN resource manager for available memory and CPU #2197

Closed coyotemarin closed 4 years ago

coyotemarin commented 4 years ago

This change allows you to replace looking at a cluster's number and type of instances with connecting directly to the YARN resource manager and querying it for available memory and CPU. Fixes #2191.

88manpreet commented 4 years ago