search

Intel-bigdata / HiBench

HiBench is a big data benchmark suite.
Other
1.46k stars 766 forks source link

Running HiBench workloads in Yarn-cluster mode #197

Open srikrishnacancun opened 8 years ago

srikrishnacancun commented 8 years ago

Is HiBench supported for Yarn-Cluster mode for any of the workloads .?

srikrishnacancun commented 8 years ago

Look for terrasort ,KMeans, PageRank workloads support for Yarn-Cluster mode

yncxcw commented 8 years ago

yes, the default HiBench running on Spark is under Yarn-Client mode

srikrishnacancun commented 8 years ago

Hi

When I change the Hibench bench conf file to point to Yarn-Cluster and ran the wordcount workload and observed the Spark UI, I don't see the driver program invoked on one of the datanodes. Is there a different procedure.?

Looking forward in hearing back.

Thanks

Srikrishna

On Fri, Apr 29, 2016 at 1:24 AM, yncxcw notifications@github.com wrote:

yes, the default HiBench running on Spark is under Yarn-Cluster mode

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/intel-hadoop/HiBench/issues/197#issuecomment-215543213

carsonwang commented 8 years ago

Hi, currently Yarn-cluster mode is not supported by HiBench.