Support Mesos

[exalate-issue-sync[bot]]

NathanM commented: +1 this would be very handy for us!

[exalate-issue-sync[bot]]

Tom Kraljevic commented: No specifics in this jira, so I'm not sure exactly what was intended...

But...

The Krylov project described by Chris Severs at H2O World uses Mesos to get resources allocated for H2O and then uses ZooKeeper for the H2O nodes to find each other and form a cloud.

[exalate-issue-sync[bot]]

NathanM commented: I havent seen his preso (do you have a link?) - but it sounds like what this JIRA intends. Not having seen that work its hard to say.

My understanding would be to make H2O runnable on Mesos, much like Spark is capable of being pointed at a mesos master. http://spark.apache.org/docs/latest/running-on-mesos.html

Or how H2O currently is YARN ready.

[exalate-issue-sync[bot]]

Tom Kraljevic commented: Note that support for H2O on YARN is available now.

See my talk from H2O World here: https://github.com/h2oai/h2o-training/blob/master/tutorials/bigdataenv/H2OinBigDataEnvironments.pdf

and see the documentation here: http://docs.h2o.ai/deployment/hadoop.html

[exalate-issue-sync[bot]]

Tom Kraljevic commented: Added Joel. Joel, where is Chris Severs' Krylov talk available as a replay?

[exalate-issue-sync[bot]]

Sam Bessalah commented: Here's Chris talk https://www.youtube.com/watch?v=lu7_X_Ft-dg

I'm really interested in this.Is there any work being done on this. Or even some standardized docker containre than can H2o and/or Sparkling water within mesos using Marathon.

[exalate-issue-sync[bot]]

Michal Malohlava commented: Hi Sam,Nathan,

i put a simple docker file into sparkling water project (https://github.com/h2oai/sparkling-water/blob/master/docker/sparkling-test/base/Dockerfile). However, it just provides standalone spark cluster + sparkling water.

Nevertheless, if you have any example of docker file for mesos and marathon deployment, we can put there support for sparkling water. It should be straightforward.

[exalate-issue-sync[bot]]

parv oberoi commented: is there any timeline as to when we can expect to have mesos support for sparkling water?

[exalate-issue-sync[bot]]

Tom Kraljevic commented: Parv, there is no current timeline for mesos. The current focus is on YARN and standalone spark.

[exalate-issue-sync[bot]]

parv oberoi commented: @tom kraljevic: can you point me to the code that is responsible for the yarn integration.

[exalate-issue-sync[bot]]

Tom Kraljevic commented: The code is here: h2o-dev/h2o-hadoop

specifically look at h2odriver.java and h2omapper.java

The h2omapper.java constructs an EmbeddedH2OConfig object to help configure the cluster.

In H2O-1, there was zookeeper integration to help with cluster formation for the Krylov mesos framework, but this is not implemented currently in h2o-dev. You might find the documentation helpful. https://github.com/h2oai/h2o/blob/master/docs/uml/zookeeper_integration.md

[exalate-issue-sync[bot]]

parv oberoi commented: thnaks a lot.

[exalate-issue-sync[bot]]

Wade Jensen commented: Also keen for this feature. Hope it can be prioritised.

[hasithjp]

JIRA Issue Migration Info

Jira Issue: PUBDEV-32 Assignee: New H2O Bugs Reporter: Michal Malohlava State: Open Fix Version: N/A Attachments: N/A Development PRs: N/A