Is it possible to use mesosphere-hdfs for hadoop?

mesosphere-backup / hdfs-deprecated

[DEPRECATED] This project is deprecated. It will be archived on December 1, 2017.

Apache License 2.0

147 stars 52 forks source link

Is it possible to use mesosphere-hdfs for hadoop? #152

Open JingzhouLuo opened 9 years ago

JingzhouLuo commented 9 years ago

Hi,

We have a mesosphere cluster running on EC2 with hdfs service installed. I was able to issue "hadoop jar" command to start a mapreduce job. And it looks like the installation comes with full CDH5 distributions.

This makes us thinking if it is possible to run hadoop mapreduce job with this setting. If this is doable, what is the right practice to setup?

Thanks, Jingzhou

elingg commented 9 years ago

Glad to hear it's working for you @JingzhouLuo! MR-1 should work. If you want to run MR-2 (Yarn), I suggest checking out Myriad (www.github.com/mesos/myriad) which runs on top of HDFS.