search

Cascading / vagrant-cascading-hadoop-cluster

Deploying apache-hadoop in a virtualized cluster as easy as 1-2-3.
127 stars 49 forks source link

Virtual hadoop cluster is provisioned with openjdk-6. #21

Open sfgower opened 8 years ago

sfgower commented 8 years ago

I ran into problems running the examples of Impatient tutorial, and then found that the virtual machines were all provisioned with openjdk-6. Seems awfully old? Someone at Concurrent also just told me that Driven is not compatible with openjdk-6.

Please update the various init.pp files to specify openjdk-7 instead of openjdk-6.

I have made these changes myself, did a vagrant destroy on my virtual machines and am doing a vagrant up. Hopefully that will solve my issues, but the change on the repo could help others out.

fs111 commented 8 years ago

Yes, there is a jdk-7 based branch I meant to merge into the default branch here, but did not have the time to do so yet. I'll try to squeeze that in this week.

sfgower commented 8 years ago

Hi Andre, thanks in advance.  I did a search for init.pp files, then replaced open-jdk-6 with openjdk-6, then did a vagrant destroy, vagrant up and all services came up. I also ran some Cascading examples successfully using the virtual Hadoop cluster.So that does fix the problem. One small thing regarding the documentation. The documentation's README.md file has a problem in its "Starting the cluster".  For users who are new to Vagrant or to Hadoop (or both), they can get lost at this point. Why? The reader has to know to look for these files under the /Vagrant directory. Specifically, they have to navigate to the right point in the directories to execute these commands. If a user doesn't know this, they can get lost. Of course, if you know Vagrant and Hadoop already, that won't happen, but I would expect you want this kind of example to work for people without a lot of experience. It would be helpful if the documentation had a statement telling the user to cd to the right directory, as in.    cd /vagrant/modules/hadoop/files regards, Stefan Gower

  From: André Kelpe <notifications@github.com>

To: Cascading/vagrant-cascading-hadoop-cluster vagrant-cascading-hadoop-cluster@noreply.github.com Cc: Stefan Gower sfgower@yahoo.com Sent: Monday, February 29, 2016 2:41 AM Subject: Re: [vagrant-cascading-hadoop-cluster] Virtual hadoop cluster is provisioned with openjdk-6. (#21)

Yes, there is a jdk-7 based branch I meant to merge into the default branch here, but did not have the time to do so yet. I'll try to squeeze that in this week.— Reply to this email directly or view it on GitHub.

sfgower commented 8 years ago

Hi Andre, I just tried to start hbase on virtual hadoop cluster. The first problem is finding the right 'hbase' to run. If you do a find for 'hbase' files, there are a lot of them!Eventually, I focused on /opt/hbase-0.98.17-hadoop2/bin.Of course, the other files could be links... I then followed the instructions to start hbase, and immediately hit a problem.The hbase script still refers to openjdk-6. I have tried - so far unsuccessfully - to modify the hbase file and then start hbase. So while I have been able to alter my vagrant file and setup to start Hadoop under open-jdk7, I still can't run hbase. regards, Stefan PS - Since you live in Berlin, I wonder: have you visited the new Stone brewery that opened up in Berlin? I know many Germans think that Americans can't make good beer. But if you visit Stone in Berlin, you might find a reason to reconsider. 

  From: Stefan Gower <sfgower@yahoo.com>

To: Cascading/vagrant-cascading-hadoop-cluster reply@reply.github.com; Cascading/vagrant-cascading-hadoop-cluster vagrant-cascading-hadoop-cluster@noreply.github.com Sent: Monday, February 29, 2016 1:46 PM Subject: Re: [vagrant-cascading-hadoop-cluster] Virtual hadoop cluster is provisioned with openjdk-6. (#21)

Hi Andre, thanks in advance.  I did a search for init.pp files, then replaced open-jdk-6 with openjdk-6, then did a vagrant destroy, vagrant up and all services came up. I also ran some Cascading examples successfully using the virtual Hadoop cluster.So that does fix the problem. One small thing regarding the documentation. The documentation's README.md file has a problem in its "Starting the cluster".  For users who are new to Vagrant or to Hadoop (or both), they can get lost at this point. Why? The reader has to know to look for these files under the /Vagrant directory. Specifically, they have to navigate to the right point in the directories to execute these commands. If a user doesn't know this, they can get lost. Of course, if you know Vagrant and Hadoop already, that won't happen, but I would expect you want this kind of example to work for people without a lot of experience. It would be helpful if the documentation had a statement telling the user to cd to the right directory, as in.    cd /vagrant/modules/hadoop/files regards, Stefan Gower

  From: André Kelpe <notifications@github.com>

To: Cascading/vagrant-cascading-hadoop-cluster vagrant-cascading-hadoop-cluster@noreply.github.com Cc: Stefan Gower sfgower@yahoo.com Sent: Monday, February 29, 2016 2:41 AM Subject: Re: [vagrant-cascading-hadoop-cluster] Virtual hadoop cluster is provisioned with openjdk-6. (#21)

Yes, there is a jdk-7 based branch I meant to merge into the default branch here, but did not have the time to do so yet. I'll try to squeeze that in this week.— Reply to this email directly or view it on GitHub.