mkdir: cannot create directory - setup-hadoop.sh

[jhajek]

On line 210 of setup-hadoop.sh this is the output that follows. I tried editing diskmaster.sh to add -p to the mkdir and also added /mnt in front of /vol-01 incase It was a permission issue. What is trying to be done here?

qemu: Stopping namenodes on [master] qemu: master: Warning: Permanently added 'master,10.0.2.15' (ECDSA) to the list of known hosts. qemu: master: mkdir: cannot create directory ‘/mnt/vol-01’: Permission denied qemu: master: chown: cannot access ‘/mnt/vol-01/hadooplogs/’: No such file or directory qemu: master: no namenode to stop qemu: master: Warning: Permanently added 'master,10.0.2.15' (ECDSA) to the list of known hosts. qemu: master: mkdir: cannot create directory ‘/mnt/vol-01’: Permission denied qemu: master: chown: cannot access ‘/mnt/vol-01/hadooplogs/’: No such file or directory qemu: master: no datanode to stop qemu: Stopping secondary namenodes [0.0.0.0] qemu: 0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts. qemu: 0.0.0.0: mkdir: cannot create directory ‘/mnt/vol-01’: Permission denied qemu: 0.0.0.0: chown: cannot access ‘/mnt/vol-01/hadooplogs/’: No such file or directory qemu: 0.0.0.0: no secondarynamenode to stop qemu: stopping yarn daemons qemu: mkdir: cannot create directory ‘/mnt/vol-01’: Permission denied qemu: chown: cannot access ‘/mnt/vol-01/hadooplogs/’: No such file or directory qemu: no resourcemanager to stop qemu: master: Warning: Permanently added 'master,10.0.2.15' (ECDSA) to the list of known hosts. qemu: master: mkdir: cannot create directory ‘/mnt/vol-01’: Permission denied qemu: master: chown: cannot access ‘/mnt/vol-01/hadooplogs/’: No such file or directory qemu: master: no nodemanager to stop qemu: mkdir: cannot create directory ‘/mnt/vol-01’: Permission denied qemu: chown: cannot access ‘/mnt/vol-01/hadooplogs/’: No such file or directory qemu: no proxyserver to stop

[saipramod]

We need have an EBS attached at the time of instance launch and the volume would be mounted to /vol-01/

[jhajek]

Ok that makes sense - we need to detail the launch script and see if we can arrange this mounting... also what IAM profile are we using?

On Sun, Jun 14, 2015 at 3:38 PM, Sai Pramod Upadhyayula < notifications@github.com> wrote:

We need have an EBS attached at the time of instance launch and the volume would be mounted to /vol-01/

— Reply to this email directly or view it on GitHub https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111874794 .

Jeremy Hajek Industry Associate Professor of Information Technology and Management ext: 630-682-6075 (2-6075) Main: 312-567-5290 (7-5290) cell: (630) 296-4012 skype: jeremy.hajek

[saipramod]

None. As of now we are entirely on Eucalyptus. No Amazon.

On Sun, Jun 14, 2015 at 7:35 PM, jhajek notifications@github.com wrote:

Ok that makes sense - we need to detail the launch script and see if we can arrange this mounting... also what IAM profile are we using?

On Sun, Jun 14, 2015 at 3:38 PM, Sai Pramod Upadhyayula < notifications@github.com> wrote:

We need have an EBS attached at the time of instance launch and the volume would be mounted to /vol-01/

— Reply to this email directly or view it on GitHub < https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111874794

.

Jeremy Hajek Industry Associate Professor of Information Technology and Management ext: 630-682-6075 (2-6075) Main: 312-567-5290 (7-5290) cell: (630) 296-4012 skype: jeremy.hajek

— Reply to this email directly or view it on GitHub https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111892063 .

[jhajek]

Ok. Eucalyptus has iam support too. So just the build system needs euca credentials. Or we need to move the mounting and esb creation to the launch script (euca-run-instances) as opposed to the build. What is this mount point on ebs doing? Logs?

-----Original Message----- From: "Sai Pramod Upadhyayula" notifications@github.com Sent: ‎6/‎14/‎2015 7:37 PM To: "SAT-Hadoop/SetupScripts" SetupScripts@noreply.github.com Cc: "jhajek" hajek@iit.edu Subject: Re: [SetupScripts] mkdir: cannot create directory - setup-hadoop.sh(#4)

None. As of now we are entirely on Eucalyptus. No Amazon.

On Sun, Jun 14, 2015 at 7:35 PM, jhajek notifications@github.com wrote:

Ok that makes sense - we need to detail the launch script and see if we can arrange this mounting... also what IAM profile are we using?

On Sun, Jun 14, 2015 at 3:38 PM, Sai Pramod Upadhyayula < notifications@github.com> wrote:

We need have an EBS attached at the time of instance launch and the volume would be mounted to /vol-01/

— Reply to this email directly or view it on GitHub < https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111874794

.

Jeremy Hajek Industry Associate Professor of Information Technology and Management ext: 630-682-6075 (2-6075) Main: 312-567-5290 (7-5290) cell: (630) 296-4012 skype: jeremy.hajek

— Reply to this email directly or view it on GitHub https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111892063 .

— Reply to this email directly or view it on GitHub.

[saipramod]

Yes. we need to move it to launch script. I updated the instructions of the setup scripts repo , for the backend worker. Please let me know if it s not clear. Yes, ebs is for logs.

On Sun, Jun 14, 2015 at 8:22 PM, jhajek notifications@github.com wrote:

Ok. Eucalyptus has iam support too. So just the build system needs euca credentials. Or we need to move the mounting and esb creation to the launch script (euca-run-instances) as opposed to the build. What is this mount point on ebs doing? Logs?

-----Original Message----- From: "Sai Pramod Upadhyayula" notifications@github.com Sent: ‎6/‎14/‎2015 7:37 PM To: "SAT-Hadoop/SetupScripts" SetupScripts@noreply.github.com Cc: "jhajek" hajek@iit.edu Subject: Re: [SetupScripts] mkdir: cannot create directory - setup-hadoop.sh(#4)

None. As of now we are entirely on Eucalyptus. No Amazon.

On Sun, Jun 14, 2015 at 7:35 PM, jhajek notifications@github.com wrote:

Ok that makes sense - we need to detail the launch script and see if we can arrange this mounting... also what IAM profile are we using?

On Sun, Jun 14, 2015 at 3:38 PM, Sai Pramod Upadhyayula < notifications@github.com> wrote:

We need have an EBS attached at the time of instance launch and the volume would be mounted to /vol-01/

— Reply to this email directly or view it on GitHub <

https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111874794

.

Jeremy Hajek Industry Associate Professor of Information Technology and Management ext: 630-682-6075 (2-6075) Main: 312-567-5290 (7-5290) cell: (630) 296-4012 skype: jeremy.hajek

— Reply to this email directly or view it on GitHub < https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111892063

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/SAT-Hadoop/SetupScripts/issues/4#issuecomment-111895634 .

[jhajek]

If I comment out the last two lines (the stop-all.sh) and the Hadoop master --force and move those to the euca-run-instances script then I think I solve our chicken and egg problem of not having the EBS mounts at build time

[saipramod]

Sure thing.