search

amplab / training

Training materials for Strata, AMP Camp, etc
150 stars 121 forks source link

spark-ec2 script failed to "mkdir -p ~/.ssh" #123

Open deanwampler opened 10 years ago

deanwampler commented 10 years ago

I don't know why this didn't work on my MacBook Pro, but I suspect I was prompted to allow the ip address to be added to my known-hosts file, but the prompt was eaten by the python script. Turning off my firewall didn't help.

Anyway, fortunately, the failed command to ssh to the master and run this mkdir command was echoed to my terminal, so I could copy and run it myself, then restart the launch script replacing "--copy" with "--resume". That appeared to work (mostly; I discovered later that the wikipedia data wasn't in HDFS...).

You might add this to the list of "FAQs" you already have on the tutorial's setup page.

UFMurphy commented 10 years ago

Hahaha. Of course it has to be the infamous Dean Wampler to figure this out. This was annoying the hell out of me. Thanks, Dean. I hope all is well.

deanwampler commented 10 years ago

You said "infamous", but I think you meant to say "miscreant"...

;)

rayshih commented 10 years ago

I face this problem just 30 mins ago. I figure it out that this is just a problem that the bootstrap of the instance is too slow, such that, the ssh is not ready while this script trying to send this command.

Here is how I solve this problem: add -w 300 to make the waiting time longer.