YahooArchive / oozie

Oozie - workflow engine for Hadoop
http://yahoo.github.com/oozie/
Apache License 2.0
373 stars 160 forks source link

Unable to ssh to remote machine through oozie #838

Open marryhazarika opened 12 years ago

marryhazarika commented 12 years ago

oozie vesion : 2.3.2-cdh3u2 Trying to execute a shell command in the non-hadoop remote host from a hadoop supporting node. Steps followed: 1. Copied the workflow.xml file containing the ssh action to create a file in the remote host to HDFS.

  1. Submitted the job from a hadoop supporting node where oozie derver and oozie client both are installed. Expected Result: Should create a file in the remote host Issue faced: AUTH_FAILED: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 root@192.168.1.31 mkdir -p oozie-oozi/0000000-120614121603884-oozie-oozi-W/ssh-node--ssh/ ] | ErrorStream: Permission denied (publickey,gssapi-with-mic,password) in the log file.
maruthy commented 12 years ago

The documentation at "http://incubator.apache.org/oozie/docs/3.2.0-incubating/docs/ENG_Building.html#Passphare-less_SSH_Setup" clearly indicates that there must be a user setup with pass phrase less access to shell otherwise it will endup in PermissionDenied(publickey, password) scenario

Thanks Maruthy

maruthy commented 11 years ago

Hi,

I think there is some difference in understanding the documentation of oozie. Oozie 3.2 workflows allow ssh action and being deprecated in later versions I hope. Anyway the documentation of oozie says that “pass phrase less login must be setup” in the system in-order for the ssh action to work. I did so and it started working.

Thanks Maruthy

From: FlavSF [mailto:notifications@github.com] Sent: Friday, January 25, 2013 4:11 AM To: yahoo/oozie Cc: Venkatamaruthy J -ERS, HCL Tech Subject: Re: [oozie] Unable to ssh to remote machine through oozie (#838)

I am facing the same issue as the author. Whenever I try to do any kind of ssh command I encounter the following error: AUTH_FAILED: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 root@serverip mkdir -p oozie-oozi/0000114-130122145142848-oozie-oozi-W/mysql_delete--ssh/ ] | ErrorStream: Permission denied (gssapi-keyex,gssapi-with-mic,publickey,password).

But it doesn't make sense for few reasons:

Any ideas? Did you find resolution to your problem @marryhazarikahttps://github.com/marryhazarika ?

— Reply to this email directly or view it on GitHubhttps://github.com/yahoo/oozie/issues/838#issuecomment-12677974.

::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.


marryhazarika commented 11 years ago

Hi,

Run Oozie server as a Oozie user. If running Oozie server from different user just modify the following properties in core-site.xml under HADOOP_HOME:

hadoop.proxyuser.oozie.hosts * hadoop.proxyuser.oozie.groups *

NOTE: Change the name oozie in above properties to the user name from where Oozie server is started.

My problem was solved this way. Try this out once.

On Thu, Jan 31, 2013 at 11:40 AM, maruthy notifications@github.com wrote:

Hi,

I think there is some difference in understanding the documentation of oozie. Oozie 3.2 workflows allow ssh action and being deprecated in later versions I hope. Anyway the documentation of oozie says that “pass phrase less login must be setup” in the system in-order for the ssh action to work. I did so and it started working.

Thanks Maruthy

From: FlavSF [mailto:notifications@github.com] Sent: Friday, January 25, 2013 4:11 AM To: yahoo/oozie Cc: Venkatamaruthy J -ERS, HCL Tech Subject: Re: [oozie] Unable to ssh to remote machine through oozie (#838)

I am facing the same issue as the author. Whenever I try to do any kind of ssh command I encounter the following error: AUTH_FAILED: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 root@serverip mkdir -p oozie-oozi/0000114-130122145142848-oozie-oozi-W/mysql_delete--ssh/ ] | ErrorStream: Permission denied (gssapi-keyex,gssapi-with-mic,publickey,password).

But it doesn't make sense for few reasons:

  • I can perfectly ssh into my server using ssh root@serverip
  • Whenever I execute the above command (ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 root@serverip mkdir -p oozie-oozi/0000114-130122145142848-oozie-oozi-W/mysql_delete--ssh/) manually, it works!
  • When looking at the link provided above, I also confirm that I can ssh localhost with success

Any ideas? Did you find resolution to your problem @marryhazarika< https://github.com/marryhazarika> ?

— Reply to this email directly or view it on GitHub< https://github.com/yahoo/oozie/issues/838#issuecomment-12677974>.

::DISCLAIMER::


The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.


— Reply to this email directly or view it on GitHubhttps://github.com/yahoo/oozie/issues/838#issuecomment-12929212.

Regards, Marry Hazarika Software Engineer Guwahati, Assam

maruthy commented 11 years ago

Thanks for the update, In-fact this modifications are also stated in the documentation of oozie if I am not wrong. This settings I did in the beginning itself so I did not pay much of an attention to it. Thanks Maruthy From: marryhazarika [mailto:notifications@github.com] Sent: Thursday, January 31, 2013 12:03 PM To: yahoo/oozie Cc: Venkatamaruthy J -ERS, HCL Tech Subject: Re: [oozie] Unable to ssh to remote machine through oozie (#838)

Hi,

Run Oozie server as a Oozie user. If running Oozie server from different user just modify the following properties in core-site.xml under HADOOP_HOME:

hadoop.proxyuser.oozie.hosts * hadoop.proxyuser.oozie.groups *

NOTE: Change the name oozie in above properties to the user name from where Oozie server is started.

My problem was solved this way. Try this out once.

On Thu, Jan 31, 2013 at 11:40 AM, maruthy notifications@github.com wrote:

Hi,

I think there is some difference in understanding the documentation of oozie. Oozie 3.2 workflows allow ssh action and being deprecated in later versions I hope. Anyway the documentation of oozie says that “pass phrase less login must be setup” in the system in-order for the ssh action to work. I did so and it started working.

Thanks Maruthy

From: FlavSF [mailto:notifications@github.com] Sent: Friday, January 25, 2013 4:11 AM To: yahoo/oozie Cc: Venkatamaruthy J -ERS, HCL Tech Subject: Re: [oozie] Unable to ssh to remote machine through oozie (#838)

I am facing the same issue as the author. Whenever I try to do any kind of ssh command I encounter the following error: AUTH_FAILED: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 root@serverip mkdir -p oozie-oozi/0000114-130122145142848-oozie-oozi-W/mysql_delete--ssh/ ] | ErrorStream: Permission denied (gssapi-keyex,gssapi-with-mic,publickey,password).

But it doesn't make sense for few reasons:

  • I can perfectly ssh into my server using ssh root@serverip
  • Whenever I execute the above command (ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 root@serverip mkdir -p oozie-oozi/0000114-130122145142848-oozie-oozi-W/mysql_delete--ssh/) manually, it works!
  • When looking at the link provided above, I also confirm that I can ssh localhost with success

Any ideas? Did you find resolution to your problem @marryhazarika< https://github.com/marryhazarika> ?

— Reply to this email directly or view it on GitHub< https://github.com/yahoo/oozie/issues/838#issuecomment-12677974>.

::DISCLAIMER::


The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.


— Reply to this email directly or view it on GitHubhttps://github.com/yahoo/oozie/issues/838#issuecomment-12929212.

Regards, Marry Hazarika Software Engineer Guwahati, Assam

— Reply to this email directly or view it on GitHubhttps://github.com/yahoo/oozie/issues/838#issuecomment-12929676.

marryhazarika commented 11 years ago

http://www.skillpages.com/i/ftuf/emailinvitelanding?user=3906543b-f295-e211-a117 -c86545fe99e0&signedParameters=lnIGZaJu1ysg-yuElTIKoqLCxuFyqwtlTurH7CHamU0.eyJlbW FpbCI6InJlcGx5K2ktNTA2NDU3NS0yMTM0ODJmNzNiY2ZhNTY0NzVjMDQxYWZmZWEyMDg2YjdlOTIxZDM 4LTE4NDk0NzNAcmVwbHkuZ2l0aHViLmNvbSIsImZyb20iOiIzOTA2NTQzYi1mMjk1LWUyMTEtYTExNy1j ODY1NDVmZTk5ZTAiLCJ0byI6IjU4OGYxMmE5LTJlYzItNWFiNS05NjkzLWIyYTcwOGEyMWMzMCIsImlzc 3VlZF9hdCI6IjEzNjQzMDQ5MzUiLCJ2ZXIiOiIxIiwicCI6IjMifQ&emailMessageId=e48f217c-f29 5-e211-9f30-cfd8e9020dd7&emailType=Invite&sdate=201303261335&utm_source=EmailInvi te&utm_medium=Email&emailLinkType=Logo

Hi,

Marry Hazarika wants to connect with you.

http://www.skillpages.com/i/ftuf/emailinvitelanding?user=3906543b-f295-e211-a117 -c86545fe99e0&signedParameters=lnIGZaJu1ysg-yuElTIKoqLCxuFyqwtlTurH7CHamU0.eyJlbW FpbCI6InJlcGx5K2ktNTA2NDU3NS0yMTM0ODJmNzNiY2ZhNTY0NzVjMDQxYWZmZWEyMDg2YjdlOTIxZDM 4LTE4NDk0NzNAcmVwbHkuZ2l0aHViLmNvbSIsImZyb20iOiIzOTA2NTQzYi1mMjk1LWUyMTEtYTExNy1j ODY1NDVmZTk5ZTAiLCJ0byI6IjU4OGYxMmE5LTJlYzItNWFiNS05NjkzLWIyYTcwOGEyMWMzMCIsImlzc 3VlZF9hdCI6IjEzNjQzMDQ5MzUiLCJ2ZXIiOiIxIiwicCI6IjMifQ&emailMessageId=e48f217c-f29 5-e211-9f30-cfd8e9020dd7&emailType=Invite&sdate=201303261335&utm_source=EmailInvi te&utm_medium=Email&emailLinkType=ProfileThumb

Marry Hazarika http://www.skillpages.com/i/ftuf/emailinvitelanding?user=3906543b-f295-e211-a117 -c86545fe99e0&signedParameters=lnIGZaJu1ysg-yuElTIKoqLCxuFyqwtlTurH7CHamU0.eyJlbW FpbCI6InJlcGx5K2ktNTA2NDU3NS0yMTM0ODJmNzNiY2ZhNTY0NzVjMDQxYWZmZWEyMDg2YjdlOTIxZDM 4LTE4NDk0NzNAcmVwbHkuZ2l0aHViLmNvbSIsImZyb20iOiIzOTA2NTQzYi1mMjk1LWUyMTEtYTExNy1j ODY1NDVmZTk5ZTAiLCJ0byI6IjU4OGYxMmE5LTJlYzItNWFiNS05NjkzLWIyYTcwOGEyMWMzMCIsImlzc 3VlZF9hdCI6IjEzNjQzMDQ5MzUiLCJ2ZXIiOiIxIiwicCI6IjMifQ&emailMessageId=e48f217c-f29 5-e211-9f30-cfd8e9020dd7&emailType=Invite&sdate=201303261335&utm_source=EmailInvi te&utm_medium=Email&emailLinkType=ProfileName

Invite sent: 26 March, 2013

Continue http://www.skillpages.com/i/ftuf/emailinvitelanding?user=3906543b-f295-e211-a117 -c86545fe99e0&signedParameters=lnIGZaJu1ysg-yuElTIKoqLCxuFyqwtlTurH7CHamU0.eyJlbW FpbCI6InJlcGx5K2ktNTA2NDU3NS0yMTM0ODJmNzNiY2ZhNTY0NzVjMDQxYWZmZWEyMDg2YjdlOTIxZDM 4LTE4NDk0NzNAcmVwbHkuZ2l0aHViLmNvbSIsImZyb20iOiIzOTA2NTQzYi1mMjk1LWUyMTEtYTExNy1j ODY1NDVmZTk5ZTAiLCJ0byI6IjU4OGYxMmE5LTJlYzItNWFiNS05NjkzLWIyYTcwOGEyMWMzMCIsImlzc 3VlZF9hdCI6IjEzNjQzMDQ5MzUiLCJ2ZXIiOiIxIiwicCI6IjMifQ&cnId=3906543b-f295-e211-a11 7-c86545fe99e0&emailMessageId=e48f217c-f295-e211-9f30-cfd8e9020dd7&emailType=Invi te&sdate=201303261335&utm_source=EmailInvite&utm_medium=Email&emailLinkType=MainC allToAction

reply@reply.github.com was invited to join SkillPages by Marry Hazarika. To stop receiving emails from SkillPages click here http://www.skillpages.com/i/account/unsubscribe?signedParameters=Gm6M6O5M_rRqL7Q lWAG6ep20Kxmlde2Wmv5owSDk0HU.eyJlbWFpbCI6InJlcGx5K2ktNTA2NDU3NS0yMTM0ODJmNzNiY2Zh NTY0NzVjMDQxYWZmZWEyMDg2YjdlOTIxZDM4LTE4NDk0NzNAcmVwbHkuZ2l0aHViLmNvbSIsImlzc3VlZ F9hdCI6IjEzNjQzMDQ5MzUiLCJ2ZXIiOiIxIiwicCI6IjAifQ&emailMessageId=e48f217c-f295-e2 11-9f30-cfd8e9020dd7&emailType=Invite&sdate=201303261335&utm_source=EmailInvite&u tm_medium=Email&emailLinkType=Unsubscribe .

© 2013 SkillPages, Blackrock Business Park, Dublin, Ireland and 228 Hamilton Avenue, 3rd Floor, Palo Alto, CA 94301.