pywebhdfs / pywebhdfs

Pure Python wrapper for the Hadoop WebHDFS Rest API
http://pywebhdfs.org
Other
52 stars 30 forks source link

Add support for kerberos #16

Open laserson opened 9 years ago

laserson commented 9 years ago

This may need a bit of refactoring to deal with the request objects correctly.

pkasinathan commented 8 years ago

Any updates on adding Kerberos support?

Thooms commented 8 years ago

I'm using pywebhdfs and kerberos, and a neat solution (that requires an additionnal dependency) is to use https://github.com/requests/requests-kerberos, that works out of the box. Here is an exemple:

from requests_kerberos import HTTPKerberosAuth
from pywebhdfs.webhdfs import PyWebHdfsClient

auth = HTTPKerberosAuth()
hdfs_client = PyWebHdfsClient(request_extra_opts={'auth':auth})

# then use the client as usual

This obviously assumes that you have a correct Kerberos setup on the machine.

pkasinathan commented 8 years ago

Hi @Thooms, Thanks for sharing this one. I tried this one (on Linux machine), but facing error "ImportError: No module named win32security" I'm new to python, can you please suggest how we can fix the issue? Appreciate your help!

$ uname -a
Linux  2.6.32-504.30.3.el6.x86_64 x86_64 x86_64 x86_64 GNU/Linux
$ PYTHONPATH=$HOME/pywebhdfs:$HOME/requests-kerberos:$HOME/kerberos-sspi-0.1 python
Python 2.6.6 (r266:84292, Jul 23 2015, 15:22:56) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from requests_kerberos import HTTPKerberosAuth
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/livy/scaas/requests-kerberos/requests_kerberos/__init__.py", line 17, in <module>
    from .kerberos_ import HTTPKerberosAuth, REQUIRED, OPTIONAL, DISABLED
  File "/home/livy/scaas/requests-kerberos/requests_kerberos/kerberos_.py", line 5, in <module>
    import kerberos_sspi as kerberos
  File "/home/livy/scaas/kerberos-sspi-0.1/kerberos_sspi.py", line 18, in <module>
    import win32security
ImportError: No module named win32security
>>> quit()
$
nezynet commented 6 years ago

instead of >>> from requests_kerberos import HTTPKerberosAuth try >>> import kerberos then you can see another error message more useful. Instead of running on win32 system, I added LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib64 for kerberos. Because the error message of running 'import kerberos' was ImportError: /venv_path/lib/python2.7/site-packages/kerberos.so: symbol gss_wrap_iov, version gssapi_krb5_2_MIT not defined in file libgssapi_krb5.so.2 with link time reference

HTTPKerberosAuth is NOT only for win32.