jobbrowser: Kill button not working

[fnerdwq]

In Hue 3.9.0 connected to a kerberized (HDP 2.3.4) cluster (not testet w/o security enabled) we have the issue, that a job cannot be killed in the Jobbrowser. Hitting the kill button and clicking "Yes" in the confirmation dialog, just leaves me with a in progress "Yes..." button greyed out and the following error in the runcpserver.log:

[29/Jan/2016 11:04:04 +0100] access       WARNING  127.0.0.1 someuser - "POST /jobbrowser/jobs/application_1453892619275_0066/kill HTTP/1.1"
[29/Jan/2016 11:04:04 +0100] kerberos_    ERROR    handle_mutual_auth(): Mutual authentication unavailable on 403 response
[29/Jan/2016 11:04:04 +0100] middleware   INFO     Processing exception: The default static user cannot carry out this operation. (error 403): Traceback (most recent call last):
  File "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/db/transaction.py", line 371, in inner
    return func(*args, **kwargs)
  File "/usr/share/hue/apps/jobbrowser/src/jobbrowser/views.py", line 83, in decorate
    return view_func(request, *args, **kwargs)
  File "/usr/share/hue/apps/jobbrowser/src/jobbrowser/views.py", line 245, in kill_job
    job.kill()
  File "/usr/share/hue/apps/jobbrowser/src/jobbrowser/yarn_models.py", line 89, in kill
    return self.api.kill(self.id)
  File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/yarn/resource_manager_api.py", line 95, in kill
    return self._execute(self._root.put, 'cluster/apps/%(app_id)s/state' % {'app_id': app_id}, data=json.dumps({'state': 'KILLED'}), contenttype=_JSON_CONTENT_TYPE)
  File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/yarn/resource_manager_api.py", line 98, in _execute
    response = function(*args, **kwargs)
  File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py", line 135, in put
    return self.invoke("PUT", relpath, params, data, self._make_headers(contenttype))
  File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py", line 78, in invoke
    urlencode=self._urlencode)
  File "/usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py", line 161, in execute
    raise self._exc_class(ex)
RestException: The default static user cannot carry out this operation. (error 403)

Is this is miscounfiguration issue? Or a but I'm experiencing?

[romainr]

This might be https://github.com/cloudera/hue/commit/8747b7cb5c65d88782cd5b8e9430e1d7bc7e3be6 (wrong doAs user could be used)

An alternative would also be to issue the command sent by Hue to the RM with curl as the Hue user to confirm your Yarn supports it

[fnerdwq]

This patch is hard to test on 3.9.0, since a lot of code seems to have changed arround this.

Actually issuing a

curl -u : --negotiate  -v -X PUT -d '{ "state" : "KILLED" }' -H "Content-type:application/json" http://resourcemanager:8088/ws/v1/cluster/apps/application_1453892619275_0070/state

leads to the same error The default static user cannot carry out this operation.. Hmmm, not clear to me why that's the case. So this might be nothing on the hue side...

[romainr]

In Cloudera Manager or your YARN config , "You have to check the box for "Enable Kerberos Authentication for HTTP Web-Consoles" in the Yarn configuration to make this work[1]. Then restart Yarn"