Test crashes: Console says No valid crumb was included in request for /job/foo/doDelete. Returning 403

[salimfadhley]

Broken in Jenkins 1.519

Seems OK in Jenkins 1.480.3 (the version that comes with Ubuntu).

Getting lots of errors like this:

ERROR: test_invoke_job_parameterized (main.TestDownstreamUpstream)

Traceback (most recent call last): File "/home/sal/workspace/jenkinsapi/src/jenkinsapi_tests/systests/base.py", line 30, in setUp self._delete_all_jobs() File "/home/sal/workspace/jenkinsapi/src/jenkinsapi_tests/systests/base.py", line 39, in _delete_all_jobs self.jenkins.delete_job(name) File "/home/sal/workspace/jenkinsapi/src/jenkinsapi/jenkins.py", line 169, in delete_job data='some random bytes...' File "/home/sal/workspace/jenkinsapi/src/jenkinsapi/utils/requester.py", line 67, in post_and_confirm_status response.url, data, headers, response.status_code, response.text.encode('UTF-8'))) JenkinsAPIException: Operation failed. url=http://localhost:8080/job/foo/doDelete, data=some random bytes..., headers={'Content-Type': 'application/x-www-form-urlencoded'}, status=403, text=

Status Code: 403

Exception: No valid crumb was included in the request
Stacktrace:

(none)

---

Generated by Winstone Servlet Engine v0.9.10 at Tue Jun 18 23:39:50 BST 2013

---

Ran 1 test in 0.519s

[salimfadhley]

The solution appears to be to disable cross-site scripting protection in the config file. I've not got time to do this just yet.

[kworr]

Hi.

I came here because when I was trying to solve this one I found inner inconsistencies in the code and I don't know how to handle them correctly.

Index: jenkins.py
===================================================================
--- jenkins.py  (revision 847)
+++ jenkins.py  (working copy)
@@ -30,7 +30,7 @@
     """
     Represents a jenkins environment.
     """
-    def __init__(self, baseurl, username=None, password=None, requester=None):
+    def __init__(self, baseurl, username=None, password=None, requester=None, get_crumb=False):
         """
         :param baseurl: baseurl for jenkins instance including port, str
         :param username: username for jenkins auth, str
@@ -39,12 +39,13 @@
         """
         self.username = username
         self.password = password
-        self.requester = requester or Requester(username, password, baseurl=baseurl)
+        self.get_crumb = get_crumb
+        self.requester = requester or Requester(username, password, baseurl=baseurl, get_crumb=get_crumb)
         JenkinsBase.__init__(self, baseurl)

     def _clone(self):
         return Jenkins(self.baseurl, username=self.username,
-                       password=self.password, requester=self.requester)
+                       password=self.password, requester=self.requester, get_crumb=self.get_crumb)

     def base_server_url(self):
         if config.JENKINS_API in self.baseurl:
@@ -74,7 +75,7 @@
         return self

     def get_jenkins_obj_from_url(self, url):
-        return Jenkins(url, self.username, self.password, self.requester)
+        return Jenkins(url, self.username, self.password, self.requester, self.get_crumb)

     def get_create_url(self):
         # This only ever needs to work on the base object
Index: utils/requester.py
===================================================================
--- utils/requester.py  (revision 847)
+++ utils/requester.py  (working copy)
@@ -2,6 +2,7 @@
 Module for jenkinsapi requester (which is a wrapper around python-requests)
 """

+import json
 import requests
 import urlparse
 from jenkinsapi.custom_exceptions import JenkinsAPIException
@@ -31,14 +32,16 @@

     VALID_STATUS_CODES = [200, ]

-    def __init__(self, username=None, password=None, ssl_verify=True, baseurl=None):
+    def __init__(self, username=None, password=None, ssl_verify=True, baseurl=None, get_crumb=False):
         if username:
             assert password, 'Cannot set a username without a password!'

         self.base_scheme = urlparse.urlsplit(baseurl).scheme if baseurl else None
+        self.baseurl = baseurl
         self.username = username
         self.password = password
         self.ssl_verify = ssl_verify
+        self.get_crumb = get_crumb

     def get_request_dict(self, params=None, data=None, files=None, headers=None):
         requestKwargs = {}
@@ -61,6 +64,9 @@
             # It may seem odd, but some Jenkins operations require posting
             # an empty string.
             requestKwargs['data'] = data
+            if self.get_crumb:
+                resp = json.loads(self.get_url('{}/crumbIssuer/api/python'.format(self.baseurl)).content)
+                requestKwargs['data'][resp['crumbRequestField']] = resp['crumb']

         if files:
             requestKwargs['files'] = files

This fix is wrong because 'data' throughout the code is not always dict but can be str or unicode. I'm not that good with Jenkins REST interface to tell how this must be handlded here, for example when new job is created via create_job 'data' contains raw xml and it should be altered somehow to contain crumb alongside. Or do I need to alter request URL instead adding crumb there as a get parameter?

If you can point me to inner specifications of Jenkins or unstuck me any other way around here I'll be able to complete this.

[lechat]

For visitors looking for solution: Starting from version 0.3.2 there is a CrumbRequester class that handles CRSS enabled Jenkins. To use it, you need to initialize Jenkins object like this:

from jenkinsapi.jenkins import Jenkins
from jenkinsapi.crumb_requester import CrumbRequester

jenkins = Jenkins('http://localhost:8080', requester=CrumbRequester('http://localhost:8080'))

[fluffynuts]

I'd just like to chip in a sample which works in a password-protected environment, because it's not immediately clear and help(CrumbRequester) isn't particularly helpful since it just boosts arguments up to the parent Requester class. So I have a Helper which just checks on all jobs and invokes them if they last failed (sometimes we come in in the morning to find that literally a hundred jobs or more have failed because of an internet interruption, for example). The top of this file gives away the bits useful for this discussion:

from jenkinsapi.jenkins import Jenkins
from jenkinsapi.utils.crumb_requester import CrumbRequester

class Helper:
    def __init__(self, url, user, password):
        requester = CrumbRequester(baseurl=url, username=user, password=password)
        self._jenkins = Jenkins(baseurl=url, username=user, password=password, requester=requester)

Note that my import statement differes from that of @lechat -- I assume his is a typo? It took me a bit of digging to get to the CrumbRequester, so I wanted to share that too. Also note that the CrumbRequester requires all of the auth information -- perhaps it would be neater if the Jenkins constructor took an optional class instead of object since the other requesters are built with the same arguments?

[lechat]

Yes, better design would be just passing Requester class (and not an instance) to Jenkins and then let Jenkins object fill required fields. It is easy to implement, but it will break a lot of existing code that uses this library.

Perhaps better idea is to create shallow descendants of Jenkins class for each Requester implementation, or wrappers just like your Helper class.

Yeah, I will fix the typo! :)