jenkinsapi: Read time out exception (max_retries)

[ChaiBapchya]

ISSUE TYPE

• Bug Report

Jenkinsapi VERSION 03.11

Jenkins VERSION 2.204.2

SUMMARY

JenkinsAPI call fails with max retries exceeded with URL issue command

for jobName, job in JenkinsObj.get_jobs():

EXPECTED RESULTS

Return all the jobs without max retry error

ACTUAL RESULTS

result (after reading several jobs)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='jenkins.mxnet-ci.amazon-ml.com', port=80): Max retries exceeded with url: /job/mxnet-validation/job/centos-cpu/job/PR-17570/api/python (Caused by ReadTimeoutError("HTTPConnectionPool(host='jenkins.mxnet-ci.amazon-ml.com', port=80): Read timed out. (read timeout=10)"))

USEFUL INFORMATION

To reproduce

Example code

JenkinsObj = Jenkins('http://jenkins.mxnet-ci.amazon-ml.com/')
for jobName, job in JenkinsObj.get_jobs():
        print("Queueing job [%s]" % (jobName))

Stacktrace

Traceback (most recent call last):
  File "/Users/bapac/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/bapac/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/Users/bapac/anaconda3/lib/python3.7/http/client.py", line 1344, in getresponse
    response.begin()
  File "/Users/bapac/anaconda3/lib/python3.7/http/client.py", line 306, in begin
    version, status, reason = self._read_status()
  File "/Users/bapac/anaconda3/lib/python3.7/http/client.py", line 267, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/Users/bapac/anaconda3/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bapac/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/Users/bapac/lib/python3.7/site-packages/urllib3/connectionpool.py", line 423, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/Users/bapac/lib/python3.7/site-packages/urllib3/connectionpool.py", line 331, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='jenkins.mxnet-ci.amazon-ml.com', port=80): Read timed out. (read timeout=10)

[lechat]

Sorry, library can't fix your connectivity problems.

[ChaiBapchya]

@lechat This issue was actually resolved by adding timeout=120 and max_retries = 30 which is only possible in master branch (not the pypi released binary version)

Would be great to have latest master pushed out to the public via PyPi binary

[clarketm]

~Is there a plan to release a 0.3.12 with the latest changes from master to PyPI?~ It looks like there is a release 0.3.12 planned: https://github.com/pycontribs/jenkinsapi/issues/774

[Azee77]

Is this is going to be fixed? Will there be timeout args handling in next update on pypi ?

[rajendrarawat17]

Hi,

I am facing intermittent issue of read timeout while calling the sendmail functionality through jenkinsapi from a code running inside a docker container to a remote jenkins.. Sometime it works and sometime it give the timeout error. I tried increasing the timeout but did not help and facing still the same intermittent issue

This jenkins is same that runs the pipeline on remote docker host where it creates the container and run the code. Below is the error details , any help would be appreciated.

Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request httplib_response = conn.getresponse() File "/usr/lib64/python3.9/http/client.py", line 1371, in getresponse response.begin() File "/usr/lib64/python3.9/http/client.py", line 319, in begin version, status, reason = self._read_status() File "/usr/lib64/python3.9/http/client.py", line 280, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib64/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 451, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 340, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='', port=8080): Read timed out. (read timeout=120)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/jenkins/.local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen retries = retries.increment( File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='', port=8080): Max retries exceeded with url: /job/sendmail/api/python?tree=allBuilds%5Bnumber%2Curl%5D (Caused by ReadTimeoutError("HTTPConnectionPool(host='', port=8080): Read timed out. (read timeout=120)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/jenkins/work_arear/workspace/abc/project_release_delivery_cli.py", line 472, in main() File "/home/jenkins/work_arear/workspace/abc/project_release_delivery_cli.py", line 467, in main send_mail_fun(checksum, customer, from_password, from_username, owner, label, size_in_bytes, size_in_mb, File "/home/jenkins/work_arear/workspace/abc/project_release_delivery_cli.py", line 336, in send_mail_fun sm = SendMailJenkins(from_address=from_username, from_password=from_password, File "/home/jenkins/work_arear/workspace/abc/utilities/SendMailJenkins.py", line 41, in init self.send_mail_job = self.jenkins.get_job(self.DEVOPS_SENDMAIL_JOB) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/jenkins.py", line 156, in get_job return self.jobs[jobname] File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/jobs.py", line 87, in getitem return Job(job_data['url'], job_data['name'], self.jenkins) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/job.py", line 68, in init JenkinsBase.init(self, self.url) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 38, in init self.poll() File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/job.py", line 94, in poll self._data = self._add_missing_builds(self._data) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/job.py", line 120, in _add_missing_builds response = self.poll(tree='allBuilds[number,url]') File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/job.py", line 92, in poll data = super(Job, self).poll(tree=tree) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 60, in poll data = self._poll(tree=tree) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 70, in _poll return self.get_data(url, tree=tree) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 80, in get_data response = requester.get_url(url, params) File "/home/jenkins/.local/lib/python3.9/site-packages/jenkinsapi/utils/requester.py", line 144, in get_url return self.session.get(self._update_url_scheme(url), requestKwargs) File "/home/jenkins/.local/lib/python3.9/site-packages/requests/sessions.py", line 555, in get return self.request('GET', url, kwargs) File "/home/jenkins/.local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, send_kwargs) File "/home/jenkins/.local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, kwargs) File "/home/jenkins/.local/lib/python3.9/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='', port=8080): Max retries exceeded with url: /job/sendmail/api/python?tree=allBuilds%5Bnumber%2Curl%5D (Caused by ReadTimeoutError("HTTPConnectionPool(host='', port=8080): Read timed out. (read timeout=120)"))

[ChrisWeiss]

This is not necessarily a connectivity issue. On a complex Jenkins instance, pulling up the api/python page on a job can take more than 10 seconds. For example we have a job that uses the Git Parameter plugin and has to scan all the source branches for the api/python data that routinely takes 20 seconds to load. It would be very nice to at least expose the timeout value to the end user.

[salimfadhley]

Hi, I'm the original author. I'm not actively maintaining this project anymore. If you can show me a tested PR then I will consider a merge. That's as long as @lechat approves.

[PEhrle]

As no one has mentioned it here before: The timeout can actually (at least now) be set when creating the Jenkins object. Ex: J = Jenkins(url, user_id, api_token, timeout=180)