pycontribs / jenkinsapi

A Python API for accessing resources and configuring Hudson & Jenkins continuous-integration servers
http://pypi.python.org/pypi/jenkinsapi
MIT License
858 stars 485 forks source link

jenkinsapi Nodes class should limit api requests with tree #855

Open J17359 opened 9 months ago

J17359 commented 9 months ago
ISSUE TYPE
Jenkinsapi VERSION

0.3.13

Jenkins VERSION

2.426.3

SUMMARY

On systems with a large number of Nodes, jenkinsapi will throw a requests.exceptions.ConnectionError exception as it times out retrieving the node list from the Jenkins python api. Based on some best practices documentation, api requests should be used with tree to restrict the information pulled from the server.

Simply typing <my_server>/computuer/api/python into a web browser is enough to see a similar "504 Gateway Time-out" error.

But using <my_server>/computer/api/python?tree=computer[displayName] works. I know jenkinsapi probably needs more information about that node list than just the display name of each node, but limiting the response from the api seems to solve the timeout issue.

EXPECTED RESULTS

A Nodes class object returned when calling Jenkins.get_nodes().

ACTUAL RESULTS

An exception was thrown caused by a timeout trying to access the node list from the server.

USEFUL INFORMATION

When calling the /computer/api/python api endpoint on servers with a large Nodes list (ours has 300+), the below traceback can be observed.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/jenkins.py", line 305, in get_node
    return self.nodes[nodename]
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/jenkins.py", line 327, in nodes
    return self.get_nodes()
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/jenkins.py", line 323, in get_nodes
    return Nodes(self.baseurl, self)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/nodes.py", line 27, in __init__
    JenkinsBase.__init__(
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 39, in __init__
    self.poll()
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 62, in poll
    data = self._poll(tree=tree)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 72, in _poll
    return self.get_data(url, tree=tree)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/jenkinsbase.py", line 82, in get_data
    response = requester.get_url(url, params)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/jenkinsapi/utils/requester.py", line 167, in get_url
    return self.session.get(self._update_url_scheme(url), **requestKwargs)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/home/user/builder-automation/venv/lib64/python3.9/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='<my_server>', port=443): Max retries exceeded with url: /computer/api/python (Caused by ReadTimeoutError("HTTPSConnectionPool(host='<my_server>', port=443): Read timed out. (read timeout=10)"))