search

canonical / prometheus-openstack-exporter

OpenStack exporter for the prometheus monitoring system
GNU General Public License v3.0
128 stars 113 forks source link

crashes if a swift node is down #87

Closed vmpjdc closed 1 year ago

vmpjdc commented 5 years ago

p-o-e running on a swift-proxy crashes if a storage node is down.

We should catch the error and try another node instead.

Traceback (most recent call last):                                                                                                          
  File "/usr/local/bin/prometheus-openstack-exporter", line 702, in do_GET                                                                           
    output += collector.get_stats()                                                                                                         
  File "/usr/local/bin/prometheus-openstack-exporter", line 662, in get_stats                                                            
    self.gen_account_stats()                                                                                                            
  File "/usr/local/bin/prometheus-openstack-exporter", line 657, in gen_account_stats                                                        
    bytes_used = self._get_account_usage(account)                                                                                                       
  File "/usr/local/bin/prometheus-openstack-exporter", line 642, in _get_account_usage                                                          
    response = requests.head(account_url)                                                                                                      
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 77, in head                                                                      
    return request('head', url, **kwargs)                                                                                                          
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request                                                         
    return session.request(method=method, url=url, **kwargs)                                                                                            
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 467, in request                                                                  
    resp = self.send(prep, **send_kwargs)                                                                                                     
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 570, in send                                                           
    r = adapter.send(request, **kwargs)                                                                                                                  
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send                                                              
    raise ConnectionError(e)                                                                                                                      
ConnectionError: HTTPConnectionPool(host='10.24.0.222', port=6002): Max retries exceeded with url: /sdh/177890/AUTH_abb26b2fe803453d834071cecdb7bc21 (Caused by <class 'socket.error'>: [Errno 113] No route to host)
Pjack commented 1 year ago

Close the issue that stale over years.