HewlettPackard / python-hpOneView

DEPRECATED - no longer actively maintained. New repository: https://github.com/HewlettPackard/oneview-python
MIT License
87 stars 57 forks source link

OneView API unable to handle concurrent 'interconnect.get_statistics' calls #396

Open ghost opened 6 years ago

ghost commented 6 years ago

Scenario/Intent

Retrieve interconnect statistics at 5min collection intervals (Traffic, error data etc. for all interconnect uplinks and downlinks). Each interconnect takes between 6-12 seconds to return its statistics which is too slow for 5 min monitoring, so I am using the Python 'map' function to apply the func 'interconnect.get_statistics(uri)' on an list of Interconnect URIs as to run them concurrently. The returned performance data is then taken and inserted into a Time Series DB for visualization with Grafana.

Environment Details

Steps to Reproduce

Attempt to query more than 5 interconnects with the 'interconnect.get_statistics(uri)' concurrently on the same OneView API/Appliance.

Expected Result

The OneView API to return the performance/statistics JSON response for each interconnect concurrently.

Actual Result

The OneView API returns the following traceback. I have not had any luck with a Thread Pool > 5, I see this same behavior across multiple appliances.

Traceback (most recent call last):
  File ".\performance_oneview_interconnects.py", line 101, in interconnect_stats_uri
    result = oneview_appliance.interconnects.get_statistics(uri)
  File "C:\Python34\lib\site-packages\hpOneView\resources\networking\interconnects.py", line 91, in get_statistics
    return self._client.get(uri)
  File "C:\Python34\lib\site-packages\hpOneView\resources\resource.py", line 294, in get
    return self._connection.get(uri)
  File "C:\Python34\lib\site-packages\hpOneView\connection.py", line 338, in get
    raise HPOneViewException(body)
hpOneView.exceptions.HPOneViewException: ('This request cannot be processed at this time.', {'errorCode': 'CRM_CANNOT_PROCESS_REQUEST', 'message': 'This request cannot be processed at this time.', 'errorSource': None, 'details': 'Sufficient resources are temporarily not available to process this request.', 'data': {}, 'nestedErrors': [], 'recommendedActions': ['Please retry this operation later.']})
- Line ('This request cannot be processed at this time.', {'errorCode': 'CRM_CANNOT_PROCESS_REQUEST', 'message': 'This request cannot be processed at this time.', 'errorSource': None, 'details': 'Sufficient resources are temporarily not available to process this request.', 'data': {}, 'nestedErrors': [], 'recommendedActions': ['Please retry this operation later.']})
ghost commented 6 years ago

As a side note, perhaps Interconnect metrics are worth adding to the 'rest/metrics/capability' endpoint?

http://h17007.www1.hpe.com/docs/enterprise/servers/oneview3.0/cic-api/en/api-docs/current/index.html#rest/metrics