search

cisco-en-programmability / dnacentersdk

Cisco DNA Center Python SDK
https://dnacentersdk.readthedocs.io/en/latest/
MIT License
70 stars 31 forks source link

BAPI - Buisness API Rate Limit not working. #120

Closed Chrisputer closed 10 months ago

Chrisputer commented 11 months ago

Prerequisites

Describe the bug Rate limit not working when applying not working for the following: dnac.task.get_business_api_execution_details(execution_id=execution_id)

I'm outputting the response to a textfile and getting this after about 67'th request. {'bapiKey': '429f-aa81-4d3b-960a', 'bapiName': 'Reserve IP Subpool', 'bapiExecutionId': '42918386-d93d-4dad-9293-a5d39be58d68', 'startTime': 'Mon Jul 31 17:32:50 UTC 2023', 'startTimeEpoch': 1690824770148, 'endTime': 'Mon Jul 31 17:33:07 UTC 2023', 'endTimeEpoch': 1690824787150, 'timeDuration': 17002, 'status': 'FAILURE', 'bapiError': "BapiInvokeProcessor BAPI execution failed with error=Rate Limit exceeded; BapiName: GetPoolId, RateLimit config details: RateLimitContext{rate=100, windowUnit='minute', windowDuration=1, maxConcurrentExecutionsPermitted=0}", 'runtimeInstanceId': 'DNACP_Runtime_6d510e13-cf71-484b-bf86-39c15671bd54'}

Expected behavior Expecting to see output like this when running dnac.network_settings.reserve_ip_subpool /usr/local/lib/python3.8/dist-packages/dnacentersdk/restsession.py:476: RateLimitWarning: warnings.warn(RateLimitWarning(response))

Screenshots I can provide the text file in an email. There is company information in it that I cannot share publicly.

Environment (please complete the following information):

Additional context Nada

Chrisputer commented 11 months ago

I noticed that not all of my subpools were being imported from my CSV. So, I paced the input with time.sleep(1) for each iteration, which helped me get 222 out of 240 subpools imported and reduced the rate limit errors. Previously, only about 110 out of 240 subpools were being imported.

During further testing, I found that limiting the pacing to time.sleep(2) allowed me to import all of my subpools successfully. It appears that the advertised rate limit is variable, so I'm compensating by using the slowest input currently.

I rewrote my scripts without using the SDK, but I encountered the same behavior. It seems to be an API issue. Should I open a TAC Case for this? ResIP-Pool.txt

zapodeanu commented 11 months ago

@Chrisputer As you stated, this API has a rate limit of 100 API calls/minute with no concurrency limit. Please create a TAC case and ask BU engineering to assist with logs and troubleshooting.

wastorga commented 11 months ago

If the final status is essential, you need to add a check to verify if the status is pending and wait for those execution_ids for a bit.

wastorga commented 11 months ago

Regarding the missing RateLimit warning that is only triggered if the API returns the HTTP 429 status code, and then it will wait the amount of time of the Retry-After response header.

Chrisputer commented 11 months ago

If the final status is essential, you need to add a check to verify if the status is pending and wait for those execution_ids for a bit. Regarding the missing RateLimit warning that is only triggered if the API returns the HTTP 429 status code, and then it will wait the amount of time of the Retry-After response header.

This is essential to ensure that all the subpools get created successfully. By limiting the rate to 30 requests per minute, everything works as expected, and I can obtain the desired results.

I want to highlight that the SDK Rate limiter does not work effectively with the BAPI since it consistently outputs HTTP 200 status codes. The information about the rate limit is available in the "bapi_message" or bapi_error field nested deeper.

By carefully managing the pacing and using a rate of 30 requests per minute, I can ensure that all subpools are imported without exceeding the API rate limit. This approach has been successful in resolving the issue.

Chrisputer commented 11 months ago

@Chrisputer As you stated, this API has a rate limit of 100 API calls/minute with no concurrency limit. Please create a TAC case and ask BU engineering to assist with logs and troubleshooting.

@zapodeanu I got a TAC case opened

fmunozmiranda commented 10 months ago

It seems that a solution was found for the rate limit by bapi, with sleep, this could be an improvement in API so that the expected code is returned, with the expected headers, the issue will be closed momentarily, when the API returns what is expected, the rate limit will work.