#67 - Adjust behavior for rate limit, use lambdas to resolve codes

[Kaloszer]

Using lambdas to act on codes, can be easily extended to more codes if they appear.

Should resolve #67

The upload_indicators method sends indicators in batches of size config.ms_max_indicators_request until all indicators have been sent.

If the rate limit is exceeded (status code 429), the handle_rate_limit_exceeded method is called, which sleeps for the retry_after time plus a safe_margin, then retries the request by returning the remaining indicators with the one that caused the rate limit to be exceeded at the front.

If the request is successful (status code 200), the handle_success_response method is called, which removes the sent indicators from the list and logs the number of indicators sent and remaining.

If an error occurs, the handle_error_response method is called, which logs the error and breaks the loop.

The requests_number is incremented each time a request is successful and not being retried.

If the number of requests sent in a minute reaches config.ms_max_requests_minute, the method sleeps for the remaining time in the minute plus a safe_margin, then resets the requests_number and start_timestamp.

[cudeso]

Hello. Thank you for your great contribution. The change however removes some of the logging to track the progress of uploading indicators (https://github.com/cudeso/misp2sentinel/blob/main/RequestManager.py#L227) . I think its important to keep this in the code as it allows to check on the status of the update, and how much work (nr. of indicators) still need to be uploaded.

[Kaloszer]

@cudeso It's still here under def handle_success_response https://github.com/cudeso/misp2sentinel/pull/68/files#diff-9c4ebacca9f4bca81e9401e70409295810713adc43d58dc1c3a4c49f52e2a108R269:~:text=%22Indicators%20sent%20%2D%20request%20number%3A%20%7B%7D%20/%20indicators%3A%20%7B%7D%20/%20remaining%3A%20%7B%7D%22.format(requests_number%2C%20len(request_body%5B%22value%22%5D)%2C%20len(parsed_indicators)))

[cudeso]

Apologies. I missed it. I will do the test this WE and then add the PR afterwards. Thank you for contributing.

[cudeso]

PR included. Thank you for the contribution!