vgrem / Office365-REST-Python-Client

Microsoft 365 & Microsoft Graph Library for Python
MIT License
1.34k stars 335 forks source link

Retry-after for 429? #424

Open tomerharduf-vayyar opened 3 years ago

tomerharduf-vayyar commented 3 years ago

I am running a script that uploads a lot of files in parallel and I get this Exception: "(None, None, '429 Client Error: for url blahblah.blah)". Now I understand this Exception arises when too mant requests are sent to the server. I also understand that usually the server sends a "retry-after" back so the client would know how long to wait before sending new requests. Do you support it? If so, how do I do this?

jneuendorf commented 2 years ago

As I understand the API, you have to use .execute_query_retry instead of execute_query.

sant3e commented 1 year ago

I also have this issue. I've incorporated my script into a Pentaho (kettle) automatic job, that runs the script 2 times daily. In a given week 80% of the time i get this error, even though i did:

My use case is pulling data from a SP List with about 3k records

hmoffatt commented 1 year ago

There's an execute_query_with_incremental_retry() method in client_context.py which looks like it would handle the 429s

To try it I replaced

target_folder.files.create_upload_session(local_path, size_chunk, bar.update).execute_query()

with

target_folder.files.create_upload_session(local_path, size_chunk, bar.update).context.execute_query_with_incremental_retry()

My uploads seem to fail with error 503 though, not 429.