search

Azure / msrest-for-python

The runtime library "msrest" for AutoRest generated Python clients.
MIT License
41 stars 64 forks source link

Deserializer will raise requests.exceptions.ConnectionError #95

Closed silencev closed 6 years ago

silencev commented 6 years ago

I am currently using msrest 0.4.11 and found the Deserializer will raise this exception: requests.exceptions.ConnectionError

requests library read the raw socket to get the response body when calling response.text here.

if connection/server is not stable, it will raise ConnectionError exception from requests library

is it possible to catch this exception in Deserializer? Because user of Azure SDK will expect the exception is always CloudError.

here is the callstack of msrest 0.4.11 version:

  File "D:\home\Python35\lib\site-packages\msrest\paging.py", line 109, in __next__
    self.advance_page()
  File "D:\home\Python35\lib\site-packages\msrest\paging.py", line 96, in advance_page
    self._derserializer(self, self._response)
  File "D:\home\Python35\lib\site-packages\msrest\serialization.py", line 733, in __call__
    data = self._unpack_content(response_data)
  File "D:\home\Python35\lib\site-packages\msrest\serialization.py", line 806, in _unpack_content
    if not raw_data.text:
  File "D:\home\Python35\lib\site-packages\requests\models.py", line 847, in text
    if not self.content:
  File "D:\home\Python35\lib\site-packages\requests\models.py", line 823, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "D:\home\Python35\lib\site-packages\requests\models.py", line 752, in generate
    raise ConnectionError(e)

thanks!

lmazuel commented 6 years ago

Hi @silencev It's not a CloudError (since the semantic of CloudError is server side error, not HTTP problems), but I understand your problem. Let me think about it more in details. Thanks,

lmazuel commented 6 years ago

This is fixed in recent generation by passing stream=False. This ask requests to load right away the body in memory. This then will be caught way before deserialization, while still in "requests", and benefits from the retry strategy of "requests" automatically. Example of new generated code: https://github.com/Azure/azure-sdk-for-python/blob/master/azure-mgmt-compute/azure/mgmt/compute/v2018_06_01/operations/virtual_machines_operations.py#L829

If even after the retry from "requests" there is still ConnectionError, this will be raise as a ClientRequestError from here: https://github.com/Azure/msrest-for-python/blob/12fb980c3290daa9c83bacc7e43c3f1455904b8c/msrest/service_client.py#L366-L369