'Connection aborted.' in BulkOperation.download_result_file with ssl.PROTOCOL_SSLv23

[nilya]

We use BulkServiceManager download_entities method to get campaigns and ad_groups, code copied from this example.

After update to a3d33d8 we constantly get ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) error in BulkOperation.download_result_file method r = s.get(...) call. Example mentioned above doesn't work too.

Change to previous state fixes both our code and example:

+            ssl_version=ssl.PROTOCOL_SSLv3,
-            ssl_version=ssl.PROTOCOL_SSLv23,

Tested: Ubuntu 14.04, Python 2.7.6 and Python 3.4.0, bingads==9.3.4, requests==2.7.0.

[Viswa-Rathinavelu]

Hi Ilya - Thanks for reaching out to us. We will look into this issue.

Thanks, Viswa

From: Ilya Nazarov [mailto:notifications@github.com] Sent: Wednesday, September 2, 2015 7:12 AM To: bing-ads-sdk/BingAds-Python-SDK Subject: [BingAds-Python-SDK] 'Connection aborted.' in BulkOperation.download_result_file with ssl.PROTOCOL_SSLv23 (#8)

We use BulkServiceManager download_entities method to get campaigns and ad_groups, code copied from this examplehttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fbing-ads-sdk%2fBingAds-Python-SDK%2fblob%2f3865dc2d427bd6a87356160088e6d72f19a3c28b%2fexamples%2fdjangowebapp%2fapp%2fviews.py%23L99&data=01%7c01%7cVISWAR%40exchange.microsoft.com%7c162036007a5649c19bc008d2b3a0714b%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=G6H9RNsj7y%2fuxikljN7ewMQ9SMhAfhZWREbG%2b7vNUnY%3d.

After update to a3d33d8https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fbing-ads-sdk%2fBingAds-Python-SDK%2fcommit%2fa3d33d820c6adc1d26c4b851e5c04498ab8f7247&data=01%7c01%7cVISWAR%40exchange.microsoft.com%7c162036007a5649c19bc008d2b3a0714b%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=hiEUAQfubPgJjCVw3Y610tqS%2f2I6cXFTzuGDa2NnyKo%3d we constantly get ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) error in BulkOperation.download_result_file method r = s.get(...) call. Example mentioned above doesn't work too.

Change to previous state fixes both our code and example:

•      ssl_version=ssl.PROTOCOL_SSLv3,

•      ssl_version=ssl.PROTOCOL_SSLv23,

Tested: Ubuntu 14.04, Python 2.7.6 and Python 3.4.0, bingads==9.3.4, requests==2.7.0.

— Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fbing-ads-sdk%2fBingAds-Python-SDK%2fissues%2f8&data=01%7c01%7cVISWAR%40exchange.microsoft.com%7c162036007a5649c19bc008d2b3a0714b%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9FtG0iw%2fwBsVidlFJidrUDpHucebrLu3Jte0FkCmY7w%3d.

[eric-urban]

Our engineers applied an update today which should resolve the issue you observed. Will you please try again and let us know if you are unblocked? Thank you.

[imagineful]

Hi, I suppose the root cause is Python version is too low. TLS1.2 is not supported in python 2.7.6, That's why the connection failed when server migrated to TLS1.2. to verify this, please run the following in your python cl:

import ssl 'PROTOCOL_TLSv1_2' in dir(ssl) the above should return a False, I checked this in my python 2.7.6

I recommend you to upgrade your python version to 2.7.9 or 2.7.10(latest), TLS1.2 is supported from python 2.7.9.

[nilya]

Hi @eric-urban. No, problem isn't solved. The problem is with https://download.api.bingads.microsoft.com/ addresses only (bulk operation and generate report requests store their results on this domain). It can be accessed from Python with ssl_version=ssl.PROTOCOL_SSLv3, but not with ssl_version=ssl.PROTOCOL_SSLv23. This domain still supports only SSL 3 and TLS 1.0 protocols [1] while api.bingads.microsoft.com and api.sandbox.bingads.microsoft.com support SSL 3 - TLS 1.2 [2,3].

As a temporary solution we've pinned bingads library version to 9.3.3.

@imfu You're right that 2.7.6 doesn't support TLS 1.2. I repeated my tests with 3.4.0. Both versions can get https://api.bingads.microsoft.com addresses with ssl_version=ssl.PROTOCOL_SSLv23 but not the https://download.api.bingads.microsoft.com/ ones. We also tried pyOpenSSL module - result is the same.

Qualys SSL Labs tests: [1] download.api.bingads.com SSL 3, TLS 1.0 [2] api.bingads.microsoft.com SSL 3 - TLS 1.2 [3] api.sandbox.bingads.microsoft.com SSL 3 - TLS 1.2 [4] bingads.microsoft.com SSL 3, TLS 1.0

[remiremi]

I had the same issue, but strangely it only happens on some systems and not others, despite python, bingads and requests being at the same version. As a temporary fix I monkey patched bingads:

import ssl
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager

class Ssl3HttpAdapter(HTTPAdapter):
    """"Transport adapter" that allows us to use SSLv3."""

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       ssl_version=ssl.PROTOCOL_SSLv3)

from bingads.bulk import bulk_operation
bulk_operation.TlsHttpAdapter = Ssl3HttpAdapter

[imagineful]

Hi, @remiremi Could you please check the following things:

1 Python version (the full version name, like python 2.7.10)? 2 The openssl version python uses: go to the python interpreter(cmd line) and run

>>>import ssl
>>>ssl.OPENSSL_VERSION

3 Also you can check if the following patch works for you, currently sslv3 is supported, when migrated to TLS1.2, then the sslv3 adapter will not work.

class TlsHttpAdapter(HTTPAdapter):
    """" Transport adapter that chooses the TLS protocols based on python versions. """

    def init_poolmanager(self, connections, maxsize, block=False):
        ssl_version = ssl.PROTOCOL_TLSv1 if sys.version_info < (2, 7, 9) or sys.version_info[0:2] == (3, 3) \
                                         else ssl.PROTOCOL_SSLv23
        self.poolmanager = PoolManager(
            num_pools=connections,
            maxsize=maxsize,
            block=block,
            ssl_version=ssl_version
        )

Thanks.

[remiremi]

Hi imfu,

1. Python 3.4.2
2. This ssl version causes problems: OpenSSL 1.0.1e-fips 11 Feb 2013 I have another environment with version OpenSSL 0.9.8zg 14 July 2015, and it works correctly there.
3. It doesn't work, it's the same as what's in bingads 10.4.1: https://github.com/bing-ads-sdk/BingAds-Python-SDK/blob/master/bingads/bulk/bulk_operation.py#L21

Hope this helps.

[imagineful]

Hi, @remiremi , Thanks for your response. I did not re-pro the error on Python 3.4.2, on which the OpenSSL version is 'OpenSSL 1.0.1i 6 Aug 2014'. I tested v10 bulk download and v9 reporting download operations, both succeeded. And also tried on 3.3.5, which OpenSSL version is 1.0.1e (but NOT -fips), the cases passed too.

And you are right, the code TlsHttpAdapter is from version 10.4.1. The logic is to pick the SSL/TLS protocols based on python versions, for 3.4, it will use SSLv23, which it use the highest protocol that both the client and the server support, currently it should be TLS1.0, while SSLv3 also will work.

[nilya]

Hi, @imfu Maybe it's better to target protocols based on ssl.OPENSSL_VERSION_INFO? Because the mean of ssl.PROTOCOL_XXX constants depends on OpenSSL version not the Python one. And the same Python version can be linked with different OpenSSL library versions.

[imagineful]

We thought about this before, per my understanding, TLS 1.0 and 1.2 should supported in above 1.0.1 versions. so we could not conclude that this failure is caused by the openssl version. and from python document, 2.7.10 and 3.4 and their above versions should support TLS1.2 (currently server side has not migrated to 1.2).

Is your openssl version 1.0.1e-fips (please notice if there's a fips here)?

[nilya]

Hi. It seems that download.api.bingads.com upgraded to TLS 1.0, 1.1, 1.2. We have had pinned bingads version to 9.3.3 and it stopped to work recently. Current bingads version 10.4.1 works in our environment (Python 2.7.3, OpenSSL 1.0.1 14 Mar 2012).

I'm closing the issue.