HTTPSConnectionPool error

[brighters]

I'm conscious this error is occurring in the requests module, not django-salesforce, but have only experienced it when using django-salesforce and wondered if it was a known issue?

Occurs when trying to create a new Account object using this model:

class Account(salesforce_models.Model):

    is_person_account = models.CharField(default='012200000002ZtpAAE', db_column='RecordTypeId')
    first_name = models.CharField(db_column='FirstName')
    last_name = models.CharField(db_column='LastName')
    graduation_year = models.IntegerField(db_column='Graduation_Year__pc')
    email_address = models.CharField(db_column='PersonEmail')

ConnectionError: HTTPSConnectionPool(host='emea.salesforce.com', port=443): Max retries exceeded with url: /services/data/v37.0/sobjects/Account/ (Caused by ReadTimeoutError("HTTPSConnectionPool(host='emea.salesforce.com', port=443): Read timed out. (read timeout=3)",))

[hynekcer]

Can you reproduce it and can you connect to Salesforce successfully by another tool at the same time? I think that you probably get a similar timeout error if you write https://emea.salesforce.com/ into a browser on the same computer at the same time. I see a performance problem between 19:36 UTC, Aug 15 and 20:43 in the trust.salesforce.com log on this domain. Isn't it related?

[brighters]

I was able to connect to https://emea.salesforce.com in a browser and login with the same credentials at approximately the same time as testing, and reproduced the issue on local Mac and staging Linux environments. However, I think you may be right that there were some SF issues around that time, because on retesting now everything works. N.B. I did also set SALESFORCE_QUERY_TIMEOUT = 30, but now dropping it back to default 15, everything is still working.

Thanks for the quick reply.

[hynekcer]

It can be an interesting question, what will SFDC restrict if they have performance problems. It can be a frequency of requests, maybe a frequency of write (transactions), maybe they prefer some types of clients (normal web browsers) in order to minimize the number of unhappy people if they have problems. I can still believe that they maintain a network client neutrality and restrict only by counters, including a counter of errors.

It was an error in HTTPSConnectionPool, but maybe it was a re-connect retry after an active disconnect by the remote side. (Then the next connection is some time rejected by their firewall.) Therefore the error message is not much useful in this case.

Maybe you know if it failed on the first request or on the first write request or the Account create request was in a series of requests running as fast as possible. My goal is to offer a comparable reliability as a web browser even when SFDC is having performance problems, if the previous traffic was comparable to a normal traffic of a web browser. If you have more information, it can be still useful to gather it.

[brighters]

It was on Account creation with just one record create. So I presume just one API call. At the same time as the error I did use my own salesforce library I have for reporting purposes and ran an sobjects/Account POST (also using requests library) call which worked fine, although it's hard to be certain that one was failing and was working at exactly the same time.

a = Account()
a.first_name = profile.user.first_name
a.last_name = profile.user.last_name
a.email_address = profile.user.username
a.graduation_year = str(profile.graduation_year)
a.save()

My own library code (paraphrased for brevity)

url = 'sobjects/Account'
url = BASE_URL + url
payload = {
        ...various fields as above...
    }
access_token = get_access_token()
payload = json.dumps(payload)
headers = {'Authorization': 'Bearer %s' % access_token}
response = requests.post(url, data=payload, headers=headers)

[hynekcer]

Thanks for information. It is really very strange and fortunately very simple and similar. I expect that the number of requests was very low (at least in one of the environments comparable to web browser) and/or the open connections are reused for some reasonable time. The difference between this and your app can be in the fixed Session() and the auth attribute of the session instead of explicit 'Authorization' header. Maybe your app still had the connection in the critical period and didn't need to be restarted.

Nothing can be reproduced, OK, but for the completeness (if you still think for some reason that django-salesforce was more problematic):

• Version numbers of django-salesforce and requests package, python
• type of failed processes (a simple manage.py shell or runserver ... or a production server with some form of multi-threading/greenlets)
• If your app was restarted between the first and last problem with django-salesforce. (I understand that django-salesforce was restarted with the edited timeout.)

[brighters]

• django-salesforce 0.6.9
• requests 2.11.0
• python 3.5.1
• the failed process was in a manage.py shell and a runserver on Mac OS X, and reproduced at the same time on Linux 14.04 digital ocean server with manage.py shell

[lalatknox]

is it resolved? still happening for me on

Lead.objects.filter()

Happens alot

[hynekcer]

@lalatknox Yes, this six years old issue is closed because it was related to a timeout setting 3 seconds. That is too short for a database query, especially for Salesforce. The original poster accepted a solution by longer timeout.

A new timeout setting has been possible since 2017 as a tuple of connect timeout and data timeout, e.g. SALESFORCE_QUERY_TIMEOUT = (4, 15) # default timeout because some queries could be slow.

Another important improvement has been made in version v0.9. If the database connection has not been used for more than 300 seconds than a short dummy request with a timeout 1 second is sent as a ping to check that the connection is still live or to close a broken connection eventually and create a new.

You can start a new issue if you can describe a problem that can be reproduced.

Maybe it is not a problem in django-salesforce, but a problem with your deployment or configuration. I can help you in that case assuming that you will contribute me by a "Sponsor" button above.