search

googleapis / google-api-python-client

🐍 The official Python client library for Google's discovery based APIs.
https://googleapis.github.io/google-api-python-client/docs/
Apache License 2.0
7.78k stars 2.42k forks source link

Gmail API throws error while sending email, but still sends email #1474

Open Stephen-Bartel opened 3 years ago

Stephen-Bartel commented 3 years ago

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

If you are still having issues, please be sure to include as much information as possible:

Environment details

Linux 21.04 Python 3.9.5 pip 20.3.4 google-api-python-client : 2.10.0

Steps to reproduce

Created an email with the following code: 

def createGmailEmailNoAttachments(self, messageBody, subject, toEmail, fromEmail, html=False):
    try:
        newMessage = MIMEMultipart()
        newMessage['to']=toEmail
        newMessage['from'] = fromEmail
        newMessage['subject'] = subject
        if html: 
            msg= MIMEText(messageBody, 'html')
        else: 
            msg= MIMEText(messageBody)
        newMessage.attach(msg)

        raw = base64.urlsafe_b64encode(newMessage.as_bytes())
        raw = raw.decode()
        body = {'raw': raw}
        return body
    except:
        self.GLogger.error("An error was encountered while attempting to create gmail email")
        tb = traceback.format_exc()
        self.GLogger.exception(tb)
        return False

Send an email with the following function. 

def gmailAPISendEmail(self, message, userID="me"):
    try:
        service = self.gmailAPIService
        self.GLogger.info("Attempting to send email message")
        request = service.users().messages().send(userId=userID, body=message)
        response = self.executeGmailAPI_withretry(request)
        if response is False: 
            return False

        responseID = str(response['id'])
        self.GLogger.info("Successfully sent email message with ID (" + responseID +")")
        return responseID
    except:
        self.GLogger.error("Failed to send email message")
        tb = traceback.format_exc()
        self.GLogger.exception(tb)
        return False

The retry function in the send email function is as follows. 

def executeGmailAPI_withretry(self, request, withHTTPObject = False):
    try: 
        response_valid = False
        num_retries = 0
        while num_retries < 30: 
            try: 
                if withHTTPObject is True: 
                    response = request.execute(http=self.http_toUse)
                else: 
                    response = request.execute()
                response_valid = True
                break
            except socket.timeout:
                num_retries = num_retries + 1 
                time.sleep(0.5*num_retries)
            except: 
                self.GLogger.error("An error was encounrtered in executeGmailAPI_withretry")
                try: 
                    self.GLogger.error(f"The Method ID : {request.methodId}")
                except: 
                    pass
                try: 
                    self.GLogger.error(f"The uri : {request.uri}")
                except: 
                    pass
                tb = traceback.format_exc()
                self.GLogger.exception(tb)
                num_retries = num_retries + 1 
                time.sleep(0.5*num_retries)

        if response_valid is False: 
            self.GLogger.error(f"Could not resolve issue in 15 requests [{request}]")
            return False
        else: 
            return response
    except: 
        self.GLogger.error("An error was encounrtered in executeGmailAPI_withretry")
        tb = traceback.format_exc()
        self.GLogger.exception(tb)
        return False

The problem that I am encountering is as follows. Sometimes, when I want to send an email with these three functions, socket.timeout errors occur during execution of service.users().messages().send(userId=userID, body=message). My retry function will try to send it up to 30 times with some time delays in between. However, sometimes, when a socket.timeout error occurs, the email is still sent. This can result in several of the same emails being sent. From the code's perspective, only one email was sent, since service.users().messages().send(userId=userID, body=message) ran only once successfully without throwing an error.

So, for example, I had 4 identical emails being received, meaning that at least 3 send attempts had a socket.timeout errors in which Gmail actually did send the email and the 4th (or more) attempt executed without throwing the socket.timeout error.

Why does the Gmail API throw a socket.timeout error while sending an email, but still continue to send the email?

This creates a dilemma in the current situation.

  1. If I handle the errors, then it ensures that emails that truly cannot be sent on the first try will be sent. However, it can result in multiple identical emails being sent, due to the false errors.

  2. If I don't handle the error, then for certain, only one email at most will be sent. However, if the email truly cannot be sent, then it will certainly not be sent.

The ideal solution is that the Gmail API should only throw an error if the email truly cannot be sent.

I have tried setting various timeouts using socket.setdefaulttimeout(timeout). Although it does effect the frequency and occurrence of socket.timeout errors, the errors still do eventually occur. When it does occur with sending an email, it often sends multiple emails as described above.

Code example

# example

Stack trace

Stack trace not available. It is a socket.timeout error that is occurring when executing service.users().messages().send(userId=userID, body=message)

busunkim96 commented 3 years ago

Hi @Stephen-Bartel,

Could you share the full error message (from the socket.timeout)? A socket.timeout is most likely an issue with the connection rather than an error returned from the backend API.

Stephen-Bartel commented 3 years ago

I will try to capture the stack trace with a change to my code. In the meantime, here is an example of the socket.timeout I am seeing with one of the API calls. 

    request2 = self.gmailAPIService.users().messages().modify(userId="me", id=msgID, body=body).execute()
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/http.py", line 920, in execute
    resp, content = _retry_request(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/http.py", line 222, in _retry_request
    raise exception
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/http.py", line 191, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/google_auth_httplib2.py", line 218, in request
    response, content = self.http.request(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/httplib2/__init__.py", line 1708, in request
    (response, content) = self._request(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/httplib2/__init__.py", line 1424, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/httplib2/__init__.py", line 1376, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python3.9/http/client.py", line 1345, in getresponse
    response.begin()
  File "/usr/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
Stephen-Bartel commented 3 years ago

Traceback for when socklet.timeout error occurs with send gmail email. 

Traceback (most recent call last):
  File "/home/ubuntu/DB2/Gmail.py", line 248, in executeGmailAPI_withretry
    response = request.execute()
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/http.py", line 922, in execute
    resp, content = _retry_request(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/http.py", line 221, in _retry_request
    raise exception
  File "/home/ubuntu/.local/lib/python3.9/site-packages/googleapiclient/http.py", line 190, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/google_auth_httplib2.py", line 218, in request
    response, content = self.http.request(
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1985, in request
    (response, content) = self._request(
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1650, in _request
    (response, content) = self._conn_request(
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1589, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python3.9/http/client.py", line 1345, in getresponse
    response.begin()
  File "/usr/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

Handling of this error by retrying to send resulted in multiple emails being sent as described.