gramps-project / gramps-web

Frontend for Gramps Web
https://www.grampsweb.org
GNU Affero General Public License v3.0
397 stars 51 forks source link

SMTP "Error while trying to send e-mail." #448

Closed dominikhoebert closed 3 months ago

dominikhoebert commented 3 months ago

I can't get SMTP e-mail to work. Not sure if bug or configuration error.

I tried gmail and apple mail. SMTP and IMAP configuration

smtp.googlemail.com 587/468 TLS true/false; all combinations smtp.mail.me.com 587/993 TLS true/false; all combinations imap.mail.me.com 587/993 TLS true/false; all combinations

I tried the same configuration and credentials I use in other services, so I know they work.

I tried configuring it using docker environment variables and the admin creation page.

Every time I try the Reset Password on the login page I get the "Error while trying to send e-mail." popup and error in the logs.

When I try to create a new user I get Error 500 and this:

2024-06-05T11:06:16.636157069Z [2024-06-05 13:06:16 +0200] [22] [ERROR] Error while trying to send e-mail.
2024-06-05T11:06:16.648927424Z [2024-06-05 13:06:16 +0200] [22] [ERROR] Error handling request /api/users/test2/register/
2024-06-05T11:06:16.648970729Z Traceback (most recent call last):
2024-06-05T11:06:16.648981410Z   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 500, in send_email
2024-06-05T11:06:16.648988972Z     smtp = smtplib.SMTP_SSL(host=host, port=port, timeout=10)
2024-06-05T11:06:16.648996152Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649002192Z   File "/usr/lib/python3.11/smtplib.py", line 1050, in __init__
2024-06-05T11:06:16.649008116Z     SMTP.__init__(self, host, port, local_hostname, timeout,
2024-06-05T11:06:16.649014338Z   File "/usr/lib/python3.11/smtplib.py", line 258, in __init__
2024-06-05T11:06:16.649020781Z     raise SMTPConnectError(code, msg)
2024-06-05T11:06:16.649027812Z smtplib.SMTPConnectError: (-1, b'[CAPABILITY XAPPLEPUSHSERVICE IMAP4 IMAP4rev1 SASL-IR AUTH=ATOKEN AUTH=PLAIN AUTH=ATOKEN2] (2416B59-5e39d55722db) ms11p00im-qufv17111801.me.com')
2024-06-05T11:06:16.649034849Z 
2024-06-05T11:06:16.649041225Z During handling of the above exception, another exception occurred:
2024-06-05T11:06:16.649048261Z 
2024-06-05T11:06:16.649054652Z Traceback (most recent call last):
2024-06-05T11:06:16.649061460Z   File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 135, in handle
2024-06-05T11:06:16.649068310Z     self.handle_request(listener, req, client, addr)
2024-06-05T11:06:16.649074582Z   File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 178, in handle_request
2024-06-05T11:06:16.649081540Z     respiter = self.wsgi(environ, resp.start_response)
2024-06-05T11:06:16.649088073Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649095235Z   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1498, in __call__
2024-06-05T11:06:16.649102183Z     return self.wsgi_app(environ, start_response)
2024-06-05T11:06:16.649108666Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649115498Z   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1476, in wsgi_app
2024-06-05T11:06:16.649122451Z     response = self.handle_exception(e)
2024-06-05T11:06:16.649129440Z                ^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649136131Z   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
2024-06-05T11:06:16.649143364Z     response = self.full_dispatch_request()
2024-06-05T11:06:16.649150441Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649157225Z   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
2024-06-05T11:06:16.649164015Z     rv = self.handle_user_exception(e)
2024-06-05T11:06:16.649171273Z          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649178385Z   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
2024-06-05T11:06:16.649198946Z     rv = self.dispatch_request()
2024-06-05T11:06:16.649207716Z          ^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649215198Z   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
2024-06-05T11:06:16.649223343Z     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
2024-06-05T11:06:16.649230819Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649238077Z   File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 110, in view
2024-06-05T11:06:16.649245586Z     return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
2024-06-05T11:06:16.649253032Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649260043Z   File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 191, in dispatch_request
2024-06-05T11:06:16.649267689Z     return current_app.ensure_sync(meth)(**kwargs)  # type: ignore[no-any-return]
2024-06-05T11:06:16.649274683Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649282052Z   File "/usr/local/lib/python3.11/dist-packages/flask_limiter/extension.py", line 1307, in __inner
2024-06-05T11:06:16.649289598Z     return cast(R, flask.current_app.ensure_sync(obj)(*a, **k))
2024-06-05T11:06:16.649297563Z                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649305534Z   File "/usr/local/lib/python3.11/dist-packages/webargs/core.py", line 649, in wrapper
2024-06-05T11:06:16.649312990Z     return func(*args, **kwargs)
2024-06-05T11:06:16.649320535Z            ^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649327968Z   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/user.py", line 365, in post
2024-06-05T11:06:16.649335294Z     run_task(send_email_confirm_email, email=args["email"], token=token)
2024-06-05T11:06:16.649342960Z   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 55, in run_task
2024-06-05T11:06:16.649349966Z     return task(**kwargs)
2024-06-05T11:06:16.649357214Z            ^^^^^^^^^^^^^^
2024-06-05T11:06:16.649364655Z   File "/usr/local/lib/python3.11/dist-packages/celery/local.py", line 182, in __call__
2024-06-05T11:06:16.649372534Z     return self._get_current_object()(*a, **kw)
2024-06-05T11:06:16.649379448Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649386522Z   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/util/celery.py", line 18, in __call__
2024-06-05T11:06:16.649393818Z     return self.run(*args, **kwargs)
2024-06-05T11:06:16.649400823Z            ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T11:06:16.649407934Z   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 88, in send_email_confirm_email
2024-06-05T11:06:16.649415107Z     send_email(subject=subject, body=body, to=[email])
2024-06-05T11:06:16.649433820Z   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 519, in send_email
2024-06-05T11:06:16.649443019Z     raise ValueError("Error while trying to send e-mail.")
2024-06-05T11:06:16.649451056Z ValueError: Error while trying to send e-mail.

Love the project! Thank you for your work!

DavidMStraub commented 3 months ago

Hi!

Does Gmail actually still support SMTP with username/password (rather than OAuth2)? I thought it doesn't.

I suggest to try sending an e-mail directly from Python using the same function as Gramps Web API:

https://github.com/gramps-project/gramps-web-api/blob/c6dbfd4cc78309bb23ea1a0dbb0ed397f84f900e/gramps_webapi/api/util.py#L480-L519

If you get that to work and still see an error, please open an issue at https://github.com/gramps-project/gramps-web-api/issues. Thanks!

thickconfusion commented 3 months ago

I'm having this same problem. I have other apps using Gmail SMTP after generating app passwords. I'm getting the same error messages as @dominikhoebert . I am trying to take your advice to run the Python script but I'm failing at doing this. I am running Gramps Web Docker, and I think something else that's interesting looking at Celery:


[2024-06-21 19:16:31,515: INFO/MainProcess] Task gramps_webapi.api.tasks.send_email_confirm_email[ebde6b4c-001a-44b0-a981-fe339adca68f] received
[2024-06-21 19:17:21,553: WARNING/ForkPoolWorker-1] [2024-06-21 19:17:21,553] ERROR in util: Error while trying to send e-mail.
[2024-06-21 19:17:21,553: ERROR/ForkPoolWorker-1] Error while trying to send e-mail.
[2024-06-21 19:17:21,556: ERROR/ForkPoolWorker-1] Task gramps_webapi.api.tasks.send_email_confirm_email[ebde6b4c-001a-44b0-a981-fe339adca68f] raised unexpected: ValueError('Error while trying to send e-mail.')
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 500, in send_email
    smtp = smtplib.SMTP_SSL(host=host, port=port, timeout=10)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 1050, in __init__
    SMTP.__init__(self, host, port, local_hostname, timeout,
  File "/usr/lib/python3.11/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 1056, in _get_socket
    new_socket = super()._get_socket(host, port, timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 851, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.11/socket.py", line 836, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/util/celery.py", line 20, in __call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 88, in send_email_confirm_email
    send_email(subject=subject, body=body, to=[email])
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 519, in send_email
    raise ValueError("Error while trying to send e-mail.")
ValueError: Error while trying to send e-mail.
thickconfusion commented 3 months ago

I should clarify, not exactly the same error, but lots of overlap here. Inside the container, I verified I could ping the SMTP host.

dominikhoebert commented 3 months ago

To Test my credentials, I created the following simple python script from

https://github.com/gramps-project/gramps-web-api/blob/c6dbfd4cc78309bb23ea1a0dbb0ed397f84f900e/gramps_webapi/api/util.py#L480-L519

import smtplib
import socket
from email.message import EmailMessage
from email.utils import make_msgid
from typing import BinaryIO, List, Optional, Sequence, Tuple

def send_email(subject: str, body: str, to: Sequence[str], from_email: Optional[str] = None) -> None:
    """Send an e-mail message."""
    msg = EmailMessage()
    msg.set_content(body)
    msg["Subject"] = subject
    msg["From"] = from_email
    msg["To"] = ", ".join(to)
    msg["Message-ID"] = make_msgid()

    host = "smtp.mail.me.com"
    port = 587
    user = "mail@me.com"
    password = "pw"  # app password
    use_tls = False
    try:
        if use_tls:
            smtp = smtplib.SMTP_SSL(host=host, port=port, timeout=10)
        else:
            smtp = smtplib.SMTP(host=host, port=port, timeout=10)
            smtp.ehlo()
            if port != 25:
                smtp.starttls()
                smtp.ehlo()
        if user:
            smtp.login(user, password)
        smtp.send_message(msg)
        smtp.quit()
    except ConnectionRefusedError:
        print("Connection to SMTP server refused.")
        raise ValueError("Connection was refused.")
    except socket.timeout:
        print("SMTP connection attempt timed out.")
        raise ValueError("Connection attempt timed out.")
    except OSError:
        print("Error while trying to send e-mail.")
        raise ValueError("Error while trying to send e-mail.")

if __name__ == "__main__":
    send_email("Hello, World!", "Hello, World!", ["mail@icloud.com"], "mail@me.com")
thickconfusion commented 3 months ago

Good news...

I used your module @dominikhoebert to work out some bugs in my variables. I have it working now. In case other users still have problems, make absolutely sure you have the correct SMTP server, the correct port, and the correct TLS setting.