Mail not sent anymore after upgrade / SSL_connect returned=1 errno=0 state=error: wrong version number)

[airblag]

Since my last update of the docker containers (tag latest), the mails are not sent anymore by my loomio instance.

My setup is running docker inside a lxd container, connecting to another lxd container which is my mail server.

It used to work having SMTP_USE_SSL commented in the .env file. Since the update it seems to ignore this parameter, and I get errors of the loomio-worker like this one when mentioning myself in a comment :

loomio-worker  | 2023-04-21T10:36:58.133Z pid=8 tid=1fm0 class=ActionMailer::MailDeliveryJob jid=edbb1872bfcf3181f493ded2 elapsed=0.282 INFO: fail           loomio-worker  | 2023-04-21T10:36:58.133Z pid=8 tid=1fm0 WARN: {"context":"Job raised exception","job":{"retry":true,"queue":"mailers","backtrace":true,"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"ActionMailer::MailDeliveryJob","args":[{"job_class":"ActionMailer::MailDeliveryJob","job
_id":"03650fbd-731b-4913-a5f1-a4e72b6dd628","provider_job_id":null,"queue_name":"mailers","priority":null,"arguments":["EventMailer","event","deliver_now",{"args":[1,1555],"_aj_ruby2_keywords":["args"]}],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2023-04-21T10:32:49Z"}],"jid":"edbb1872bfcf3181f493ded2","created_at":1682073169.210124,"sentry_user":{"id":1},"enqueued_at":1682073417.850152,"error_message":"SSL_connect returned
=1 errno=0 state=error: wrong version number","error_class":"OpenSSL::SSL::SSLError","failed_at":1682073171.637881,"retry_count":3,"error_backtrace":"[...]","retried_at":1682073282.036305}}   
loomio-worker  | 2023-04-21T10:36:58.133Z pid=8 tid=1fm0 WARN: OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: wrong version number                                                                                                                                                                      loomio-worker  | 2023-04-21T10:36:58.133Z pid=8 tid=1fm0 WARN: /usr/local/lib/ruby/2.7.0/net/protocol.rb:44:in `connect_nonblock'               
loomio-worker  | /usr/local/lib/ruby/2.7.0/net/protocol.rb:44:in `ssl_socket_connect'     
[...]

The error_backtrace is base64 but decodes to binary, so I'm not sure how to analyze it.

On my Mailserver, I get

Apr 21 12:36:58 kmail postfix/submission/smtpd[172477]: connect from loomio[10.23.136.2]
Apr 21 12:36:58 kmail postfix/submission/smtpd[172477]: lost connection after UNKNOWN from loomio[10.23.136.2]
Apr 21 12:36:58 kmail postfix/submission/smtpd[172477]: disconnect from loomio[10.23.136.2] unknown=0/2 commands=0/2

I actually do not need SSL since both containers are running on the same host. Executing an interactive shell in loomio-worker, and trying to openssl s_connect -starttls smtp mail.fqdn:587 gives me an OK, so the container is trusting the letsencrypt certificate and cann connnect to it.

Any idea of changes that might have broken this in the last 2 months ?

[robguthrie]

First thing: I'd recommend you track the stable branch. For now that will work around this issue.

But yeah.. this is a problem to be fixed. I suspect this is something due to our update to Rails 7.0 recently.

You probably know, but we don't write our own SMTP stuff in Loomio, this is just Rails, but maybe we need to improve the config variables to allow you to get the settings right?

[airblag]

As a workaround, I edited the /loomio/config/application.rb in the loomio-worker container replacing

ssl: ENV['SMTP_USE_SSL'].present?,

with

ssl: false,

killing the sidekiq process and it then worked for me.

Now I tried again commenting the SMTP_USE_SSL in my .env and restarting the containers and it seems to work. I'm not sure what happened, maybe some stuff changed also with docker-compose pull

[robguthrie]

Great to hear you're back up again.