Mail not working after Update

[cruzope]

Hello,

we are now using Wekan version 5.90 and UCS version 5.0-1 errata177.

A few weeks ago we updated Wekan to an newer version, to be honest I can´t remember the actual version right now. Since this update the mail feature isn´t working anymore. We checked the config but the mail config in UCS hasn´t changed after the update. After the update the configuration fields in Wekan Administration Panel disappeared but that´s intentional if I´m not mistaken?

My next guess was to check the logs for errors, unfortunately I can´t find the logs for the mail feature.

Is there another way for us to troubleshoot the issue?

Thanks in advance.

Best regards.

[xet7]

Does this help?

https://github.com/wekan/wekan/wiki/UCS#3-mail-settings

[cruzope]

Thanks for the link. We already went through the steps and also had a look at https://github.com/wekan/wekan/wiki/Troubleshooting-Mail

But as mentioned we can´t find specific logs for the mail feature or wekan at all.

[t0mcat1337]

Same for me but with a complete fresh install of wekan 5.94 in UCS 4.4-8 errata1091. Can't get sending Mail make work, not even with the above mentioned links / docs.

This is what I've done and found out:

In UCS I configured the Mail URL as needed the way described here https://github.com/wekan/wekan/wiki/Troubleshooting-Mail in the "Example: UCS" Section. (BTW: In this section, the Mail Settings in Wekans Admin Panel are mentioned despite they are gone, as described on the same page at the top).

So the resulting URL is like this for example: smtp://ucsuser:ucspassword@ucs.host.name:25/?ignoreTLS=true&tls={rejectUnauthorized:false}&secure=false

The docker container is rebuilt. Now I login to Wekan with an UCS user that has admin permissions in Wekan. I open "Admin Panel" -> "Email" The first time I click the Button "Send a test email to yourself" leads to this error:

Sending email failed
Error trying to send email: Email protocol in $MAIL_URL ('smtp://ucsuser:ucspassword@ucs.host.name:25/?ignoreTLS=true&tls={rejectUnauthorized:false}&secure=false') must be 'smtp' or 'smtps'

WTF? There IS "smtp" as Email protocol as the URL is printed correctly in the SAME error message. And what is more: When clicking the button for a second time, this is the result:

"Email sent: ucsuser@ucs.domain"

BUT: The mail never arrived.

When displaying the docker container logs with:

docker logs wekan-app

this can be found:

====== BEGIN MAIL #0 ======
(Mail not sent; to enable sending, set the MAIL_URL environment variable.)
Content-Type: text/plain
From: 'Wekan Notifications '' <noreply.wekan@ucs.domain>
To: ucsuser@ucs.domain
Subject: SMTP Test Email
Message-ID: <f9639ee1-4f1b-76f8-c88c-e48c17097cb5@ucs.domain>
Content-Transfer-Encoding: 7bit
Date: Tue, 18 Jan 2022 01:09:19 +0000
MIME-Version: 1.0

You have successfully sent an email
====== END MAIL #0 ======

Notice the output at the top saying: "(Mail not sent; to enable sending, set the MAIL_URL environment variable.)" BUT: The MAIL_URL Env Var IS set in the container, as can be checked this way:

# docker exec -it wekan-app bash
wekan@2fda79fccd95:/$ echo $MAIL_URL
'smtp://ucsuser:ucspassword@ucs.host.name:25/?ignoreTLS=true&tls={rejectUnauthorized:false}&secure=false'

What is going wrong here? And why are the outputs so misleading ("EMail procol not smtp" -> but it IS; MAIL_URL not set -> but it IS)...

[xet7]

@t0mcat1337

What email server software you use?

[xet7]

@t0mcat1337

Does sending email with nodemailer work?

https://github.com/wekan/wekan/wiki/Troubleshooting-Mail#debugging-your-smtp-mail-server

With some known email service: https://nodemailer.com/smtp/well-known/

Or smtp: https://nodemailer.com/smtp/

Does it work with any nodemailer code examples?

Currently not all those options are in WeKan. They will be added later.

[xet7]

I currently use AWS SES for sending email, it works for me: https://github.com/wekan/wekan/wiki/Troubleshooting-Mail#example-aws-ses

I don't currently use other services for sending email.

[cruzope]

Hello,

at the moment we use Exchange Server Version 2013. t0mcat1337 has the exact problem as we have.

@t0mcat1337 are you using an Exchange Server as well?

[t0mcat1337]

@xet7 @cruzope I'm using the same UCS Server as Mail Server where wekan is installed. In the end this is a postfix. Other services on my network are using it, too with no special configuration, besides the standard ones (Host, Port, Encryption, User, Password)

[t0mcat1337]

@xet7 Just tested the nodemailer thing... This works without any problems with this code (translated the MAIL_URL to corresponding JSON vars in nodemailer code like this):

var nodemailer = require('nodemailer');

let transporter = nodemailer.createTransport({
  host: "ucs.host.name",
  port: 25,
  ignoreTLS:true,
  tls:{
         rejectUnauthorized:false
  },
  secure: false, 
  auth: {
    user: "ucsuser",
    pass: "ucspassword",
  },
})

let info = transporter.sendMail({
  from: 'Wekan Notifications <noreply.wekan@ucs.domain>',
  to: 'ucsuser@ucs.domain',
  subject: 'NODEMAILER TEST',
  text: 'TEST',
  html: '<h1>TEST</h1>'
})

As you can see... plain standard config for unencrypted SMTP at Port 25.

Even tested SSL encrypted (I'm using a valid Let's encrypt cert for this) on Port 465, which works without problems, too:

var nodemailer = require('nodemailer');

let transporter = nodemailer.createTransport({
  host: "ucs.host.name",
  port: 465,
  secure: true, 
  auth: {
    user: "ucsuser",
    pass: "ucspassword",
  },
})

let info = transporter.sendMail({
  from: 'Wekan Notifications <noreply.wekan@ucs.domain>',
  to: 'ucsuser@ucs.domain',
  subject: 'NODEMAILER TEST',
  text: 'TEST',
  html: '<h1>TEST</h1>'
})

[t0mcat1337]

@xet7 I even entered the docker container (docker exec...) and inside it tried nodemailer with the code above... works without problems, too...

[t0mcat1337]

@xet7 @cruzope I digged deeper and found the solution and answers to my questions in wekan source code:

1) Solution For Wekan in UCS the Mail URL is set in UCS itself in the App Settings. Here one enters a Mail URL. This results in a docker-compose.yml (located here: /var/lib/univention-appcenter/apps/wekan/compose) with an env var set like this:

environment:
...    
      # https://github.com/wekan/univention/issues/6
    - MAIL_URL='smtp://ucs.host.name:25/?ignoreTLS=true&tls={rejectUnauthorized:false}&secure=false'
...

Notice the ' surrounding the Mail URL String. This is what enters the Container... a Mail URL surrounded by ' . (BTW: This is also with "MAIL_FROM")

In /build/programs/server/packages/email.js this string then is parsed with "urlModule.parse(...)" and results in an mailUrl Object like this:

{ protocol: null,
  slashes: null,
  auth: null,
  host: null,
  port: null,
  hostname: null,
  hash: null,
  search: '?ignoreTLS=true&tls=%7BrejectUnauthorized:false%7D&secure=false%27',
  query: 
   { ignoreTLS: 'true',
     tls: '{rejectUnauthorized:false}',
     secure: 'false\'' },
  pathname: '%27smtp://ucs.host.name:25/',
  path: '%27smtp://ucs.host.name:25/?ignoreTLS=true&tls=%7BrejectUnauthorized:false%7D&secure=false%27',
}

which of course ends up in "Email protocol in $MAIL_URL" not being smtp or smtps, as the string with ' can't be parsed correctly (also notice the "%27" in url.path) and throwing the exception.

So this seems to be a bug in UCS creating the docker-compose.yml with ' in the MAIL_URL, although not being entered in the corresponding App Setttings Dialog in UCS.

Simply removed the ' in the docker-compose.yaml and mails are sent!

2) The different behaviour of the "Send a test email to yourself" button This is explained in the function "getTransport":

var getTransport = function() {
  // We delay this check until the first call to Email.send, in case someone
  // set process.env.MAIL_URL in startup code. Then we store in a cache until
  // process.env.MAIL_URL changes.
  var url = process.env.MAIL_URL;
  if (this.cacheKey === undefined || this.cacheKey !== url) {
    this.cacheKey = url;
    this.cache = url ? makeTransport(url) : null;
  }
  return this.cache;
}

As you see, when getTransport is called the first time during wekan runtime "this.cacheKey" is not set, so it enters the "if" trying to build the transport (makeTransport(url)) . In our case this errors out with the exception in "makeTransport" (see above). So "this.cache" becomes "null" and now "this.cacheKey" is set. When coming across this code the second time (press the Button "send a test mail..." again), the "if" isn't entered again as "this.cacheKey" contains the url string. So no "makeTransport" is called -> no exception, but "this.cache" (containing "null") is returned to the calling function "Emal.send":

Email.send = function (options) {
...

  var transport = getTransport();
  if (transport) {
    smtpSend(transport, options);
  } else {
    devModeSend(options);
  }
};

And here the "transport" gets "null" resulting in entering the "devModeSend" function, which just prints the Mail to stdout and so it can be seen with "docker logs....". But the Wekan Web GUI sais "Mail send", despite it never was.

Conclusions: To 1) Somehow the created docker-compose.yaml has to get rid of the ' in MAIL_URL (don't really know who is responsible) or The ' has to be removed from the url string in email.js before being parsed.

To 2) Perhaps "this.cacheKey" should only be set if makeTransport was succesful... or "this.cache" should also be checked in the "if"... and dont't say "mail send" in the GUI when it really is not being sent.

[t0mcat1337]

Again To 1) Finally I think somebody has to remove the ' from "/var/lib/univention-appcenter/apps/wekan/compose/docker-compose.yml.template" here:

    environment:
  ...
      # https://github.com/wekan/univention/issues/6
      - MAIL_URL='@%@wekan/MAIL_URL@%@'
      - MAIL_FROM='@%@wekan/MAIL_FROM@%@'
      - WITH_API=true

[xet7]

@t0mcat1337

Thanks! I'll look at Univention WeKan docker-compose.yml content.

Another option would be that in WeKan email.js there I would search-replace those %27 etc

[xet7]

@t0mcat1337

It seems you are right!

In config, where I usually change WeKan docker image version, was this:

      - MAIL_URL='@%@wekan/MAIL_URL@%@'
      - MAIL_FROM='@%@wekan/MAIL_FROM@%@'

So I changed it to this:

      - MAIL_URL=@%@wekan/MAIL_URL@%@
      - MAIL_FROM=@%@wekan/MAIL_FROM@%@

And did click Save and Approve for release for WeKan 5.95.

Probably I will also do new release for WeKan, to add translations etc.

[xet7]

I think that the problem is mostly about removing those single quotes surrounding MAIL_URL etc.

For WeKan MAIL_URL etc, it should be OK that URL itself is urlencoded.

We'll see when Univention WeKan 5.95 becomes published sometime.

[t0mcat1337]

Sure... glad I could help a little :)

[xet7]

@t0mcat1337

Thanks, it's a huge help! Some others have also mentioned about email problems at Univention WeKan, but I had no idea where to even start figuring it out.