shubhank008 commented 3 years ago

Issue Summary

Currently, using Ghost v4.0, if you try to update the emails in your Admin Settings of Ghost under Members->Email and try to update either support or newsletter, they are set to noreply@domain.tld
They should ideally use the "from" address if set in your config's mail part.

To Reproduce

The bigger problem is not this, its the fact that if you set custom SMTP and even "from" address in your config's mail part, and try to update the emails given in your Admin Area above, Ghost ignores whatever is set in your config and still uses noreply@domain.tld or no-reply@domain.tld which creates issue with SMTP config as when Ghost tries to send such mails (confirmation mails), the SMTP server will reject it as the "from" mail is different from the actual userid/smtp emailid.

Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead?

This is how postfix (SMTP) logs it:

NOQUEUE: reject: RCPT from unknown[xx.xx.xx.xx]: 553 5.7.1 <noreply@dly.in>: Sender address rejected: not owned by user user1@dly.in; from=<noreply@dly.in> to=<user1@dly.in> proto=ESMTP helo=<[127.0.0.1]>

The logic for this part (and how I fixed it) lies here:
core/server/services/members/settings.js

This is how Ghost currently does it

const [,toDomain] = email.split('@');
        let fromEmail = `noreply@${toDomain}`;
        if (fromEmail === email) {
            fromEmail = `no-reply@${toDomain}`;
        }

Ideally it should atleast check if a from address exists in your config.mail or use the smtp id instead. This just limits the usage of mail and SMTP in general to a very handful of providers.

I had to fix it by overriding fromEmail to use email itself and then I was able to update the mail in my Admin:

let fromEmail = `noreply@${toDomain}`;
        if (fromEmail === email) {
            fromEmail = `no-reply@${toDomain}`;
        }
        fromEmail = email;

Technical details:

Ghost Version: v4.0
Node Version:
Browser/OS: Chrome
Database: MySQL

ErisDS commented 3 years ago

Hey @shubhank008, I hear that you are frustrated but I have to be honest. I've read this 3x and I don't quite follow what you are saying.

Please can you go back to the issue template we provided and supply a step by step reproduction case demonstrating the problem so that we can try to reproduce it?

shubhank008 commented 3 years ago

Hey @shubhank008, I hear that you are frustrated but I have to be honest. I've read this 3x and I don't quite follow what you are saying.

Please can you go back to the issue template we provided and supply a step by step reproduction case demonstrating the problem so that we can try to reproduce it?

Woops sorry, edited my initial issue above. I am not frustrated as I found a way to fix it (albeit using a solution provided by another user in ghost forums facing same issue and pointing the problem in settings.js).

To try and explain it a bit more:

In ghost admin panel, under Members settings, if we try to update the email ids used for Newsletters and Support, they are set to noreply@yourdomain.com when ideally they should be set to what we have set as the emailid in our config.json file (the mail.from config field).
So even a user sets a "from" email ID to be used in config.json, its not used by ghost for support and newsletters as they continue to use that noreply id which is set through the "core/server/services/members/settings.js" file.
Now you would think, the fix is as simple as just updating the default email id in Admin panel from "noreply" to youremail@domain.com but here is the problem, when you try to make that update, Ghost sends a confirmation mail to that new email id you are trying to set and it tries to do so using the current email id that is set in Members area, which is "noreply@"
The problem that follows after it is there is no such id "noreply@" if using SMTP and as such, the confirmation mail is never sent and instead the SMTP server responds with a error shows in my initial post stating this email ID is not correct and different from your SMTP details (i.e. in our SMTP we are using youremail@ but Ghost sends that mail with noreply@)
The proper way to do this should have been to honour and use the actual ID that is set in the config.json file (config.mail.from) as you can see from my temporary solution to bypass that noreply check, the moment we do that and use actual emailID directly, it works.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

mascali33 commented 3 years ago

This is still a problem, using Ghost 4.18 with a custom SMTP server, ghost is not using the from adress specified in config and uses noreply@blog-domain.tld instead of mysmtpuser@mailserver.tld like said in config. Ghost should only use the noreply@domain.tld when no "mail__from" value is set

EDIT: if it can help my smtp server says this in the logs

2021-10-16T00:16:28.006942750Z Error: Can't send mail - all recipients were rejected: 553 5.7.1 <noreply@blog-domain.tld>: Sender address rejected: not owned by user mysmtpuser@mailserver.tld
2021-10-16T00:16:28.006948759Z at SMTPConnection._formatError (/var/lib/ghost/versions/4.18.0/node_modules/nodemailer/lib/smtp-connection/index.js:784:19)
2021-10-16T00:16:28.006957568Z at SMTPConnection._actionRCPT (/var/lib/ghost/versions/4.18.0/node_modules/nodemailer/lib/smtp-connection/index.js:1626:28)
2021-10-16T00:16:28.006963505Z at SMTPConnection.<anonymous> (/var/lib/ghost/versions/4.18.0/node_modules/nodemailer/lib/smtp-connection/index.js:1579:30)
2021-10-16T00:16:28.006969228Z at SMTPConnection._processResponse (/var/lib/ghost/versions/4.18.0/node_modules/nodemailer/lib/smtp-connection/index.js:947:20)
2021-10-16T00:16:28.006975068Z at SMTPConnection._onData (/var/lib/ghost/versions/4.18.0/node_modules/nodemailer/lib/smtp-connection/index.js:749:14)
2021-10-16T00:16:28.006980988Z at TLSSocket.SMTPConnection._onSocketData (/var/lib/ghost/versions/4.18.0/node_modules/nodemailer/lib/smtp-connection/index.js:189:44)
2021-10-16T00:16:28.006987058Z at TLSSocket.emit (events.js:400:28)
2021-10-16T00:16:28.006992786Z at addChunk (internal/streams/readable.js:293:12)
2021-10-16T00:16:28.006998398Z at readableAddChunk (internal/streams/readable.js:267:9)
2021-10-16T00:16:28.007003719Z at TLSSocket.Readable.push (internal/streams/readable.js:206:10)
2021-10-16T00:16:28.007018429Z at TLSWrap.onStreamRead (internal/stream_base_commons.js:188:23)
2021-10-16T00:16:28.007025055Z
2021-10-16T00:16:28.009970197Z [2021-10-16 00:16:28] INFO "POST /members/api/send-magic-link/" 500 368ms

as you see in the logs, I've set the from address to be "mysmtpuser@mailserver.tld" but ghost doesn't send it with this address

mascali33 commented 3 years ago

Its also in the docs https://ghost.org/docs/config/#from-address

Ping @ErisDS

shubhank008 commented 3 years ago

Its also in the docs https://ghost.org/docs/config/#from-address

Ping @ErisDS

Quite disappointed has been months since it was reported and in so much details and it was a 2 line code fix (well rather override the config) but the bug its still there. Makes me wonder do not a whole lot ghost users use SMTP or check/test their mails ?

mascali33 commented 3 years ago

Its also in the docs https://ghost.org/docs/config/#from-address Ping @ErisDS

Quite disappointed has been months since it was reported and in so much details and it was a 2 line code fix (well rather override the config) but the bug its still there. Makes me wonder do not a whole lot ghost users use SMTP or check/test their mails ?

Sometime it seems they don't see the issues :\ Can you try to open a pull request fix and reference this issue? Maybe they will see the problem :\

ErisDS commented 3 years ago

Reopening this as a valid bug.

There is some weird interplay between the bulk email and SMTP email settings going on. When trying to setup the support and newsletter settings, we send an email to try to validate that the email address being set is usable.

Although these settings are for bulk email, the emails we send are sent via SMTP. The logic for what the from address should be needs reviewing.

It may be a case that we can simply change this to use the SMTP "from" address, but I will need to check how this behaves on Ghost(Pro) where we make use of the ability to setup mailgun via config rather than the UI.

Does someone want to put together a PR for this change and we can collaborate on testing all the cases?

github-actions[bot] commented 2 years ago

Our bot has automatically marked this issue as stale because there has not been any activity here in some time.

The issue will be closed soon if there are no further updates, however we ask that you do not post comments to keep the issue open if you are not actively working on a PR.

We keep the issue list minimal so we can keep focus on the most pressing issues. Closed issues can always be reopened if a new contributor is found. Thank you for understanding 🙂

tryfan commented 2 years ago

This is still an issue.

Supplying mail__from as an env var does not change the email from address in a container. Just tested on 4.36.1.

ErminMerdanovic commented 2 years ago

I encounter the issue today as well.

mgk1231 commented 2 years ago

This is still a valid bug. As of today this issue is not yet resolved

Antonyrahul commented 2 years ago

Facing this issue. Is there any temporary workaround for now. I am using aws smtp

shubhank008 commented 2 years ago

Facing this issue. Is there any temporary workaround for now. I am using aws smtp

You can use the temporary codefix I listed in my original report (the very top post)

Antonyrahul commented 2 years ago

Facing this issue. Is there any temporary workaround for now. I am using aws smtp

You can use the temporary codefix I listed in my original report (the very top post)

I did that fix and restarted the server and the mail is still being sent from noreply. Is there something i am missing . I have cleared the cache and tried too but it does not work

marnixhoh commented 2 years ago

Still running into the same issue. Related:

15191

15222

So many people must be impacted by this...

shubhank008 commented 2 years ago

Anyone still facing this god forsaken issue, my fix still works except the location/file where you need to make the change has changed in latest ghost (v5) from core/server/services/members/settings.js to core/server/services/settings/settings-bread-service.js

And don't forget to set the from field in your config's mail section

"mail": {
    "from": "'Name' <something@domain.tld>",
    "transport": "SMTP",
    "options": {
      "auth": {
        "user": "something@domain.tld",
        "pass": "123456"
      },
      "host": "mail.domain.tld",
      "port": 587
    }
  },

The logic for this part (and how I fixed it) lies here: core/server/services/members/settings.js

This is how Ghost currently does it
const [,toDomain] = email.split('@');
        let fromEmail = `noreply@${toDomain}`;
        if (fromEmail === email) {
            fromEmail = `no-reply@${toDomain}`;
        }
Ideally it should atleast check if a from address exists in your config.mail or use the smtp id instead. This just limits the usage of mail and SMTP in general to a very handful of providers.

I had to fix it by overriding fromEmail to use email itself and then I was able to update the mail in my Admin:
let fromEmail = `noreply@${toDomain}`;
        if (fromEmail === email) {
            fromEmail = `no-reply@${toDomain}`;
        }
        fromEmail = email;

marnixhoh commented 2 years ago

I had this issue too, and I found the solution, but forgot to post it here...

Anyway, after having created multiple issues here on Github and on the Ghost Forum, someone from their staff gave the answer on this forum thread.

I hope it helps you and others :)

ErisDS commented 2 years ago

@marnixhoh I think you are confusing a different issue. @shubhank008 If you have a fix for the original issue, please raise a PR.

JTonyC commented 2 years ago

Hi, I'd also like to provide more detail around this issue, caused a problem for me. Bit of background, I've successfully deployed an azure multi-container instance, with persistent storage (Azure File Share). It uses caddy docker image for lets-encrypt and reverse proxy, the mysql-8 docker image for DB and the docker image for ghost v5.

My compose file looks like this:

environment:
      url: ${GHOSTURL}

      mail__from: ${MAILFROM}
      mail__transport: ${MAILTRANSPORT}
      mail__options__host: ${MAILHOST}
      mail__options__port: ${MAILPORT}
      mail__options__secureConnection: false
      mail__options__auth__user: ${MAILUSER}
      mail__options__auth__pass: ${MAILPASS}

The accompanying local .env file defines these values as the MAILFROM and GHOSTURL.

MAILFROM=noreply@domain.tld GHOSTURL=https://blog.domain.tld

I also have my own postfix box. If a member attempts sign up, the mail from address used by Ghost is: noreply@blog.domain.tld, which is ignoring the 'mail__from:' set in the environment, further resulting in googlemail rejecting it as spam. This is because as you can see from the logs below, the mail has a from address of the default, 'noreply@blog.domain.tld' (undesired) and not 'noreply@domain.tld' as expected and defined in environment. See below [redacted] postfix mail log.

ghost-postfix

I have found though, going into /ghost/#/settings/members?showPortalSettings=true, under support email address, shows that it is indeed noreply@blog.domain.tld. Changing this too the desired email and confirming ownership by receipt, will change the address used for member sign-ups, after doing so, I now get this in the postfix logs.

ghost_good_mail_send

Status 250 = Ok is basically smtp speak for, we got and will deliver your mail, sure enough, the google account received the mail.

So, why isn't the environment variable defined for 'mail__from' being honoured? Restarting the container group does save the address so at least membership/transactional mails are functional, took a while to work this out mind.

Also, why the constraint for mailgun as bulk mail sender? SendGrid for example, offers 100 mails a month for free, using your own domain, whereas mailgun has an associated cost for the same service.

Better value proposition for a blogger just starting out.

Apologies for the long post, just thought it might help someone struggling with this same issue and to help understand and narrow down what needs fixing in a PR.

Great app btw, thanks

lnlclay commented 2 years ago

I had this issue too, and I found the solution, but forgot to post it here...

Anyway, after having created multiple issues here on Github and on the Ghost Forum, someone from their staff gave the answer on this forum thread.

I hope it helps you and others :)

I thought my postfix box was broken until I find this issue here, 18 months later still not fixed. This is not helping since my postfix server is not configured to receive email, thus no way to confirm the link it's sending.

lvnilesh commented 1 year ago

Hope this PR gets merged soon https://github.com/TryGhost/Ghost/pull/15989

github-actions[bot] commented 1 year ago

Our bot has automatically marked this issue as stale because there has not been any activity here in some time.

The issue will be closed soon if there are no further updates, however we ask that you do not post comments to keep the issue open if you are not actively working on a PR.

We keep the issue list minimal so we can keep focus on the most pressing issues. Closed issues can always be reopened if a new contributor is found. Thank you for understanding 🙂

jenishngl commented 1 year ago

Hi, I'd also like to provide more detail around this issue, caused a problem for me. Bit of background, I've successfully deployed an azure multi-container instance, with persistent storage (Azure File Share). It uses caddy docker image for lets-encrypt and reverse proxy, the mysql-8 docker image for DB and the docker image for ghost v5.

My compose file looks like this:
environment:
      url: ${GHOSTURL}

      mail__from: ${MAILFROM}
      mail__transport: ${MAILTRANSPORT}
      mail__options__host: ${MAILHOST}
      mail__options__port: ${MAILPORT}
      mail__options__secureConnection: false
      mail__options__auth__user: ${MAILUSER}
      mail__options__auth__pass: ${MAILPASS}
The accompanying local .env file defines these values as the MAILFROM and GHOSTURL.

MAILFROM=noreply@domain.tld GHOSTURL=https://blog.domain.tld

I also have my own postfix box. If a member attempts sign up, the mail from address used by Ghost is: noreply@blog.domain.tld, which is ignoring the 'mail__from:' set in the environment, further resulting in googlemail rejecting it as spam. This is because as you can see from the logs below, the mail has a from address of the default, 'noreply@blog.domain.tld' (undesired) and not 'noreply@domain.tld' as expected and defined in environment. See below [redacted] postfix mail log.

I have found though, going into /ghost/#/settings/members?showPortalSettings=true, under support email address, shows that it is indeed noreply@blog.domain.tld. Changing this too the desired email and confirming ownership by receipt, will change the address used for member sign-ups, after doing so, I now get this in the postfix logs.

Status 250 = Ok is basically smtp speak for, we got and will deliver your mail, sure enough, the google account received the mail.

So, why isn't the environment variable defined for 'mail__from' being honoured? Restarting the container group does save the address so at least membership/transactional mails are functional, took a while to work this out mind.

Also, why the constraint for mailgun as bulk mail sender? SendGrid for example, offers 100 mails a month for free, using your own domain, whereas mailgun has an associated cost for the same service.

Better value proposition for a blogger just starting out.

Apologies for the long post, just thought it might help someone struggling with this same issue and to help understand and narrow down what needs fixing in a PR.

Great app btw, thanks

Changing setting under /ghost/#/settings/members?showPortalSettings=true indeed solved my issue

phillipnhenry commented 1 year ago

It seems Ghost core has zero interest in the issues that are collected then shelved in this Issue.

Interesting thing is GhostPro behaves similarly but they offer a fix. Upgrade to "business" plan and pay $$$$$ "though this does require a Business plan, alongside a $50/month add-on for support"

Correction: from support, " rather than seeing the 'via m.ghost.io' it would be 'via your-custom-sending-domain.com', e.g. the custom sending domain instead. "

@ErisDS @JohnONolan If I am mistaken and there is a good reason for how these issues are being handled please let me know.

ErisDS commented 1 year ago

Please see my responses over on the PR here: https://github.com/TryGhost/Ghost/pull/15989#issuecomment-1766567698

TL;DR we have a piece of work to change how from addresses work in a holistic way, that is scheduled to be started on immediately.

phillipnhenry commented 1 year ago

Please see my responses over on the PR here: #15989 (comment)

TL;DR we have a piece of work to change how from addresses work in a holistic way, that is scheduled to be started on immediately.

@ErisDS Thanks for getting the ball rolling on fixing the various issues with how ghost deals with mail. Please provide specifics explaining what change how from addresses work in a holistic way means.

Will we have appropriate control over setting email from addresses consistent with Mailgun and others? Will the sent via nonsense be a thing of the past? Will Ghost begin setting mail envelopes consistent with industry standards?

It would set some minds at ease if the plan was actually shared with folks rather than springing the result on us. We all saw how the surprise approach works vis-a-vis the admin settings UI change debacle.

henrywithu commented 1 year ago

The issue remains. For "Support email address", why is it fixed? (I'm using docker) It's like being forced to send verification email from noreply@example.com, and it's hardcoded

ivosabev commented 11 months ago

How is it possible this issue to not be fixed for 2 years?! I am very close from moving to a different platform, because of something so simple.

henrywithu commented 11 months ago

How is it possible this issue to not be fixed for 2 years?! I am very close from moving to a different platform, because of something so simple.

Actually I solved it (if you really want to use personal email to send newsletter or support), by setting email alias like noreply@example.com and no-reply@example.com in your SMTP service provider. Anyway it's NOT recommended to use personal email to do so, because there're chances that the email is considered as spam.

JohnONolan commented 11 months ago

Google and Yahoo are introducing major changes to enforce DMARC in the next 2 months. The core team is currently doing a larger piece of work to support DMARC alignment in email sending, which overlaps-with and will also resolve this issue.

These changes will be released in January 2024.

adamu commented 10 months ago

I just set up Ghost in Docker, made sure I configured the mail__from environment variable (and confirmed it's set in the container), tried to make a test subscription request and:

ghost  | [2024-01-27 14:28:25] WARN Invalid from address used for sending emails: noreply@localhost
ghost  | [2024-01-27 14:28:26] ERROR "POST /members/api/send-magic-link/" 500 787ms
ghost  |
ghost  | Failed to send email. Reason: Message failed: 554 Message rejected: Email address is not verified. The following identities failed the check in region AP-NORTHEAST-1: noreply@localhost.
ghost  |
ghost  | "Please see https://ghost.org/docs/config/#mail for instructions on configuring email."

SimonBackx commented 10 months ago

🎉 This issue and related issues have been fixed in v5.78.0.

Changes in Ghost v5.78.0

The verification step is dropped when changing the newsletter or support address. You can choose any support and newsletter email address in the UI as long as your SMTP-server / Mailgun account can send from it.
The mail.from config will be the default address for all outgoing emails:
- Staff notification emails no longer use the made up ghost @ domain email address
- Newsletters no longer default to noreply @ domain if no from address is set
- Member related emails (signin/signup/comment notifications...) will continue to be send from the chosen support address (Portal settings → Account page), but will now default to the mail.from config instead of noreply @ domain if no support address is set.

What do you need to do

Update Ghost to v5.78.0
You don’t need to make any changes to your existing configuration if you already set the mail.from config. If no mail.from config is set, we’ll continue to default to noreply@domain everywhere, just like before.

We've updated the documentation about the config, and recommend to check out our post on the forum about DMARC alignment and the new requirements from Google and Yahoo.

sonyarianto commented 2 months ago

How about create config for mail.support in addition to mail.from, so mail.from is only for sender from address, but the mail.support can be any e-mail address that intended for support e-mail, but this support e-mail will not treated as sender e-mail address when Ghost sending communication to user/member.

I think this will makes the configuration more flexible and accomodate several use cases.

MFYDev commented 1 month ago

3 years later and this issue is still happening https://github.com/TryGhost/Ghost/issues/21423

TryGhost / Ghost

The from address on the confirmation email for updating email in Admin Settings (Support/Newsletter) is incorrect #12802

Issue Summary

To Reproduce

Technical details:

15191

15222