MailAdapter design

[drew-gross]

Hey, we are super excited to see everybody jumping to help with email verification! Since so many people are working on this, I thought it would be a good idea to get everyone working on it together to discuss what the API should look like. We want to have the smallest API possible to reduce the burden on implementers of the interface, while still being enough to satisfy the needs of Parse.

We also want to open the door for implementers of the MailAdapter interface to bring some new innovations to the table, such as customizing the emails content based on fields on the Parse.User object.

I think the ideal interface for a MailAdapter to expose would be a small handful of functions: sendEmailVerificationEmail(user, link) to send the email on signup, and sendPasswordResetEmail(user, link) for password resets. We could also ask the MailAdapter provider to provide the 4 user facing page templates with some other functions such as generateChooseNewPasswordPage(user), generatePasswordChangedPage(user), generateEmailVerifiedPage(user) and generateInvalidLinkPage(user).

To kick off the conversation about what else we might want from this interface, here some ideas to discuss:

A sendPasswordResetSuccess(user) function. A lot of services will email you when you change your password, as a defense against hacking. Maybe the Parse server should also do this.

Should the user facing pages generation be mandatory? We could provide defaults, or choose not to.

Should we think of a way to expose the MailAdapter in Cloud Code? That would increase the burden on the implementer of MailAdapter, as they would have to create a very general purpose interface, but it could also be very useful.

Parse.com implements user facing pages by asking you to provide a link to a template. Should we re-use that concept? It would aid in transitioning off of Parse.com.

Do we want to give the MailAdapter any other information to work with? Like a Parse.Installation object, maybe? We could even give the adapter the full power of the Parse JS SDK, so it can add some features like the ability to write to the user object, so it can, for example, save when the password was last reset. Of course, any MailAdapter can accept data in it's constructor that lets it do that anyway, but we could make it easier.

I'll leave the 3 PRs working on email verification open, and we can take the best ideas from those 3 PRs, plus what we can come up with here.

@taylorstine @jamiechapman @maysale01 @gfosco @nlutsenko @lucianmat @flovilmart

[flovilmart]

Top of my mind and architercturally speaking:

Mail requests should be stored in separated objects, leaving just emailVerified in _User.

We should pass the full Parse.User object to the methods, that would allow external implementation to have a better control, tracking of sends etc.. Installation seem irrelevant at that point as it is unlikely to be linked to a particular User.

[flovilmart]

The question about the templates begets a bigger question. What is the use case of parse-server? Standalone or as a module. Personally, I'd go the standalone approach, to replicate, and isolate the concerns.

Going the standalone approach, implies that the templates should be provided by configuration. Do we want to go that way?

[drew-gross]

I agree with the standalone approach and having the templates provided by choosing a configuration (ie. by choosing a MailAdapter) having the configuration be done in JS at the time of MailAdapter instantiation could be very powerful, and let the power users do awesome things by using an advanced MailAdapter, while also letting the beginner developers get started with a simple MailAdapter that chooses most of the defaults automatically.

Storing mail requests in a separate object is a great idea, but enabling that by default would cause backwards compatibility issues due to existing apps potentially already having data in that place. Making it possible for an advanced MailAdapter that stores requests wherever the user wants sounds like a good idea though.

[gnz00]

I personally believe that email handling and template rendering are beyond the scope of the Parse Server project. I'm leaning towards Express plugin over standalone.

We could effectively plug this whole issue by adding some CloudCode mechanism to hook into. For example, a user could import a parse-server/cloudcode-mailgun-hooks plugin that automatically registers hooks for certain classes, i.e. email verification, password change notifications, password reset for _User with Mailgun, or write their own handlers with any node library they want.

However, if the consensus is to include some core mailing ability, here is an approach that keeps things flexible https://gist.github.com/maysale01/5390485e676ee3745960. Could also add a locals hook to each template definition:

const myMailTemplates = {
    password_reset: {
        html: require("./views/password_reset_html.js")(dust), // Precompiled dust template
        text: require("./views/password_reset_text.js")(dust),
        locals: function(req) {
            return {
                title: "Some hardcoded stuff",
                name: `${req.user.firstname} ${req.user.lastname}`,
                address: [`${req.user.addr_num} ${req.user.addr_street}`, req.user.addr_city, req.user.addr_state, req.user.addr_zip].join(","),
                tracking: req.Parse.server.analyticsProvider.tracking
            }
        }
    }
};

Sidebar: logic for a shouldSendUserConfirmation shouldn't be baked directly into RestWrite.js (or at least not in the runDatabaseOperation method), rather there should be a step in the handler flow where a model is validated against some configurable constraints. Possibly a system defined afterSave trigger for '_User' that checks configuration to see if email verification is required, a validator service, or model classes with static validate methods.

[flovilmart]

Also, for the whole modularization of the project, we should be able to pass a module name, from configuration or command line in order to load a 3rd party adapter when it makes sense. In this scenario, we could decouple the main package from the adapters, providing base adapters, but letting anyone who wants a sendgrid adapter use that one without actually requiring any code besides the configuration.

[flovilmart]

The mail adapter should also provide a simple send method that would take standard parameters.

[drew-gross]

That seems reasonable, they would really need to implement that in order to make the other functions anyway.

[corbanb]

Are there any ideas on when this will be merged in for us to start working off of? Seems like its gone quite here for a little bit.

[flovilmart]

@corbanb there are multiple concerns about the mail sending architecture and actually adapter architecture in general: in #290.

[rendragon83]

Is this still being worked on or ready? I have been going through all the conversations about this and it looks like all of the other ones have been closed except this one.

[jamiechapman]

@rendragon83 not sure, not much discussion has happened for a couple of weeks regarding this side of things. I guess we probably need to reach a decision soon so we can ensure that password resets etc are working for standalone API server(s).

[taylorstine]

Let's hop on the green button express with one of these two https://github.com/ParsePlatform/parse-server/pull/250 https://github.com/ParsePlatform/parse-server/pull/583

[corbanb]

+1

[maruthi-wal]

@flovilmart @gfosco

If any one did the password reset functionality, Please provide me the steps??

I need to implement this feature with my local parse server.

Thanks, Maruthi.

[drew-gross]

Hey, @maruthi-wal, Parse Server 2.1.4 has experimental support for sending password reset emails. Because it's still experimental, it's not documented and could change in the next version. I wouldn't recommend using it in production just yet, but if you want to help us test the feature before marking it as non-experimental, you can set verifyUserEmails: true in your Parse Config, and add an email adapter such as our SimpleMailgunAdapter.

[dcdspace]

I use Sendgrid with custom email templates for transactional email via cloud code, so would I be able to send password reset/email verification emails using Sendgrid as oppose to Mailgun? It would be great if I could simply access the url to the reset page and pop that in my email HTML. Thanks!

[drew-gross]

That is coming and will be possible by the time password reset becomes non-experimental

[432player]

Verification and reset emails not working since migrating my Parse app's MongoDB

I've completed step 1 of my Parse migration by moving to my own MongoDB server. Everything is working fine except emails sent by Parse.com. Again, I've only migrated the DB so the cloud code (and Parse REST API) are still being hosted by Parse.com.

Clicking on any email verification or password reset email link ends with the "Invalid Link" error page. I've confirmed the token and username in the emails are correct.

My only thought is the Parse.com hosted email verification handler is not able to communicate with the external MongoDB server I migrated to (but the the Parse.com REST API is working fine so that doesn't make sense).

A CLUE THOUGH:

I've noticed that the reset link sent to my email worked fine before the migration, and a minute after the migration it creates a new link with a new _perishable_token in the mongoDB and is the same in the new link sent in the email but is an Invalid Link. And the funny part is the link before the migration to mongoDB STILL WORKS. It's like the old _perishable_token before the migration is somehow cached in the Parse server and allows me to even reset the password as many times as I like (each time same old token before the migration) but the new tokens being generated are not being checked in the request_password_reset Rest API function I guess.

Any ideas? Is this happening to anyone else?

[flovilmart]

@432player how did you configure parse-server?

[432player]

The Parse Server is still running on Parse.com. We havent migrated to our own Parse-Server. We've migrated only the DB to our mLab mongoDB

[flovilmart]

So that seems unrelated to this issue, as it's a migration problem and you're not running parse-server at all

[432player]

True, but we tried asking this everywhere and no one seems to have anything to say about this. This problem is apparently happening in the request_password_reset and without migrating to our own Parse-Server there seems nothing we can do about it. But we still want to remain a while in Parse before migrating the whole service. So I believe this is a bug in the Parse Rest API and with a lot of hope that someone will recognise this issue to allow us to remain on Parse.com a while longer. The new _perishable_tokens are not being checked correctly when the mongoDB is self hosted.

[silvermana]

@flovilmart @432player +1. I'm having the same issue. I'm close to launching the app I've been working on, and wanted to only migrate the MongoDB for now (as was suggested by the timeline in Parse's migration guide).

If I remove all my custom email templates it works fine. So I agree with @432player that this is a Parse.com API bug, or a bug in the email templates.

[mahabubakram]

Thanks @drew-gross and @flovilmart for the support. As I am using the mandrill for email service. I am waiting for the custom email provider and also for the password reset. Hopefully next version will come with a nice wiki on it. As its quite crucial.

[corbanb]

@drew-gross where can I find info on the mail adapters? I'd like to start testing here with verifyUserEmails: true and the mail adapter if possible.

[drew-gross]

We don't have any documentation because the feature is still experimental. To test, you can import our SimpleMailgunAdapter, initialize it with fromAddress and mailgun info, and pass it in your server initialization in the emailAdapter parameter, as well as using the verifyUserEmails option

[432player]

Hi @drew-gross , we would like to turn on the email verifications option on the Parse-Server. We have a SimpleMailgunAdapter running and working well with the reset password. I tried adding the verifyUserEmails: true to the ParseServer Init but it didn't do the trick. var api = new ParseServer({ ... verifyUserEmails: true, emailAdapter: simpleMailgunAdapter, ... });

any ideas on why this is not working? by the way we're running npm 2.1.6 Thanks

[drew-gross]

Thanks for testing experimental features! Can you be more specific about what isn't working? Do you receive an email with no link? Do you receive a link that doesn't set emailVerified to true? Does any email get sent at all? (check you mailgun logs)

[432player]

Hey @drew-gross thanks for the rapid reply, we're experiencing no email being sent at all. For the reset password we do get emails and they reset the password. But on signup we do not receive an email at all for verification even when verifyUserEmails: true, is set to true. Thanks

[drew-gross]

Can you try opening the SimpleMailGunAdapter file and adding a console.log statement inside the sendEmail function to see if it is getting called? Also, do you have any beforeSave or afterSave on the _User? If so, could you try removing them and seeing if that changes anything?

[432player]

Very strange, I was just about to test what you said and suddenly it is now working. I'm not really sure how to approach this now. Perhaps it needed a while to jump start. I've tested this twice now and all is working. I've added the verifyUserEmails: true about an hour ago.

[thamer898]

Hello @flovilmart @drew-gross

Do you have any tutorial how to configure Reset user password using parse version 2.1.6 ?

Regards

[gowridev]

How to integrate SMTP server for email validation functionality in parse server?

[hshinde]

@432player Did you find an answer to your problem? I am facing the same issue.

[432player]

Hi @hshinde it worked for but I'm not really sure how. Just make sure your Parse server version is above 2.1.4 and add verifyUserEmails: true at the init of ther server

[andresgarza]

@hshinde / @432player The README was recently updated with basic instructions on how to setup the email adapter.

[ChrisGrant]

Is it possible to use the MailAdapter in CloudCode? I can't see an obvious place where it is exposed.

[hshinde]

@432player thanks. My bad. I did not elaborate the issue. I am facing the original problem where we have migrated the database to MongoDB and still using Parse.com server. It seems you had to migrate your Parse server to make the email verification work. I will try that approach too. But I was still wondering whether there is any solution to make it work with Parse.com servers.

[drew-gross]

@ChrisGrant It's currently not exposed by Parse Server but you can just require the sendgrid or mailgun package in your Cloud Code, initialize it according to it's docs, and then send email with it.

[ChrisGrant]

@drew-gross thanks. Are there any plans to do so? It'd be more convenient than having to initialize it twice in my opinion. I can open another issue if you think it's worth it?

[drew-gross]

I think it's an interesting idea, but there are some big open questions about how it would work. Mailgun, for example, requires that you provide a fromAddress to send any email. This is provided in the initialization of the adapter, which means that it can't be changed if you send email from Cloud Code - but I'm sure people would want to change it. If we start adding a top of options to the email adapter, though, it gets harder to implement, and also makes it harder to swap different email adapters in and out easily. Lots of tradeoffs to consider. Opening an issue to discuss might be a good idea, yes.

[jcguarinpenaranda]

Hello. So, is the email verification working by now?

[hshinde]

yes, we have tested it with Mandrill. It's working without any issue.

On Tue, Aug 30, 2016 at 9:40 PM, Juan Camilo Guarin Peñaranda < notifications@github.com> wrote:

Hello. So, is the email verification working by now?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ParsePlatform/parse-server/issues/275#issuecomment-243492299, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDbL-fn7WMhXCQdAKYiHsir8Go28ngZks5qlFXYgaJpZM4HUxNv .

[flovilmart]

It should be working by now.

Follow the documentation in the wiki if you have any doubts.

[artonragsdale]

I'm no longer on that project so I'm not certain, but I believe it was fixed.

On Tue, Aug 30, 2016 at 12:09 PM, Juan Camilo Guarin Peñaranda < notifications@github.com> wrote:

Hello. So, is the email verification working by now?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ParsePlatform/parse-server/issues/275#issuecomment-243492299, or mute the thread https://github.com/notifications/unsubscribe-auth/AADYe_dVaV0A6YLAXKQJjn3Obhfel9J5ks5qlFXXgaJpZM4HUxNv .

[alarrosa14]

For those looking for custom templates for password reset and verification emails, go to parse-server-mandrill-adapter. v1.2.0 of that adapter supports Mandrill's templates.

[rowntreerob]

FWIW : native parse-server && std-parse-mailgun-adapter working fine in localhost / express testing..

mongo connected to DEV instance on mLab's cloud ( NOT localhost/mongo )

Details of node config ( using heroku's $HOME/proj/.env ) to inject local env...

NOTE: - check for inmail in 'spam' folder.

env:
DATABASE_URI=mongodb://me:34@ds0.mlab.com:13931/beacontst
SERVER_URL=http://localhost:1337/parse
PUBLIC_SERVER_URL=http://localhost:1337/parse

index.js:
new ParseServer({
  databaseURI: databaseUri || 'mongodb:..',
  serverURL: process.env.SERVER_URL || 'https://mighty-id-65221.herokuapp.com/parse',
  verifyUserEmails: true,
  publicServerURL: process.env.PUBLIC_SERVER_URL || 'https://mighty-id-65221.herokuapp.com/parse',
  emailAdapter: {
  module: 'parse-server-simple-mailgun-adapter',
  options: {
    fromAddress: 'no-reply@yayatv.tv',
    domain: 'sandbox2babd8b07fca4fe81e.mailgun.org',
    apiKey: 'key-f16d78'
  }

[gowridev]

Can we send mail invites using cloud code which uses mail adapter? If possible,could you share sample cloud code?

[majidhassan]

@gowridev I'm not sure what exactly you meant with "send mail invites". But I send regular emails using the Mailgun adapter. Here's some sample code:

var Mailgun = require("mailgun-js")({apiKey: mailgun_apikey, domain: mailgun_domain});

function mailgunSend(data, promise) {
    Mailgun.messages().send(data, function (error, body) {
        if (error) {
            promise.reject(error);
        }
        else {
            promise.resolve();
        }
    });
}

var promise = new Parse.Promise();

var data = {
    from: "Some Email <no-reply@somedomain.com>",
    to: "receiver@somedomain.com",
    subject: "Hello",
    html: "<html><body>Hello</body></html>"
};

mailgunSend(data, promise);

Env: parse-server-mailgun@2.1.12 parse-server@2.2.25-beta.1

[flovilmart]

There is a default form for password resets, as long as you provide a mail adapter, this will work