Notify submitter & drafter via contact email when enrollment status changes

[brainwane]

We should send a provider an email notification when their enrollment status (pending, modified, approved, or rejected) changes.

(Once this is implemented, it should be documented in psm-app/docs/userhelp/source/service-admin-help.rst .)

[brainwane]

If a service agent or service admin submitted the enrollment, we should notify them as well. (If a provider started a draft and then a service agent finished and submitted it, we should notify both of them. If three or more users were involved in modifying or submitting the enrollment, we should notify all of them.)

[brainwane]

@cecilia-donnelly If I understand correctly this is not in our deliverables for the end of December 2017; if I'm wrong please add this to the milestone.

[kfogel]

Note that this will need to include a mechanism for configuring the outbound SMTP gateway (e.g., a state would probably want the PSM to use whatever SMTP server the state's other email-emitting MMIS modules use). If the PSM isn't going to set up full SPF/DKIM/DMARC itself -- which it probably can't -- then it at least needs to empower system administrators to tell it how to use whatever they've already got set up. Sending email reliably in the age of spammers is no fun :-).

[jasonaowen]

@kfogel working outgoing email is already a requirement, and is configured through the application server -- which, happily, means we do not need any custom code at all!

[cecilia-donnelly]

@jasonaowen awesome! Thanks for letting us know.

[jasonaowen]

The easiest, first step is to send out emails when an enrollment application is either approved or rejected. We will need email templates for each. See, for example, the registration success email template.

If the contact email address is present, we would send to that; if it is not (if "same as above" is checked), we would send to the email address entered under the "personal info" section of the enrollment. One thing that is not clear to me: would we want to send an email to both the contact email address and the personal info email address, if they differ? Would we want to send an email to the user's email address, if it differs from both?

I'd also like some clarity on the other two triggers, an enrollment state becoming "pending" and an enrollment being modified.

An enrollment enters the "pending" state when the user finishes the enrollment process; do we want an email for that (basically, acknowledging the enrollment was submitted)? If so, we'll also need a template for that.

Service agents are no longer able to edit pending enrollments; service admins are the only ones who can. When an admin modifies and resubmits an enrollment, should they be copied on all future emails (pending, approve/reject)? It seems likely to me that would result in a lot of noise for them, as they might be the one clicking approve/reject. If so, we only have their user account's email address; is that acceptable?

We do not yet have enrollment sharing (as discussed in #10), but once we do coauthors will need to be informed, too, as @brainwane mentions. Again, the only information we will have there is the user's email.

[cecilia-donnelly]

• I like the idea of sending an email to the user's email address, as opposed to the personal info or contact address. Note that as part of #34 we might be sending them some kind of email where they'll need to click a link to create an account, so we'll know that email address is working (as opposed to the contact or personal info addresses, which we will know have the right form but won't have tested).
• Pending: yes, let's send an email acknowledging that the enrollment was submitted.
• That question about service admins is a tough one. Maybe we can offer them the option? If they click "Edit," then "Save" or "Submit," they could get a popup asking whether they want to receive email updates about this application. Extra points if we track how often they say yes or no, for future decisions like this one.

[jasonaowen]

Ah, good catch that the the email submitted with the enrollment is not verified!

So, maybe it would make sense to send an email to both of those (personal info email address, and contact info email address) to confirm, just like we want to do with #34 for user email addresses.

Otherwise, it seems weird to collect these additional email addresses but not use them - as a user, I think I might be surprised and a little frustrated if I submitted an enrollment and all the emails around it went to me instead of the contact person I entered.

If that makes sense, we should spin off a new issue to track that. It'd probably be more ideal to do that first and this second, but we could absolutely do an incremental version of this and then augment it once we have email validation working.

[brainwane]

My take:

• Send email to the user account's email address and, if different, the personal info email address as well, upon acceptance/rejection of the enrollment.

• If the user submitted a contact info email address that is different from the personal info email address, then send notification to the contact info address instead of the personal email address upon acceptance/rejection of the enrollment.

[cecilia-donnelly]

I talked to @chj124 earlier and he confirmed the following:

• Currently, applicants receive an acknowledgment that their application has been received and a notification at the end of the process: notifying that they were accepted or denied, with reason for denial if applicable
  • If they're accepted, they have to sign and return something saying that they agree to provide services and giving them information about where to find rates, where to submit claims
• We may need to contact a group or facility that the provider is affiliated with, to confirm that the affiliation is correct
• Admins should see information about sent communications in their dashboard. This goes along with something we've had a little conversation about: new applications need to be assigned to an admin. Currently the PSM shows all applications to each admin user, which isn't a useful case management workflow. In the future, when processing a case: someone will be assigned to work the application as a "case manager" on the state side. They'd see activity about their cases on their dashboard (e.g. notifications).

[chj124]

Just to add....

When using workflows, or work assignments, one admin should be able to see all applications and who they are assigned to. Other admins should only be able to see the applications assigned to them.

There should be an option to assign a group of applications to an admin.

When the application is filed there should be a notification email for the provider, and the agent (if applicable), and for the facility if multiple providers practice there.

The notices could be different. Notice to facility would be that provider X applied to provide services from their facility.

[jasonaowen]

After speaking with @cecilia-donnelly today, we decided that we would like to retain sent emails, at least for status change notifications. We can probably start without that and add it incrementally, since we don't have any deployments and so don't have to worry about losing sent emails in between.

[jasonaowen]

A few thoughts on implementation:

• We have code in RegistrationServiceBean that sends email notifications based on templates; I was imagining extracting that code to a more generic/reusable class and using it for this feature, as well.
• If we want all notifications to be recorded, we should fix the password reset / new account handling (see #34), as currently we should probably not record those notifications. I haven't thought very hard about the database setup for that, but it doesn't feel particularly risky to me.
• The status notifications would probably be sent in AcceptedHandler, RejectedHandler, and EnrollmentPageFlowController (for submission and resubmission).
• The notification templates feel pretty bikeshed-esque, so it might be worthwhile to work on those first, or at least start a WIP PR with a first draft to start gathering that kind of feedback early rather than blocking an otherwise-complete PR on wordsmithing.

[dan9000]

I think this is an accurate summarization of the requirements here:

• Send email notifications for enrollment status changes (pending, modified, approved, rejected)
• Send email notifications to individual provider
• Send email notifications to organizational provider if they have multiple individual providers
• Send email notifications to service admins and/or service agents if they request it when submitting/modifying an enrollment
• Use different email templates for each type of recipient (individual provider, organizational provider, service agent/admin) and type of status change (pending, modified, approved, rejected)
• Send notifications to the contact email address or as a fallback the personal email
• Send notifications to the account address
• Display information about sent communications in admin dashboard
• Store sent messages for auditing
• Document feature in psm-app/docs/userhelp/source/service-admin-help.rst

[cecilia-donnelly]

@dan9000 Yes, thank you! The only specific change I would make to these is around "facilities." The PSM doesn't currently have any concept of a "facility." Vermont does, which is where @chj124's comments are coming from. I would instead think of it as an organizational provider, if an individual provider is linked to that group.

I notice you have notifications going to both the contact email/personal email and the account email -- I think @jasonaowen and I left a dangling item there.

One more thing: we want to be able to audit sent messages, so there'll need to be some way for us to store (and eventually, though not in this issue, display) sent messages.

[dan9000]

WIP PR (https://github.com/SolutionGuidance/psm/pull/678) has the initial notification service and templates in place but it's not being called from the accepted/rejected handlers and I'm not sure why. Also work still needs to be done to gather which emails the notification should be sent to. And the admin dashboard info about sent notifications and persistence of sent emails needs to be done.

[cecilia-donnelly]

Still to do:

• [ ] Call notification service from accepted/rejected handlers (that is, on change of application status)
• [ ] Get the account email and the "contact" email (if different), and send the notification to both of those.

The admin dashboard info that @dan9000 mentioned is a next step, beyond the scope of this issue. We do, however, need to:

• [ ] Store sent messages for auditing

[slifty]

@cecilia-donnelly I'll want to better understand how we want to store sent messages (this will be best discussed over the phone)

Other open question is the actual mechanism for communicating emails to the service. I'll put some thought to this as well after we catch up, and share notes here.

[slifty]

Wanted to post an update here -- I've merged in @dan9000's code, set up my environment to be able to test email sending, and got caught up with the changes and general shape of the code related to this ticket.

As @dan9000 mentioned, the notification integration with the RejectHandler and AcceptHandler is in place but for some reason isn't being called. It is possible those handlers may not be getting invoked at all... I need to dig a little more, but will avoid spinning wheels too much.

[slifty]

I know @jasonaowen has worked on these files so once he is back I imagine things will be more clear.

Rejecting an application triggers the reject method in EnrollmentController

Eventually that winds up at BusinessProcessServiceBean:completereview()

It doesn't appear that the AcceptedHandler or RejectedHandler get triggered. Is it possible they aren't actually connected to the approval process?

[jasonaowen]

For those following along at home, @slifty and I were able to pair on this yesterday and untangle things a bit!

The AcceptedHandler does trigger on a successful approval, but the enrollment application must first pass the screening rules (see also #572), and failures there are not well-presented to the user.

The RejectedHandler turns out not to be triggered when an admin rejects an enrollment application in the review UI, because instead of calling reject, it instead calls rejectTicket, which bypasses the handler.

@slifty will...

1. reduce the scope of the first pull request to only send an email on approved enrollments (since that part is working as expected)
2. open an issue for RejectedHandler not getting invoked
3. fix that issue (either by verifying that the RejectedHandler code is not used and deleting it, or by changing the admin UI to use the jBPM workflow instead of bypassing it - and delete the unused reject or rejectTicket method + endpoint to avoid future confusion!)
4. based on how that issue is resolved, add email notifications for rejections

[cecilia-donnelly]

This can be closed once #737 and #750 are merged.

[cecilia-donnelly]

Closing now that the associated PRs are merged. Note some related work is happening in #770 (and PR #771).