search

OpenFn / Miracle-Feet

Jobs for Miracle Feet's CommCare to Salesforce integration
1 stars 1 forks source link

readme

MiracleFeet OpenFn Integrations

Jobs for Miracle Feet's CommCare to Salesforce integration, using the Open Function platform. See the MiracleFeet India repository for more on the India CAST app integration.

N.B., all changes to the master branch will be automatically deployed to Open Function.

1. CommCare-Salesforce Integration for Automated Data Sync

MiracleFeet has configured CommCare to deliver the "CAST" application to support partner treatment providers and service delivery across 26 countries.

MiracleFeet has implemented OpenFn to automate one-way data integration between CAST (CommCare) and the MF Global System (Salesforce) for real-time monitoring and reporting across CommCare form submissions and case updates.

Technical Overview

  1. Users submit new forms or modify cases in CommCare
  2. A webhook configured in the CommCareHQ project automatically forwards this data to MiracleFeet's OpenFn project Inbox as Messages
  3. OpenFn jobs are triggered to process & upload this CommCare data to Salesforce

diagram

Job Notes

  1. These jobs receive case, location, and form data from CommCare and perform upserts on related business objects in Salesforce in real time.
  2. We're considering allowing a single job to load data to a variety of different SF instances—making Miracle Feet's OpenFn project effectively a router for various CommCare-Salesforce implementations.
  3. Mappings that use humanProper(...) (e.g.,field('gciclubfoot__Brace_Problems__c', humanProper(state.data.properties.brace_problems)),) will work for new language-common version.

Support

MiracleFeet global administrators are responsible for integration monitoring & reprocessing of any failed runs. For additional OpenFn support, users may contact support@openfn.org.

2. Salesforce-Infobip Integration for Automated SMS Alerts

Originally the SMS scheduling solution relied on CommCare form updates received in the OpenFn inbox to schedule messages. It has been re-designed to achieve the same functionality based on patient details fetched from Salesforce. See here for the data flow diagrams.

Watch the solution overview video for the original CommCare-Infobip SMS integration for patient visit reminders and educational SMSs.

And see here for the original project concept.

Technical Overview

Prerequisites Patients' Salesforce information necessary for SMS scheduling is now being synced from CommCare via the existing Add new patient, Update Patient, Add new visit and Update visit jobs when an update happens in CommCare. These update the new Salesforce fields: SMS Treatment, SMS Original Treatment, SMS Treatment Start Date, and the Pronoun fields. We also make use of the Patient First Name, Guardian First Name, Brace Problems Type, Last and Next Visit dates, and the Send SMS and SMS Opt In fields to create messages and verify consent.

Inbox Security is set up for the projec with Basic Authentication.

How it works...

  1. An OpenFn job [SMS-Flow-1] Fetch contacts from SF runs daily to query contacts from Salesforce and fetch their treatment and appointment information.
  2. A second OpenFn job [SMS-Flow-2] Schedule SMSs will be triggered to schedule relevant SMS alerts in the Infobip Portal

We have separate jobs to schedule "Bracing Night" messages and to handle changes in clinic status.

Data flow diagrams

A high-level overview of the SMS jobs can be found here.

The following diagrams describe the SMS scheduling/cancelation logic for the different flows.

  1. Alerts: scheduling logic for educational messages in the [SMS-Flow-1] Fetch contacts from SF flow.
  2. Reminders: Appointment reminder scheduling logic in the same SMS Flow.
  3. Opt-out: SMS cancelation logic when a patient opts out of some or all of the SMS-s, handled in the same SMS flow.
  4. Deletion: Deleting SMSs for previous treatment in case the treatment has changed or stopped in the same flow.
  5. Bracing night: Logic for scheduling "Bracing night" messages in the "Bracing Night" flow.
  6. Deactivated clinic: Logic for canceling all SMSs in case a clinic is deactivated, handled in the deactivation flow.
  7. Suspended clinic: Canceling appointment reminders for suspended clinic in the deactivation flow.

Alert IDs in the diagrams (e.g. Alert 8) refer to # Alert in the Master Mapping Table. Admins can check all information on the alert (scheduling time, condition for scheduling, SMS text and translations) in the sheet by Alert ID.

Notes on Scheduling Logic

Overview

  1. SMS-s are scheduled in Infobip using constructed bulkId-s. bulkId are constructed from the alert ID (e.g. casting_intro-1 and the CommCare case_id: casting_intro-1-XXX. They are used to uniquely identify SMS-s in Infobip.
  2. SMS-s are scheduled using upsert logic in Infobip: when scheduling or deleting an SMS, OpenFn first checks if an SMS with the corresponding bulkId has already been scheduled for the future or already sent. If this is the case, no changes are made. If the SMS hasn't been scheduled or sent, it gets scheduled.
  3. In case of deletion, OpenFn checks if an SMS with the corresponding bulkId has been scheduled for the future or already sent. If it is scheduled, it gets canceled.
  4. Educational SMS alerts are scheduled following the logic outlined here.
  5. Visit reminders and missed appointment reminders are scheduled to be sent according to the next visit date indicated in the form. We don't send missed appointments reminders more than 1 week after the appointment date.
  6. Detailed scheduling conditions and SMS content can be found in the master mapping table. The current Salesforce-based solution uses the Salesforce mapping table. (The previous, CommCare-based job uses a different mapping table.
  7. Both scheduleSMSv2 and bracing-night jobs use the "[CURRENT] Master Mapping Table Salesforce" mapping. ScheduleSMSV2 schedules alert 8 and 9 of the bracing-night messages and all the other (non-bracing-night) alerts. The bracing-night job only schedules alerts 9B to 9P.
  8. SMS scheduling times are adjusted for local time zones.
  9. SMS-s are only scheduled to be sent between 8am-8pm local time.
  10. We only schedule SMS if Guardian_1_Phone_Landline__c == false on the Patient's Salesforce profile.
  11. Errors returned by the Infobip API for incorrect phone numbers are uploaded regularly to Contacts' SMS Error field. We only schedule SMS to patients where this is empty, and we delete scheduled alerts for Contacts that have any SMS Errors marked.
  12. Due to limitations of the Infobip API, we don't schedule any messages for more than 6 months ahead.

Logic for setting missed appointment reminders SMS reminders are scheduled to be sent if a patient misses an appointment they are scheduled for. This is set when a new next_visit_date is added. If there were alerts set for previous visits, those are canceled in Infobip.

Used values: last_visit_date/original_next_visit_date for previous scheduled visits, next visit date for next visit

Steps

  1. Check if there’s a last visit date. If so, check if these bulkIDs have been scheduled visitAfter1-{form.case.@case_id}-{last_visit_date} visitAfter2-{form.case.@case_id}-{last_visit_date} If yes, delete them.

  2. Schedule the reminders with the current next visit date visitAfter1-{form.case.@case_id}-{next_visit_date} visitAfter2-{form.case.@case_id}-{next_visit_date}

Scheduling a message on the Infobip API

curl --location --request POST 'https://apidomain.infobip.com/sms/1/text/advanced' \
--header 'Authorization: App ${api_key}' \
--data-raw '{"bulkId": "bracing_day-6-345678", "messages":[{"from": "+111","destinations": [{"to": "+111"}],"text": "A long time ago, in a galaxy far, far away...","sendAt": "2021-12-24T18:20:00.000"}]}'

Opt-in, opt-out

Treatment changes, stopping treatment

When a patient is marked as stopped, suspended or completed treatment, they will no receive educational messages and visit reminders. If they move from one treatment to a different one, educational messages for the original treatment are stopped and SMS campaign for the new treatment is scheduled.

Scheduling "Bracing Night" SMSs

Since September 2021 it's not possible to schedule messages on the Infobip API for more than 6 month ahead.

The Bracing Night SMS campaign runs for more than 3.5 years. We use the following process to schedule these messages:

We run the [Bracing-night 1] Fetch bracing night contacts from SF job to check on SF if there's any contact that has Bracing Night set as SMS Treatment.

The [Bracing-night 2] Schedule bracing night SMSs will then automatically launch to schedule any necessary SMSs for the next 3 months, based on the SMS_Treatment_Start_Date__c field.

Clinic status and SMS scheduling

Clinic status is set in Salesforce, in the Status field.

All "Actively supported" clinics are eligible for SMS (although not all have it turned on)

We run the [Deactivation-1] Fetch Inactive Clinics from SF job daily to check for clinics that are inactive and disable SMS-s for associated patients guided by the below logic:

Ongoing Management

To enroll a new country to SMS

  1. Turn on SMS for the CommCare user (send_sms field)
  2. Add country to Fetch contacts and Fetch bracing night contacts jobs, e.g.: Account.Country__c in ('Nigeria', 'Uganda')

To update SMS scheduling times, SMS templates, SMS language codes, or add new translations:

  1. Make a copy of the Master Mapping Table.
  2. Make your edits in the new sheet (see an example here).
  3. Notify OpenFn support at support@openfn.org to implement the changes.

Migration Checklist

Access here.

OpenFn Process for converting Mapping sheet to JSON object

  1. Copy the whole content of the mapping table
  2. Paste on this converter at the left pane https://csvjson.com/csv2json
  3. Get the generated json
  4. Paste it on content-input.json
  5. Execute node mappings/converter.js
  6. Copy the resulting content from content-output-minified.js https://github.com/OpenFn/Miracle-Feet/blob/master/mappings/content-output-minified.js
  7. Paste it in the job in place of mapping variable at line 3. If there is code formatting enabled, it might expand the contents. In that case, undo that last change to make sure content is on one line. There are 2 jobs to update:
  8. The entire mapping sheet should be added to: https://github.com/OpenFn/Miracle-Feet/blob/master/schedule_smsV2.js
  9. Only the "Bracing Night" messages to be added to: https://github.com/OpenFn/Miracle-Feet/blob/master/2.scheduleContactSMS.js

To add/remove alerts or reminders, update CommCare field names, change opt-in/opt-out conditions, add new CommCare forms that trigger SMS-s, or any other change, contact support@openfn.org.

SMS scheduling only happens for patients whose Clinic Status in Salesforce is either in "Actively Supported" or "Temporarily Suspended". In order to add or remove clinic from the scheduling flow, change the Clinic Status accordingly. To opt a clinic into SMS alerts, this Status should be updated in Salesforce.

Training

Administrator Training Slides
Change Management Training Slides

SMS Implementation Next Steps

Salesforce-based flow roll-out

Original flow roll-out