Impact monitoring documentation (brainstorm to be broken out into separate issues)

[MaxDiz]

Impact monitoring is a core feature set that we offer CHT users. It is currently not well documented, and needs additional information on:

• Dashboard user overview
• Dashboard setup tutorial
• Impact logic for priority use cases
• Database schema

Existing documentation resources to reference:

• Data flow
• Data interpretation best practices

This is a large body of work that we may want to split into pieces, but it would be helpful to first look at it comprehensively. We may also want to take into consideration upcoming product improvements related to transitioning away from Klipfolio.

[helizabetholsen]

Thanks @MaxDiz. The current plan right now is to build out our Impact Monitoring documentation in alignment with our metrics consolidation work - and data use engagement with our partners (both Medic hosted and self-hosted).

I think the flow of this documentation should include the following sections:

What is Impact Monitoring? (high level overview)

• Our metrics
• Our approach to responsible data use
• Our data flow
• Our dashboards
• Engaging with the CHT community

Technical Details (overview)

• Impact monitoring set-up overview
• Reference impact monitoring set-up @yrimal and Helen thinking about this now, ideally will wait to develop and share until the metrics consolidation process is complete.
• Dashboards [move from Klipfolio to Superset, creating dashboards specific to our priority use cases and priority metrics, as well as quarterly and goal reporting @MaxDiz we should make sure the product approach is aligned with the Impact Monitoring plan for metrics consolidation]
• Ideally we will also include our data dictionary and other resources but this might be in a separate section

Data Interpretation (overview)

• Medic's data driven culture
• Guide to interpreting our impact metrics
• Phases of interpretation mapped to deployment phases
• Key questions and considerations

The graphic above is a more up to date visual of our current data flows. I think we should consider including this in the "Data Flow" section linked above.

The "data interpretation best practices" deck is not intended for external audiences at this time but I think we could build that out - or add those content into another resource. The following resources can inform our documentation for Impact Monitoring:

• Introduction to Impact Monitoring Needs to be updated
• Impact Monitoring Overview
• List of CHT Impact Metrics
• Medic Mobile Privacy & Data Protection Policy
• An Overview of Medic's Data Driven Culture

[helizabetholsen]

We also have specific documentation for our impact monitoring priority use cases in this folder: https://drive.google.com/drive/u/1/folders/1MO70BWGoUqqLkWJwMjaRxb8XDZ1uw_jV

Here are a couple of examples:

• ANC Impact Logic
• Interpreting ANC Metrics needs to be revised

[helizabetholsen]

Impact Metrics guide included in docs site #100

[helizabetholsen]

Hi all. I also think as we consider Impact Monitoring & Analytics Documentation, there are some broader questions to consider:

• Discussion on where this is included in our documentation site - do we consider impact monitoring/metrics/analytics a feature or do we want this to be a section with multiple guides?

• Distinction between partner-focused analytics and our impact monitoring work. Also, I think this could be a good catalyst for discussing how targets are set and if those should align with our impact metrics more closely, include R&L team review?

• Do we think the documentation we include also ties into our capacity building and longer term vision for building out our monitoring to be a legit public good