CMAM programme responsiveness

[JosephineCarenNdawula]

We have 4 indicators for CMAM programme responsiveness

1. cure rate
2. defaulter rate
3. non-response rate
4. death rate

We have one administrative process indicator RUTF consumption rate

[JosephineCarenNdawula]

@OxfordIHTM/neji

8 CMAM programme responsiveness

1. We are adding a new indicator

   • admission rate (new admissions/total screened)

2. We have the following questions:

   • What do the negative values of RUTF consumed mean?
   • Why is it that (Beginning of month + New admissions- total discharge) is not equal to beginning of month for the subsequent month?
   • What is the total population per state?

[ernestguevarra]

@OxfordIHTM/neji #8 CMAM programme responsiveness

1. We are adding a new indicator

• admission rate (new admissions/total screened)

2. We have the following questions:

• What do the negative values of RUTF consumed mean?
• Why is it that (Beginning of month + New admissions- total discharge) is not equal to beginning of month for the subsequent month?
• What is the total population per state?

• new indicator looks good!
• well spotted! in general this means that the catchment clinic within the locality reporting negative RUTF ran out of RUTF. This means that some or most (depending on the deficit) are admitted cases but most likely not receiving RUTF. So form an operations point of view, this is an indication of either or all of these things:
  • high risk of admitted cases not getting effective treatment hence either higher non-response and/or higher defaulting and/or higher non-response rates at the locality level.
• well spotted. this should indicate to you that record keeping in general was not good especially with regard to performing the calculations for carry over cases from previous month. when we investigated this at that time, admissions and discharges were generally correct but the excel sheet that the programme manager was using didn't have updated formula for calculating carry over cases. So, here, I wouldn't rely on the beginning month data but also I will not spend a lot of time trying to re-create that. From a responsiveness point of view. admissions and discharges over time are the most sensitive indicator to test whether the programme is responsive to the need.
• For the population values for locality, state and national, I will provide this on Monday during class.

[JosephineCarenNdawula]

Thanks Ernest.

On Tue, 6 Feb 2024, 12:51 Ernest Guevarra, @.***> wrote:

@OxfordIHTM/neji https://github.com/orgs/OxfordIHTM/teams/neji #8 https://github.com/OxfordIHTM/ihtm-hackathon-2024/issues/8 CMAM programme responsiveness

1. We are adding a new indicator

   • admission rate (new admissions/total screened)

2. We have the following questions:

   • What do the negative values of RUTF consumed mean?

   • Why is it that (Beginning of month + New admissions- total discharge) is not equal to beginning of month for the subsequent month?

   • What is the total population per state?

   • new indicator looks good!

   • well spotted! in general this means that the catchment clinic within the locality reporting negative RUTF ran out of RUTF. This means that some or most (depending on the deficit) are admitted cases but most likely not receiving RUTF. So form an operations point of view, this is an indication of either or all of these things:

     • high risk of admitted cases not getting effective treatment hence either higher non-response and/or higher defaulting and/or higher non-response rates at the locality level.

   • well spotted. this should indicate to you that record keeping in general was not good especially with regard to performing the calculations for carry over cases from previous month. when we investigated this at that time, admissions and discharges were generally correct but the excel sheet that the programme manager was using didn't have updated formula for calculating carry over cases. So, here, I wouldn't rely on the beginning month data but also I will not spend a lot of time trying to re-create that. From a responsiveness point of view. admissions and discharges over time are the most sensitive indicator to test whether the programme is responsive to the need.

— Reply to this email directly, view it on GitHub https://github.com/OxfordIHTM/ihtm-hackathon-2024/issues/8#issuecomment-1929462105, or unsubscribe https://github.com/notifications/unsubscribe-auth/BDB7BMHHG5OHRLIRL2FDI2TYSIRNDAVCNFSM6AAAAABCZP2DZWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRZGQ3DEMJQGU . You are receiving this because you commented.Message ID: @.***>

[JosephineCarenNdawula]

@OxfordIHTM/neji

8 CMAM programme responsiveness

1. We'd like clarification concerning the site status. Are we right in assuming that in 2017, when it was = TRUE was when program implementation was active?

2. The computation of RUTF varies in terms of decimal places, from 1 to 2 to 3. So for example is 1.0 =10 and 11.00= 1,100 and 11.000= 11000?

3. What does it mean for screening to become N/A for all states in the year 2018?

[ernestguevarra]

@OxfordIHTM/neji #8 CMAM programme responsiveness

1. We'd like clarification concerning the site status. Are we right in assuming that in 2017, when it was = TRUE was when program implementation was active?
2. The computation of RUTF varies in terms of decimal places, from 1 to 2 to 3. So for example is 1.0 =10 and 11.00= 1,100 and 11.000= 11000?
3. What does it mean for screening to become N/A for all states in the year 2018?

I don't understand what you are asking about in number 1.

As for RUTF, as already mentioned earlier, data in the RUTF consumed column is not reliable. I will not spend my time working with that data. I like your razor sharp focus on this variable but I will save you the trouble by saying don't use this variable.

NA in any fields would mean missing data rather than nothing happening or 0.

[ernestguevarra]

These are the typical indicators of programme responsiveness in general and which apply to CMAM programming as well.

1. Does the programme address need? in this case, when the need is high (prevalence), does the admission to the programme react to that need (i.e., increase in admissions as well). This is usually best shown with a trend/time series plot of admissions over time and then overlaying this plot with a seasonal calendar. The seasonal calendar should indicate times of the year when acute malnutrition/wasting is expected to be high (lean periods/lean season which are usually a few weeks planting/sowing season when grains and outputs from previous harvest are already dwindling or low; periods of high incidence of malaria and/or diarrhoea and/or ARI or other endemic childhood illnesses).
2. Does the programme retain those that are admitted? For a programme to be responsive, it should be able to retain those afflicted by the condition until they are cured. If there is high defaulting, particularly at periods of time when other socio-economic factors can play a part in compliance to the treatment, then the programme is not being responsive to needs in a more general way beyond just treatment.
3. Is the programme performing well - a responsive programme is able to treat all or nearly all of its target population. And it needs to do this consistently over time. So, the picture of performance that is responsive is that of consistently high cure rates (should be at least greater than 75% and default rate default rate of less than 15%, death rate less than 10%

[ernestguevarra]

You can get the population per locality from the map data called sudan2. The variable for population is called Total_Pop

You can get the population per state from the map data called sudan1. The variable for population is called POP_2018

[ernestguevarra]

Regarding seasonal calendar, doing a quick Google search with the search term - "seasonal calendar for Sudan", you can get the following:

https://reliefweb.int/report/sudan/sudan-seasonal-hazards-calendar-typical-year

https://fews.net/east-africa/sudan/seasonal-calendar/december-2013

[ernestguevarra]

@OxfordIHTM/neji, here is a tutorial/guide that you may find relevant to your topic. These might give you some ideas to simplify your workflow in case you need it.

https://github.com/orgs/OxfordIHTM/discussions/32

[ernestguevarra]

team @OxfordIHTM/neji, you can now view the final/current version of the final report based on your inputs/contribution (alongside that of team hinata).

https://oxford-ihtm.io/ihtm-hackathon-2024

The report will automatically update when the contributions of the other teams are made into pull requests and get approved to merge into main.

[ernestguevarra]

On a related note, in case you are interested, here is a link to a dashboard we produced for the Sudan Ministry of Health in relation to the same data that you worked on as a team:

https://katilingban.shinyapps.io/sudancmamdashboard/

This is the typical approach to handling the kind of data you worked on from the perspective of programme responsiveness.

The code that runs the dashboard can be found here - https://github.com/katilingban/sudanCMAM

[ernestguevarra]

Team @OxfordIHTM/neji, I made some minor edits to your plots in the report just to align them with the style of the other plots from the other groups. I've also made your table outputs in the report a little bit nicer. See if you notice the subtle changes.

https://oxford-ihtm.io/ihtm-hackathon-2024

Some questions, comments, and suggestions for the future:

1. I am curious what you mean by your statement in the report - "CMAM likely did not produce a substantial effect on the indicators analysed 💀"

My reason for asking is that I am a bit confused with what it means. The data you were working on was specifically for CMAM programme so the indicators for cure rate, etc. is directly produced by the programme. So, I wonder whether by indicators you meant the indicators in the survey? Otherwise, I am curious what you mean by this.

2. With regard to responsiveness in relation to the programme performance indicators, the analysis over time is important here and the disaggregation of the results by State is also important. A responsive CMAM programme is able to keep cure rates high (recognised standard for cure rate is above 75%) over time and over space and it will only be able to achieve this if it keeps defaulting low, deaths low, and non-response rate low. This is because these metrics added together is equal to 100% everytime (common denominator of total discharges). So, over time, what you want to see in a responsive programme is that cure is always above 75%. And then over space/location, you want this cure rate above 75% in all locations. So, looking at national level, the programme looks fantastic because cure rates are very high (well above 75%) and it is consistent yearly and even monthly over the years. However, when you disaggregate the performance per state over time, there is an interesting revelation. This level of performance at national level is not consistent in all states and not consistent over time. These are hidden by the national figures and this is what provides insight as to where MoH needs to focus on. And once you add the other metrics there, you see that in some of the poor performing areas, defaulting is the issue while in some isolated states, death rate is the issue. These are critical problems of the programme that are hidden by the national figures.

I think your multiple lines plot for each state plot carries some of this information but it is very busy plot and hard to see which states are the ones underperforming. Also, you don't see in the plot what the possible reasons are for the low performance in specific states.

3. For the layout of the plots, I converted your y axis by just multiplying by 100 during the plotting itself (well done for following what I said earlier of keeping values as they are and then converting in the last minute when you need to) and just used the % symbol instead.

4. Sometimes, axis labels are not needed because the tick values are already self-explanatory. So, I removed the year label.

5. I put your legend on top. This gives the plot full width on the graphics device which is always nice and I think that for a reader, it makes more sense for legends to be on top because as you read from top to bottom, you get to see the legend first and then you look at the plot.