AFS initiating survey dicts changes

[aspina7]

just starting out on changing the survey dictionaries from Dharma to Kobo. We are still waiting on the dictionary for nutrition.

[aspina7]

This PR is necessary in order for the survey template updates in Sitrep to function.

[zkamvar]

I think I now understand why the tests don't pass:

The Nutrition dictionary has not yet been migrated to the Kobo version. If I exclude Nutrition from the tests, then the tests pass.

@aspina7, can you add the updated dictionary and the associated options if you have it?

[aspina7]

I think I now understand why the tests don't pass:

The Nutrition dictionary has not yet been migrated to the Kobo version. If I exclude Nutrition from the tests, then the tests pass.

@aspina7, can you add the updated dictionary and the associated options if you have it?

Yep nutrition survey not available yet - still waiting on it from the mhealth team. So we just need to turn of nutrition for now

[zkamvar]

I've added CI for GitHub so that we don't have to rely on the travesty that is Travis-CI and Appveyor

[zkamvar]

This PR should not be merged yet because the Nutrition dictionary is not completed and will break things upstream.

[aspina7]

This PR should not be merged yet because the Nutrition dictionary is not completed and will break things upstream.

@pbkeating

[pbkeating]

Feck.. will follow-up… these things are a right pain in the ass

From: Alex Spina notifications@github.com Sent: 01 November 2020 07:33 To: R4EPI/epidict epidict@noreply.github.com Cc: Patrick Keating Patrick.Keating@london.msf.org; Mention mention@noreply.github.com Subject: Re: [R4EPI/epidict] AFS initiating survey dicts changes (#13)

CAUTION: This email originated from outside MSF. Do not click links, open attachments or provide information unless you recognise the sender and know that the content is safe..

---

This PR should not be merged yet because the Nutrition dictionary is not completed and will break things upstream.

@pbkeatinghttps://github.com/pbkeating

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/R4EPI/epidict/pull/13#issuecomment-720047913, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2NEQ2QTQ64NPF2EVIES3DSNUFKNANCNFSM4RTR5S2Q.

[aspina7]

@pbkeating in the mortality dictionary:

• [ ] both the nutrition and the malaria modules have muac, weight and height vars (i.e. duplicated). Necessary and intentional?
• [x] in some of the restritction rules for mortality (i.e. the "relevant" column) - you refered to age_months_calc, which does not exist anymore. Updated in the version on github.
• [ ] for measles_vaccination is for age_months <61 (i.e. five years), but months only for those under 2 years, in the code I have used age_years < 5, but should the dictionary be updated?
• [ ] for reason_no_care in the malaria children module - the rule refers to health_carevariable but think it should be care_fever
• [ ] malaria_treatment is a duplicate of malaria_treatment_preg - presumably can just keep the first which can then be used for the add-on module too?
• [x] under hsb module for death section, the rules for source_written_hsb, source_date_death_hsb and source_written_death_hsb all refer to source_death but this no longer exists and should be source_death_hsb. Updated on github version.

[pbkeating]

@pbkeating in the mortality dictionary:

• [ ] both the nutrition and the malaria modules have muac, weight and height vars (i.e. duplicated). Necessary and intentional?

This was intentional as MUAC is not a standard part of the survey but maybe the variable names don't have to change since it's unlikely you would use both the nutrition and malaria modules in the same survey

• [x] in some of the restritction rules for mortality (i.e. the "relevant" column) - you refered to age_months_calc, which does not exist anymore. Updated in the version on github.

Removed this throughout. Thanks for catch

• [ ] for measles_vaccination is for age_months <61 (i.e. five years), but months only for those under 2 years, in the code I have used age_years < 5, but should the dictionary be updated?

Thanks again, replaced this with age_years <5 in the excel also

• [ ] for reason_no_care in the malaria children module - the rule refers to health_carevariable but think it should be care_fever

Yes, you are right - this changed after I updated all the variable names to try to standardise them somewhat and forgot to go back. I've updated it in the excel also

• [ ] malaria_treatment is a duplicate of malaria_treatment_preg - presumably can just keep the first which can then be used for the add-on module too?

These two questions have different answers and questions. Although malaria_treatment_infant should be the same as malaria_treatement_preg

• [x] under hsb module for death section, the rules for source_written_hsb, source_date_death_hsb and source_written_death_hsb all refer to source_death but this no longer exists and should be source_death_hsb. Updated on github version.

Yes, good catch. Have updated the excel also

[aspina7]

@pbkeating in the mortality dictionary:

• [x] both the nutrition and the malaria modules have muac, weight and height vars (i.e. duplicated). Necessary and intentional?

  This was intentional as MUAC is not a standard part of the survey but maybe the variable names don't have to change since it's unlikely you would use both the nutrition and malaria modules in the same survey

Will leave as is

• [x] in some of the restritction rules for mortality (i.e. the "relevant" column) - you refered to age_months_calc, which does not exist anymore. Updated in the version on github.

  Removed this throughout. Thanks for catch

• [x] for measles_vaccination is for age_months <61 (i.e. five years), but months only for those under 2 years, in the code I have used age_years < 5, but should the dictionary be updated?

  Thanks again, replaced this with age_years <5 in the excel also

updated github excel to read ${age_years}<5

• [x] for reason_no_care in the malaria children module - the rule refers to health_carevariable but think it should be care_fever

  Yes, you are right - this changed after I updated all the variable names to try to standardise them somewhat and forgot to go back. I've updated it in the excel also

updated github excel too

• [x] malaria_treatment is a duplicate of malaria_treatment_preg - presumably can just keep the first which can then be used for the add-on module too?

  These two questions have different answers and questions. Although malaria_treatment_infant should be the same as malaria_treatement_preg

okay will leave - but just to note that for malaria_treatment there is a restriction to only pregnant ladies.

• [x] under hsb module for death section, the rules for source_written_hsb, source_date_death_hsb and source_written_death_hsb all refer to source_death but this no longer exists and should be source_death_hsb. Updated on github version.

  Yes, good catch. Have updated the excel also

[aspina7]

@pbkeating still on mort dict:

• [x] treatment_delay - restriction should be ${care_illness_last} = "yes", updated on github version
• [x] reason_first_hf_selected restriction should be ${place_first_hf} = "other" or ${place_first_hf} = "hospital" or ${place_first_hf} = "primary_health_centre" or ${place_first_hf} = "primary_health_unit", updated on github version

variable naming consistency between vaccination and nutrition:

• [ ] number_children in nutrition and children_count in vaccination - should choose one and be consistent
• [x] num_qua**n**d_penta_woc should be num_quad_penta_woc - updated on github version (otherwise ruins iteration)
• [x] num_poliodrop_hf_woc and num_poliodrop_camp_woc should be changed to remove the num_ to be consistent with others - updated on github version (otherwise ruins iteration)

[aspina7]

@pbkeating @zkamvar @nsbatra

Could you have a look to make sure data isnt a mess / review code for these please?

To trial the functions:

• pull this project
• go to the survey_dicts branch
• Build package (under the buildtab next to git click install and restard - or just ctrl+shift+b)
• functions - demonstrations of how to use is on the readme , but note that varnames = "name" since swiching to kobo
  • msf_dict_survey() is for creating the survey dictionaries (based on the excel files which live in inst>extdata)
  • gen_datacreates fake data (wrapper of other functions below)
  • Other internal functions (note: all function scripts are in the R folder):
    • template_data_frame.Rtakes a dictionary and generates a dataframe with categorical vars pre-filled (called from gen_data)
    • gen_msf_data.R has hard-codes to make msf specific data more realistic
    • msf_dict.R is used for creating the outbreak dictionaries (based on the excel files which live in inst>extdata)
    • utils.R has various different helper functions used by the other larger functions (used in gen_msf_data)
    • utils-generators.R has various functions generating random samples for variables (used in gen_msf_data)

Also worth noting there is a discussion in this pull-request about how to add new organisations / dictionaries.