Operations Report - Add additional metrics for Spanish implementation

[robertsamm]

Placeholder. Specific requests will be added once defined. Amelia to schedule a meeting to discuss.

In preparation for the Spanish implementation release in June, we will need to add some metrics to the Operations Report to monitor. For example, will be able to use the Preferred Language variable to see the number of Spanish speakers who enroll and who sign the Spanish version of the consent.

[robertsamm]

@KELSEYDOWLING7 metrics requested for Spanish for the Operations Report below

Preferred Language Var: RcrtCS_ConNotLang_v1r0 (255077064) Spanish consent forms variable responses: https://github.com/episphere/connect/issues/915#issuecomment-2127271219 Consent version variable: 454205108

1. New table: count of active and passive verified participants by preferred language (english vs Spanish) stratified by site a. Title: ‘Preferred Language of Verified Participants by Recruit Type and Site’ b. Footnote: Preferred language is an editable variable, this figure represents a snapshot of participants preferred language at the time the report was ran. c. Please place after Figure 1.2
2. New table: Metric that shows the number of active and passive verified participants that signed the English or Spanish version of the consent form (does not change over time) a. Title: ‘Language Consented for Verified Participants by Recruit Type and Site’ b. Footnote: The consent language variable does not change over time. c. Please place this after the one above. We will use this to compare to the table above to see if people tend to stick to the preferred language they signed the consent, or if they change their preferred language.
3. 2 new time trends modified from existing one: timetrend that shows the number of verified overtime by English and Spanish preferred language. see Number of Verified Over Time metric-but keep this existing one in the report. 1 figure for Overall Sites and 1 figure for BSWH only. There would only be two lines on each graph for English and Spanish. a. Please title: ‘Number Verified Over Time by Preferred Language for All Sites/for BSWH’ b. See Figure 1.2 Number of Verified Over Time metric as example-but keep this existing one in the report c. Please place after the new figure above in item 2
4. 1 new table duplicated from existing: copy ‘how recruits heard about the study’ but limit it to just those with Spanish as preferred language a. Title ‘ How Recruits Heard About Study with Spanish as Preferred Language’ b. See table 1.17 how recruits heard about study c. No need to make a separate one for just English, will just be overall and Spanish
5. 1 new table duplicated from existing table: Show current status in workflow for all recruits with preferred language=Spanish a. Title: ‘Current Status in Workflow for All Recruits with Spanish as Preferred Language’ b. No need to make a separate one for just English, will just be overall and Spanish c. Please place after table 1.7 d. See table 1.7 current status in workflow
6. New table: Time From Sign-In to Submission of User Profile in Minutes stratified by English and Spanish and limited to people who submit UP (not needed by site) a. Title ‘Time from Sign-In to Submission of User Profile in Minutes by Preferred Language Among All Recruits’ b. Please place after table 2.3 ‘sign in to consent’
7. 4 new tables duplicated from existing: Completion of Baseline Activities Among Verified Participants for All Sites – have one overall, one English, and one Spanish + also do site-specific overall, English, Spanish for BSWH only a. All sites titles: ‘Completion of Baseline Activities Among Verified Participants for All Sites with English/Spanish as Preferred Language’ b. BSWH titles: Completion of Baseline Activities Among Verified Participants for BSWH with English/Spanish as Preferred Language’ c. No need to do other sites by English/Spanish
8. 1 New table: Baseline Data Completeness by Overall, English, Spanish using table 7.3s completion categories but replacing the QOL survey columns with Language (completion x language) a. Title: Baseline Data Completeness by Preferred Language’ b. Please place after table 4.4 c. See table 7.3 as an example

[KELSEYDOWLING7]

@robertsamm To confirm for number 7, Table 4.5 still suffice for" Completion of Baseline Activities Among Verified Participants for All Sites" and then once BSW is added to all reports, that will get a site-specific overall table (new Table 4.14). So what's needed is two versions of Table 4.5, one for Spanish one for English, and two versions for Table 4.14, one for Spanish and one for English? Where do you want those tables located?

Other then that, these are all coded and ready to be tested once we have stage data

[KELSEYDOWLING7]

@robertsamm These test tables/figures are using DEV data. Please let me know if I missed anything or if anything needs to be clarified:

1. Preferred Language of Verified Participants by Recruit Type and Site

2.Language Consented for Verified Participants by Recruit Type and Site

3.Number Verified Over Time by Preferred Language for All Sites/for BSWH: Note- there was only 1 Spanish and 1 English on 6/16 for BSW so those dots are overlapping, that's why you can't see the Spanish line just yet

4. How Recruits Heard About Study with Spanish as Preferred Language : Not enough data for the code to run but the fix is very simple. Might be able to fix with refactoring of Operations code.

5. Time from Sign-In to Submission of User Profile in Minutes by Preferred Language Among All Recruits

6. Time from Sign-In to Submission of User Profile in Minutes by Preferred Language Among All Recruits: Can't produce finalized table without running the full report but here's the preview. Do you need the footnote from table 2.3 as well?

7.Completion of Baseline Activities Among Verified Participants for All Sites/BSWH with English/Spanish as Preferred Language : Should 'Missing all Flags' be removed ?

8. Baseline Data Completeness by Preferred Language

[robertsamm]

Hi @KELSEYDOWLING7 thanks for creating these new metrics for Spanish! Sorry it took so long to provide feedback. My comments are below. Overall, these all look great and I don't have too much feedback. It would be helpful to look at these again once they are incorporated into the revised Ops report and we have Spanish data in prod to reference. Right now nothing major is jumping out at me but it's hard to tell with such little data.

1. Preferred Language of Verified Participants by Recruit Type and Site - this looks good. Only change needed currently is to change 'Total Recruits' labels in the 2 columns and 1 row to 'Total Verified Participants'. I'd like to look at this one again once we have the prod data for it, I'm assuming the 'not active' column would likely go away and then we would have data to fill in the 'active' column for Spanish.

2. Yes please add the same footnote as table 2.3

3. Yes, I think the 'missing all flags' should be removed. I assume this is in a QC check already correct? Where it would flag if there was a null value for a flag when there shouldn't be?

[KELSEYDOWLING7]

@robertsamm Great! For the QC rule, the Biospecimen rule should be in place already; the module flag completions QC is in the works, Jake and I are still checking, testing, and implementing IMS's new QC rules.

[KELSEYDOWLING7]

As mentioned in the punch list meeting, these metrics are done, but waiting on the first preferred language of Spanish for all tables to populate