Weekly Survey Metrics Report Revamp

[robertsamm]

We would like to revamp the Weekly Survey Metrics Report now that we have completed the Ops Report and have added more metrics to the report for recently released surveys. Please see below. This is a lower priority request, higher priority items should take precedent.

General Report Flow Updates

• [x] 1. Retitle report as: Connect Weekly Survey Module Metrics

• [x] 2. Add table of contents to report and separate metrics into sections (similar to how Ops Report is presented now). -Suggested categories: Baseline Survey, SSN Survey, Menstrual Cycle Survey, Quality of Life Survey, Connect Experience Survey, Survey Metrics by Language Completed -Tables and Figures should also be renumbered to go with the section it is in

• [x] 3. Add 'general information' summary to report. Suggested summary language: This Weekly Survey Module report contains metrics related to Connect survey completion including baseline surveys and follow up surveys. Additional metrics will be added as new surveys are released. Metrics related to the biospecimen surveys can be found in the Weekly Survey Metrics. Metrics related to overall baseline study activity completion, including baseline survey completion, can be found in the Weekly Operations Report. [add info about analytic data set?] The Survey Module report is run on a weekly basis and encompasses data through [add info from data run].

• [x] 4. For 'Survey Metrics by Language Completed', at beginning of section please add note about when Spanish was added. Note: Spanish was made available in MyConnect beginning August 1, 2024.

Updates to current metrics

• [x] 5. Table 10. Portion of SSN Given by Submission Status of SSN Survey -Add' Among Verified Participants' to end of table title -Label 'Total' column and row as 'Total Verified' -Make 'Total' column a column percent

• [x] 6. Add column percent to Table 12. Completion of 3-Month Quality of Life Survey by Baseline Data Complete

• [x] 7. Add footnote to table 13: 3-Month Quality of Life Survey Completion Rate by Site Defined Sex Among Eligible Participants Footnote: Sex defined as the de-identified sex data reported by sites at time of invitation.

• [x] 8. Add footnote to table 14: -Month Quality of Life Survey Completion Rate by Age Among Eligible Participants Footnote: Age defined as the de-identified age data reported by sites at time of invitation, not current age at time of survey completion.

• [x] 9. Spelling error in 'Table' title for Table 17. 3-Month Quality of Life Survey Completion Rate by Demographic Among Eligible Participants

• [x] 10. Table 18. 3-Month Quality of Life Survey Completion Time from Start to Submit Among Eligible Participants -Percentages don't add up to 100%. Can you please look into this? Should be out of the ~19,000 that submitted the QOL survey. -Please add to end of table title 'That Submitted the Survey' -Please change 'total' row label to 'Total Submitted QOL Survey' -Please add cumulative percent column to the right (similar to Table 6.1-6.4)

• [x] 11. Please add footnote to Table 20. Preferred Language by Site Footnote: Preferred language is based on language of signed consent form and current user profile language status'

• [x] 12. Add Connect Experience Survey to tables 22, 22a, and 22b for Survey completion by survey language. Can abbreviate the survey to 'CES' and place after 'QOL'.

New Metrics to Be Added

• [x] 14. Please add similar tables to Table 13-16 (QOL survey completion by age, race, sex) but for the Initial Survey. -All 3 tables (including race) can be pulled from the de-identified site reported demographic variables. -Instead of 'not started, started', submitted', please use these categories: 'All, 2 or 3 sections, BOH only, None' (from table 4.1 in Ops Report).

• [x] 15. Add a new categorical table for QOL survey completion stratified by time from verification to survey release among both prospective and backlog participants (one table total) -Metric title: QOL Survey Completion By Time from Verification to Survey Release -Survey completion categories should be 'Not Started', 'Started', 'Submitted'. -Time from Verification to Survey Release should be: 'More than 1 Year' (~July 2021-Dec 2022), '6 Months - 1 Year' (~Jan 2023-June 2023), '3 Months - 6 Months' (~July 2023-Dec 2023), 'At 3 Months' (anyone prospective)

• [x] 16. Add a new metric that looks at among those who completed the QOL survey, how many people went back in to complete their baseline surveys on the day of or after completing the QOL survey (some of this data was looked at when pulling the extra duplicate report data for Michelle). Should be stratified by each baseline survey section and the biospecimen surveys. -Metric title: Baseline Survey Completion On Day of or After QOL Survey Completion

• [x] 17. Repeat request 16 but for the Connect Experience Survey

[KELSEYDOWLING7]

Hi Amelia, thanks for making this checklist! It makes it MUCH easier to keep track of everything. I had some down time and these were all pretty quick changes.

Please let me know if I misinterpreted any of the new requests. I made the assumption that number 14 should only be out of those who completed CES, and that the rows in number 15 should be those verified at those dates: 'More than 1 Year' (~July 2021-Dec 2022), '6 Months - 1 Year' (~Jan 2023-June 2023), '3 Months - 6 Months' (~July 2023-Dec 2023), 'At 3 Months' (anyone prospective). Also we already had a short intro just to explain the module acronyms. Please let me know if you want that kept, removed, or moved. A good alternative spot might be under the "Baseline Survey" header.

CCC-Weekly-Module-Metrics_RMD.pdf

[robertsamm]

Hi @KELSEYDOWLING7 yay! I had noticed you made the checkmarks on a recent github issue so thought it might be easier to do the same thing here! And wow, that was fast! This is looking great - I just have a few small edits:

• [x] 1. Short intro just to explain the module acronyms - Agree, lets move that portion of the intro to the baseline survey section!

• [x] 2. Table of Contents - Sections themselves look good. Just need to add a table of contents page at the top where when you click the section in the TOC it jumps to that section (similar to Ops Report)

• [x] 3. Table 4.2-4.2b on baseline data completeness - can you please change the order of the categories so 'both survey and samples' is at the top and 'neither' is at bottom so it goes in order from most to least completed? (similar to how we changed it for table 5.2 for the CES). Sorry I didn't catch this earlier

• [x] 4. For the initial request 14 to add age, race, sex tables for the Initial Survey - sorry for the confusion! This should be among 'all verified participants' (not only those who completed CES). For your tables I think the only things that need to change are: -Make the total out of Verified Participants and change the total labels to reflect that -Change title to: 'Initial Survey Completion Rate by Site Defined Sex/Age/Race Among Verified Participants' -Make total column a column percent, but leave other cells as row percents -Move the 3 tables to the baseline survey section

• [x] 5. Table 4.10 QOL Survey Completion By Time from Verification to Survey Release - this looks good! Just a few minor tweaks: -For the title and category label row can you make it 'Time from QOL Survey Release' to clarify it's the QOL survey and not initial survey release time? -Can you leave the row percents as is, but make the total column a column percent please?

• [x] 6. Table 4.11 Baseline Survey Completion on Day of or After QOL Survey Completion - this also looks good! To shorten the category labels could change 'Completion Time' label to 'Survey Completed After QOL' and then list the surveys below (e.g. BOH, MRE, SAS, ...., None etc)

• [ ] 7. Table 5.8 Baseline Survey Completion on Day of or After CES Survey Completion - suggest making the same changes to Table 5.8 for CES

[KELSEYDOWLING7]

Good morning, few questions/comments. Table 5.8 already has the abbreviations for CES so I believe we should be good there. Then, I'm a bit confused at what the new title should be for 4.10. Can you clarify specifically what you'd like the title and category row label to be?

[robertsamm]

Good morning @KELSEYDOWLING7!

For table 5.8 - it should have the same changes applied in number 6/table 4.11 where the category row label would state 'Survey Completed After CES' and then the survey labels in the rows are shortened to just refer to the survey itself and drop 'completed after CES' in each label

For Table 4.10 - Just need to add another 'QOL' into the title and label so Title: QOL Survey Completion By Time from Verification to QOL Survey Release Row label: Time from Verification to QOL Survey Release

[KELSEYDOWLING7]

Sorry, maybe I'm missing something? I think 5.8 already has all that:

CCC-Weekly-Module-Metrics_RMD.pdf

[robertsamm]

Hi @KELSEYDOWLING7 I was envisioning something more like below for Table 4.11 and 5.8. What do you think? That way the 'completed after QOL/CES' doesn't have to be repeated in each row.

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

Table 4.11. Baseline Survey Completion On Day of or After QOL Survey Completion |   -- | -- Survey Completed After QOL | N(%) BOH | N(%) MRE | N(%) SAS | N(%) LAW | N(%) BUM | N(%) BU | N(%) MW | N(%) COVID | N(%) None | N(%) Total Submitted QOL Survey | N(%)

[robertsamm]

Forgot to add, I still need to review the changes to the rest of the report which I will do shortly!

[brotzmanmj]

One small request, can you include a note somewhere (the summary on the first page or in the Initial Survey section, a note to indicate the date when all four modules were in MyConnect?

[KELSEYDOWLING7]

Sure thing, do we have that date anywhere?

[robertsamm]

Yes, July 5th, 2022 is when all 4 sections were available in prod

[KELSEYDOWLING7]

How does this sound:

Starting July 5th, 2022, all four Initial Survey Modules were in MyConnect. In order of how the sections appear on MyConnect, the Initial Survey Sections are:

BOH= Background and Overall Health (Module 1)

LAW= Where you Live and Work (Module 4)

MRE= Medications, Reproductive Health, Exercise, and Sleep (Module 2)

SAS= Smoking, Alcohol, and Sun Exposure (Module 3)

[robertsamm]

@KELSEYDOWLING7 yes, I think that sounds perfect!

[robertsamm]

Hi @KELSEYDOWLING7 final suggested edits hopefully!

• [x] Table 1.9-1.11 - titles still mention CES survey. Should only reference the initial survey and footnotes added back in. -Can you change titles to 'Initial Survey Completion Rate by Site Defined Sex/Age/Race Among Verified Participants' -Can you also add in the same footnote as the other age/race/sex: Footnote for sex: Sex defined as the de-identified sex data reported by sites at time of invitation. Footnote for race: Race defined as the de-identified race data reported by sites at time of invitation. Footnote for age: Age defined as the de-identified age data reported by sites at time of invitation, not current age at time of survey completion.

• [x] Move menstrual cycle survey metric(s) from Weekly Survey Report to Weekly Biospecimen Report - after discussion with biospecimen team, think it makes more sense to delete this from the Survey Report and instead move it to the Biospecimen Report because the menstrual cycle survey is dependent upon the answers and data from the biospecimen survey which those metrics are found in the Biospecimen Report. @erincschwartz can suggest where to add this on the Wkly Biospec Report. The entire section for the menstrual cycle can be deleted from the Survey Report.

• [x] Update general summary to note menstrual cycle survey and COVID survey metrics can also be found in the Weekly Biospecimen Report. -Suggest revising 3rd sentence in summary to: Metrics related to the baseline biospecimen surveys, menstrual cycle survey, and COVID survey can be found in the Weekly Biospecimen Metrics reports.

• [x] Update baseline survey summary as outlined in comment above

• [x] Table 4.11 and 5.8 updates discussed above

[KELSEYDOWLING7]

So do we want to remove the mention of the Menstrual cycle survey from these tables as well? The other biospecimen surveys (BU, BUM, and COVID) are on there too. Should those be removed?

If not, please see below:

CCC-Weekly-Module-Metrics_RMD.pdf

[robertsamm]

@KELSEYDOWLING7 no, I would leave the mention of the menstrual cycle and covid surveys in those metrics but for the one metric that is specific to menstrual cycle only that one can be moved to the biospec weekly report

[KELSEYDOWLING7]

Got it, ok that table/section was removed in that last version I just posted

[robertsamm]

Thanks @KELSEYDOWLING7 I just reviewed the latest version and it looks great! The footnotes need to be added in to the age/race/sex tables 1.9-1.11. After that, this looks perfect and is ready for prod on Monday!

[KELSEYDOWLING7]

Oh weird! The footnotes were all in the code but weren't rendering. That's resolved now, please let me know if we're good to close the issue. We're also updating the report automation this afternoon for Monday. CCC-Weekly-Module-Metrics_RMD.pdf