SrvMC_MenstSrvElig_v1r0 flag showing "N/A" for individuals that should be eligible

[mnataraj92]

Kelsey's QC report shows a handful of individuals that have either the BUM Survey or the BU Survey completed with SrvBlU_MENST60_v2r0 answered yes, but the SrvMC_MenstSrvElig_v1r0 flag is showing "N/A" instead of yes. The Connect IDs are here: https://nih.app.box.com/file/1445746617081

[brotzmanmj]

Whoa there are a lot of Connect IDs on that list. @kelsey can you tell us how many there are, and when they completed their BUM or BU survey? Maybe link to a CSV file that includes the date? We need to tread very carefully as to how we handle this, especially if they completed that BUM or BU survey more than 45 days ago. Also please include if they completed the BUM or BU survey, since that will tell us if these were research collections or clinical collections.

I looked up one person on the list out of curiosity, in the SMDB. They gave their sample and completed their biospec survey in 2022 and got all of the menstrual cycle survey reminder when we would have expected them to, back in 2022.

[KELSEYDOWLING7]

@brotzmanmj Hi Michelle, I uploaded that csv list in box here https://nih.app.box.com/file/1446538544915. I also included whether or not they competed the Menstrual Cycle Survey already.

I triple checked the query and the csv output file; all have completed either the BU or BUM survey with MENS60=yes, all have a completion time stamp. and all have a blank Menstrual Cycle Survey Eligibility flag. Please note there's no one with a "no" set for the eligibility flag, it is either yes if they are or null if they are not.

[brotzmanmj]

Thanks @KELSEYDOWLING7 I'll review the data now and report back with suggestions

[brotzmanmj]

There are 172 participants on this list. Of them, I would classify them in two groups:

Group 1: Research Collections (151 of the 172)

• 107 have completed the MCS
• 30 started the MCS but did not complete it
• 14 did not start the MCS
• They ranged in date of BUM survey completed from 6/29/2022 to 12/07/2022
• Some of them got the MCS notification emails and others did not; it's not clear to me why, would need more investigation

Group 2: Clinical Collections (21 of the 172)

• None have started the MCS
• They ranged in date of BU survey completed from 6/14/2023 to 7/26/2023

All 172 are too long ago (more than 45 days) to be asked to complete the MCS survey now. We do NOT want to do anything that would trigger email notifications to them at this time or in the future.

Here is what else we know: "For the Research Menstrual Cycle Survey notification, we implemented this change in prod on 12/9/2022: Updated conditionality to accomodate new derived variable for menst cycle survey eligibility- SrvMC_MenstSrvElig_v1r0 (289750687) = Yes (353358909) replaces two previous conditions: Blood/urine/mouthwash survey status (265193023) = Submitted (231311385) and answer to Menst60 question from bio survey (D_299215535.D_112151599) = Yes (353358909)." I think that explains why the research participants have null derived MCS eligibility variable, because it did not exist at the time and we did not backfill it. Therefore, no action is needed except to ensure that we do NOT backfill it in a way that would result in them getting MCS survey reminders in the future.

For the 21 clinical collection participants, the derived variable should have been in place for them in June/July of 2023. I have no idea why it wouldn't have been derived. But it's too late now for them to complete the survey. So we should leave it null.

I think the only actions to take are: 1) @Davinkjohnson how can we ensure that the derived variables for these individuals remains null so that we don't accidentally trigger the MCS notifications to them in the future? 2) @KELSEYDOWLING7 and @mnataraj92 mark these individuals as known QC issues for this derived variable so we don't re-investigate them in the future. 3) @KELSEYDOWLING7 add this check to the weekly QC run so we catch it faster. The MCS survey can only be completed within 45 days of the BU / BUM survey so we need to know as early as possible if this derived variable is not calculating.

Any other thoughts?

[KELSEYDOWLING7]

@brotzmanmj The rule is part of the Weekly Recruitment QC as of last week, and I've updated it to exclude anyone who completed the Blood Urine or Blood Urine Mouthwash Survey prior to 12/9/22. That change will also be applied on Monday's QC report @rebexxxxxx

I'll also remove the 21 Clinical participants that can no longer be notified. So Monday's report will likely have no rows yet for this rule

[brotzmanmj]

Thanks @KELSEYDOWLING7. I noticed this issue is still open.

@mnataraj92 please check the QC report and see if there are any new issues with this since Kelsey updated it to remove the ones that were too long ago to resolve.

@Davinkjohnson can we take any action to make sure the derived variable does not update for the people in the backlog, so they don't get the MC survey in the future?

@KELSEYDOWLING7 are there any checks you can run from Feb 16 to now to confirm that SrvMC_MenstSrvElig_v1r0 has derived correctly for everyone since we last investigated this? And also check if everyone with SrvMC_MenstSrvElig_v1r0 = yes since February 16 to now got their MC survey notifications?

[mnataraj92]

Hi @brotzmanmj there are no new issues with this in the QC.

[KELSEYDOWLING7]

@brotzmanmj Yes, SrvMC_MenstSrvElig_v1r0 has derived correctly for everyone after 2/16/24

Assuming participants are to receive their first MC reminder 5 days after completing (so looking at completion dates of 2/17/24-3/22/24 BU/BUM completion): -There are 41 people with SrvMC_MenstSrvElig_v1r0 = yes with Clinical BU surveys completed since February 16th that have not gotten MC survey reminders -There are 28 people with SrvMC_MenstSrvElig_v1r0 = yes with Research BU surveys completed since February 16th that have not gotten MC survey reminders.

I've upload those participants CIDs and BU/BUM completion times on Box here: https://nih.app.box.com/file/1484981298089

[brotzmanmj]

@KELSEYDOWLING7 thanks for checking. Before I look at this, out of curiosity, can you add to the list if the people in question completed the MC survey, and if so, the date/time submitted? Because I looked at the conditionality and that is the only condition that would prevent someone from getting the reminder.

[KELSEYDOWLING7]

@brotzmanmj Your hunch was right, they all have the MC survey completed. Most actually completed the MC survey the same day as their biospecimen survey

[brotzmanmj]

@KELSEYDOWLING7 thanks for confirming. So, the emails are working as expected. For the notifications report that we just settled on this week, can you add this check for the MC notifications? And then we can close this issue.

(as an aside, this probably means that a lot of people are misinterpreting the MC survey. I'll start a separate email about that with you and Julie and Marie Josephe)

[KELSEYDOWLING7]

@brotzmanmj Of course. And all set, those are already on that notifications report

[sonyekere]

Hi @brotzmanmj and @KELSEYDOWLING7 , Can this issue be closed?

[KELSEYDOWLING7]

@sonyekere From my end this is good to close. We have the QC to monitor any future issues

[brotzmanmj]

Agreed