agenda-live study review (Summer 2021)

[emach039]

Participants filling out the interest form multiple times; some accidental, some do it because they are ineligible and wish to participate.

EM/SM/AP to adjust protocols:

If confirmed duplicate participant:

1. identify which ID has data (must wait until they continue with one if no ID has data; can reach out to see if they need guidance)
2. delete ID on REDCap that has gone stale (does not have data)

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

If multiple eligibility forms show discrepancy (eligible/ineligible):

1. Flag on tracker
2. Take them on their word (assume that ineligible was a mistake)
3. Decision can be made later whether or not to use the data.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

EM/SM/AP to adjust REDCap:

Update message after participant completes eligibility form that indicates they will receive an e-mail within 1-2 minutes, please do not complete this form a second time.

+revisions from JA in Slack:

You will receive an email concerning your eligibility in the next couple of minutes; please do not fill out this form again. Please note that the day/time of part 2 of the study will be scheduled after you complete part 1). Even if you have already signed up for part 2 on Sona, you will have to select a new available time on Calendly, which will be offered to you when you complete part 1 of the study. (The time listed for part 2 on Sona is not the actual date of the part 2 visit; the one selected on Calendly will be the actual meeting time.) For any questions, please email [address]. Do you understand that you will shortly receive an email regarding your eligibility for this study and that you will need to schedule part 2 after completing part 1?

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

EM/AP/SM: rename variables inside REDCap (when should we re-clone?)

Hold for now.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

EM/AP/SM: prepare to transition REDCap projects to production mode

All three to hold before changing dev/prod at this stage. JA to follow-up with Viraj on differences.

JA :white_check_mark:

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

EM/AP/SM: add number of questions to participant information

add note to show participants how to click the survey queue at the top so they can see how many questionnaires they have left

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

GB/JA: discuss procedure for checking encryption and deleting from Zoom Cloud

Move to data monitoring to-do list.

JA :white_check_mark:

JA: optimize tracking sheet with max()

Hold. JA check into tracking sheets for color coding.

JA :white_check_mark:

ALL: Discuss splitting part 1 and part 2 on Sona.

No action at this time.

ALL: Policy on data backup frequency/timelines.

Current protocols set at 48 hours, which is working well for team. If falls on weekend, no later than the following Monday.

ALL: How are we on Pavlovia tokens?

George will replenish.

GB :white_check_mark:

[SarahMalykke]

Add to the instrument: "Decision about 1B" --> why not? A brief explanation from the participant why they do not wish to continue with the second part of the study

Added by Jess: The only relevant section found in the IRBs says: Participants will then be informed that Part 1A is complete and that they might be recontacted via email/text and invited to schedule a time to participate in Part 1B via Calendly.com. They will be reminded that they are free to decline any future invitations or to request that their contact information be deleted at any time.

George to make a suggestion in Slack. Then AP/EM/SM to implement into REDCap.

Follow-up discussion on 10/14: one question, not required, multiple choice:

• not comfortable being on a Zoom call
• already received credit I needed for the semester
• don't have interest to continue on
• scheduling conflict
• other (please specify)
• prefer not to say

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[SarahMalykke]

Alert participants that have completed the consent form, but not continued with the rest of part 1A - perhaps an automated email that is sent a day or two after they complete the consent form (but still haven't completed any other part of 1A)

George to play around with way to calculate dates on conditional logic to add another reminder. Emily noted that there is currently a process that reminds them at the one week mark.

GB :x:

Decision to reduce the time before automated e-mail goes out if someone completes consent but doesn't complete all the questionnaires. Currently at one week, switching to three days.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

Incorrect e-mails from participants

Arina: It is not letting me edit, is there any way I could change it or should I just create a new Id for the participant and input the same information except for the email

Jess: I just did some digging and it looks like our REDCap administrators do not provide us (that is, any project admin, George included) with lock/unlock record access. As a result, we cannot change data within the surveys. Perhaps this is something we should ask Viraj about (but maybe later, to keep them focused on our data recovery question).

EM to mention potential solution in Slack.

EM :white_check_mark:

[jessb0t]

Is there any e-mail delay between completing 1a and starting 1b? What is the purpose of the "Completion of Part 1A" instrument on REDCap?

SM+EM: to modify REDCap behavior so interest in 1B is immediately sent after completing 1A.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

Are identifiers flagged in REDCap?

EM/AP/SM to update REDCap: The following four fields on the eligibility form need to be marked as identifiers:

1. user_fullname
2. user_email
3. user_phone
4. birth (date of birth on demographics questionnaire)

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

EM/AP/SM to update protocols: Exporting REDCap data should have "Remove tagged identifier fields" checked.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

For REDCap CSV files that have already been backed up, EM/AP/SM to do tonight:

1. pull a fresh backup of the REDCap CSV data
2. save new backup to HPC
3. delete all existing CSVs on HPC

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[apoly1]

1) Possibly changing the wording for eligibility question on vision in the future. At the moment it asks "Do you have normal or corrected-to-normal vision (i.e., can see color normally, and if you need glasses, you are wearing them or contact lenses)?" which might be confusing to some of the participants since it seems like it's a little frequent for the participants to answer "no" (maybe thinking that they do not have corrected vision).

See notes below.

2) Adding a comment to the protocol to unlock the zoom meeting when the participant leaves it since they will be unable to return back if it's not unlocked.

Exists in SC and RWE, ML to check.

ML :white_check_mark:

[jessb0t]

Should we automate the dropping of identifiers for RAs so they don't have to remember to click? This is in the User Rights section if one edits privileges for RAs:

Test.

JA :white_check_mark:

Make change to all three live studies.

JA :white_check_mark:

[jessb0t]

Trackers and color coding: overwrite with black when participants not continuing?

JA to fix color-coding for participants that are ineligible. Add y/n column for eligible/ineligible.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

Cross check of eligibility forms:

Questions about Prolific on ML and RWE, but not SC:

Delete this when eligibility form is revamped.

Sona ID (empty variable) on ML but not SC or RWE:

Leave for now, but will not re-create with new eligibility form.

[jessb0t]

Zoom Recording Renaming

Change protocol for renaming to clarify only renaming top-level to replace participant name.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

Eligibility Form

test stop logic does not impact data we receive about ineligibility.

EM :white_check_mark:

revamp eligibility form:

• delete yes/no age question (add branching logic if they pick outside the acceptable range)
• add stop (or branching logic) to all eligibility form questions.
• add question to eligibility form: are you on mobile now?
• change vision question: do you have normal vision? follow-up: Is it corrected to normal? (For example, you wear glasses, had corrective surgery, etc.)
• add colorblindness question: Are you colorblind?
• do not include prolific questions

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

From George on Slack:

I was chatting with Sarah and one thing to be on the look out for, now that people can auto sign-up for part 1b, after completing part 1a, is that some might sign up, but still not have completed all tasks yet (pavlovia tasks) for part 1a. Please note that participants cannot complete part 1b until they complete those (or at least attempt them). Also, whenever someone is found to have not attempted a pavlovia task, you of course need to email them to let them know they cannot move on to part 1b until they attempt all tasks. You then send the link. But, we really need to understand why this is somewhat common. So, when emailing in these cases, please also include the following statement/question (and enter their response as an additional column in the tracker for now): "I would also like to please ask a favor. We are trying to improve how we run this study online, and so it would be helpful to understand why this task(s) might have been missed and/or not completed. Can you please let me know if it was intentional that you did not complete this task (that is OK if so, and we will not count this against you in any way)? Or, if this was a mistake, can you please explain why you think you might have missed this task by accident?"

SC :white_check_mark: ML ::white_check_mark: RWE :white_check_mark:

[jessb0t]

Incomplete Pavlovia tasks

How to collect post-task questionnaire information if only complete a subset of the tasks?

From George:

With that said, it appears that it is not uncommon for participants to not complete some pavlovia tasks before signing up. For now, I think the plan of just emailing these participants and requiring the to complete before doing part 1b is a great plan and you all are doing great implementing that. I do want to think about ways we can improve this though after july but before fall. That is, ideally we can modify pavlovia and redcap slightly so that redcap has knowledge of pavlovia completion in an automatic fashion. I have some vague ideas of ways to potentially do this, but nothing concrete yet. So, any suggestions are welcome and let's add this to our list of things to try to optimize after july but before fall.

From Arina:

it seems that the best way of confirming that the participants complete their tasks in the future is to create a link between redcap and pavlovia if it’s possible. If it’s not an option, maybe if redcap has a questionnaire setting where a participant has to give a right answer to the question to be able to continue (the right code/answer will be provided upon completing pavlovia task), so that way we could program pavlovia to have the last “slide” with the key code. This way the participant will not be able to skip pavlovia tasks and continue with the study. The issue here is that some participants might just drop the study in the middle of it if they feel stuck/frustrated with this situation, and maybe we will have to provide more guidance to the participants through emails so that could slow down the completion of the tasks, but I don’t think there is a way around having technical issues anyways.

thoughts from meeting

1. Prohibit participants from continuing with questionnaires unless they can answer a question generated at the end of Pavlovia (e.g., written out (with instructions on capitalization) "science" or list of options from which to choose the correct response). Need to determine if it is possible to "hold" user at an instrument until they give the "right" answer (option). Add participant messages explaining expectations and recourse if they don't know how to continue.
2. Consider compiling all three EF tasks into a single Pavlovia link.

GB :question: :question: :question:

Follow up 10/14: Arina had found that we could use a free text box to input a code and notify the participant if they do not provide the right code. Possible to add a message directing the participant to the study lead if they don't have the code.

Change surveys not to say "Thank you for taking this survey" - George would like us to change this across all of them so that it just says "Please continue to the next survey." >> confirmed by SM that this does not appear for participants due to survey queue logic!

RWE :white_check_mark:

Someone to check both SC/RWE and ML to see if the logic between the two paradigms is identical from pre-Pavlovia questionnaire and the Pavlovia tasks, using "999" as participant number.

GB :x: SM :white_check_mark:

[jessb0t]

Eligibility Form Completion Message

Suggestion from Arina:

I think it would be a good idea to also add into survey completion text something like “Thank you for taking the survey. You will receive further instructions shortly to the email that you mentioned. Please do not complete this form again. If you have committed any errors or have questions email us at [email].” Does this sound good?

already implemented

[jessb0t]

Slack Channel

Can we rename "Live Study Mgmt" (since we are done "setting up")?

JA :white_check_mark:

[jessb0t]

Summer Numbers for All Three Studies

Collated table here.

GB :eyes: :eyes: :eyes:

[emach039]

New additions to discuss for the interest form:

1. After asking if they're on mobile or not, a question appears that says: "Note that for the rest of Part 1 and Part 2, you should be on a desktop or laptop to complete the surveys that follow." Regardless if they're on mobile or not, this will appear to clarify that they need to be using a computer,

All agreed this was a good idea. Emily included in revised eligibility form.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

2. For the email address box: "What is your email address? Make sure this is correct, as this is what we will be using to contact you." This would be an added layer of caution to make sure the email inputted is correct, as multiple participants have accidentally typed the wrong email across all studies.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

JA: maybe we could do a "confirm" field (so they have to input their e-mail twice)?

See if we can do a "Confirm e-mail" field with logic if the two e-mail fields don't match up.

AP: :x:

[emach039]

Issues with branching logic in the age question for the interest form:

"What is your age?", with a box for the participant to manually type their age.

I am unsure if it is possible to do branching logic for an interger range, to inform the participant that this age makes them ineligible. Additionally, it's not possible to create "STOP" logic for these kinds of questions, as it's interger only and not set values.

Arina has solution (in main ML REDCap), she put it in Slack. Also discussed that we could send eligibility form to participants so they know what they answered.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

Subsequently, decisions were made to drop the stop logic but send the eligibility form to participants via e-mail.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

Summer>Fall Pause

Do we need to do anything to shut down data collection? (Saw a new REDCap form come in this week.)

Sona currently shutdown; nothing specific to be done at this stage.

[SarahMalykke]

I wonder if it's possible to connect Pavlovia to REDCap in a way so the participants can't move on to the next task without completing the previous task to avoid participants completing 1A and signing up for 1B while still missing a task or two.

JA: I think Arina has a suggestion above. Participants come back to REDCap after each Pavlovia task, right?

discussed above.

[jessb0t]

Can we get Part 1B to be worth more Sona credit?

JA to include in IRB amendments 9/2021.

JA: :white_check_mark:

[apoly1]

We should have a protocol including the information of what to do if the researcher/confederate knows the participant, probably it should be a simple text about asking another RA to handle it, but it's good to have it written.

Follow-up 10/14: if experimenter (or confederate) know the participant before you run, get someone else to run them. If you don't realize it until you're on the call, try to follow normal procedure (but don't be weird like pretending that you don't know them). Mark on the tracker that you know them and add a note. We won't use the data, but it is not fair to the participant not to give them the experience. All three protocols need to be updated:

SC :white_check_mark: ML :x: RWE :white_check_mark:

[jessb0t]

Consent Automated Invitation - Do On Computer

Emily and Sarah to adjust content to match Arina's (as applicable to SC and RWE). (Sent on Slack.) Emily sending hers also on Slack for Arina to compare.

SC :white_check_mark: ML :white_check_mark: RWE :white_check_mark:

[jessb0t]

When Studies Run Long

Adjust protocols to add Sona credit if session runs over, after confirming with participant if they want to continue.

Follow-up 10/14: If you think the study is going to run over, say the following to the participant: "We might run over a little bit. If we do, would you like to continue? You don't have to. We will round up to the nearest half-hour and you will get the credit for all that time."

SC :white_check_mark: ML :x: RWE :white_check_mark: