Test Survey Collection

[kmorton2017]

We need to ensure that the survey is properly collecting participants' information from the platform.

Steps to QA: 1) Login as a student. 2) Ensure that the pop-up modal text has changed to say "Win a $100 gift card and take our post-season survey. Take our post-season survey." 3) When you click on the survey, you should not see any questions related to email name, school, city, state, country, team, or age since this should be collecting automatically. 4) Note any mistakes or typos on the survey so Katie can edit. 5) Upon completion, note the name(s) of your student test account. 6) Go to Survey Monkey (login info in Assessment vault) and find the 2019 Technovation Student Post Survey. Click on analyze results. Download an export. 7) Ensure that your survey responses are recorded. Scroll all the way to the righthand side of the excel doc to ensure that your email name, school, city, state, country, team, and age have been properly recorded.

Repeat steps #1-7 for a mentor account. Note: the name of the survey for #6 is 2019 Technovation Educator/Mentor Post Survey.

@nickiredescent can you help me with this? I will also be testing.

--

[nickiredescent]

Tested. Survey does not directly ask for identifying information (name, age, etc.). Survey Monkey is properly exporting and saving our participant data. I cannot get the pop-up to show up on my dashboard. I've tried with previously made accounts and newly created accounts. I've tried with all the different judging settings.

[stenington]

@nickiredescent I see some code in there that suppresses the survey modal if the user has already taken it and limits the number and frequency of reminders they receive if not, and I don't see any way that those things get reset. So my guess is that most of our test accounts either took the pre-survey or reached the limit of reminders, and now they are not being shown the modal even though the link changed to be a post-survey.

I'm guessing what I have to do is go in and remove the record of users being shown the survey, and reset the reminder count to 0. I would have to do this in production as well, and I suspect as soon as I do survey modals will start popping up on dashboards so we would either need to coordinate to do it when the modals should start appearing, or I think if we leave the survey link blank, no modal will be shown until it gets filled in.

Anyway, let me look at testing my hypothesis on staging and I'll update here when I've changed the data.

[stenington]

Okay, so I cleared out the following fields for one user, and saw the modal immediately upon refreshing the dashboard: a.update(survey_completed_at: nil, reminded_about_survey_at: nil, pre_survey_completed_at: nil, reminded_about_survey_count: 0)

The survey link presented to me looked like https://www.surveymonkey.com/r/TWRLM9G?name=Lama%20Hussein&school=Friends%20International%20&city=Bethlehem&state=West%20Bank&country=PS%20-%20State%20of%20Palestine&email=katie+s4@iridescentlearning.org&team=Kitaab&age=14 but I'm not seeing the custom variables when I export from Surveymonkey, so I'm not sure what the disconnect is. (Possibly I'm just exporting wrong from Surveymonkey.)

As for the data getting cleared out, I assume it's okay to clear and we're not relying on it for later analytics? It's data about whether or not they completed the pre-survey, and when.

[stenington]

As for clearing out data for further QA and to make this work in production, I'm not sure what the best approach is. @nickiredescent @kmorton2017 do you just want to ping me in Slack if you need survey data cleared while testing?

[stenington]

Oh, I was wrong, the data is in fact coming through. There are empty name, email, etc. columns early in the spreadsheet, but I do see the custom variables at the far righthand side of the doc as described above.

[ophelieh]

@stenington That sounds right. That's how it was last season. @nickiredescent and @kmorton2017 Could you test again today? Thanks!

[nickiredescent]

Retested by logging into random accounts. I was not able to consistently see the pop up: Survey Pop Up:

• alli+14s
• kai+3s
• nick+9s
• nick+10s
• nick+5s

No Survey Pop Up:

• katie+8s
• mackenzie+2s
• ophelie+1s
• katie+14s
• kai+1s
• nick+7s

I couldn't identify any patterns with what worked and what didn't. With the accounts that did receive the pop up, immediately logging out and back in did not have a pop up the second time. I'm unsure if there's a cooldown built in. I did not take any surveys for any of the accounts.

[stenington]

@nickiredescent I'm not too surprised by this. I hadn't cleared out the survey reminder data for all users, just the one user I was going to test. I can clear it out for all users, I was just waiting for an official "go ahead" to make sure we're all on the same page.

As for cooldown, yeah it does look like there's some logic like that in there too. Here's what I think I'm seeing:

1. If a survey link has been properly configured in the admin settings page, the user has not taken it, and the user has address details (not sure why this one's a factor) go on to step 2. If not, don't show the reminder.
2. If they haven't yet been reminded, they're registered for the season, and they registered over 2 days ago, show them the reminder.
3. Otherwise if they have been reminded, if they've only been reminded once and it's over 2 days since that reminder, remind them again.
4. Otherwise don't show them a reminder.

Kind of complicated, but that's what it looks like.

[nickiredescent]

@stenington If I had to guess, we need an address in step 1 because if we select a random gift card winner, we need to know what country they're in so we can follow proper procedure. I guess otherwise, we're just waiting for a final ok. Thanks for the update!

[kmorton2017]

@stenington so if i am understanding correctly, all you need to do is reset the survey counter so that it starts popping up on people's dashboard. currently they are "maxed out" because they have seen it so many times.

if that is the case, please go ahead and reset the survey counter so they have the ability to start seeing it on their dashboard.

FYI for the time being i cleared out the modal text since we do not have a survey for them to take right now but will have one for them next week.

[stenington]

@kmorton2017 Cool, sounds good.

I've reset all current accounts on staging for testing. When this is ready for production, I will reset current accounts there as well so that when you put the survey links in, they'll start showing up for people right away.

[ophelieh]

@nickiredescent and @kmorton2017 Could you both retest this?

Thanks!

[nickiredescent]

Tested. Can confirm that the post survey pop up is showing up.

[kmorton2017]

@stenington can you reset the accounts on production so that they are able to see the modal text appear?