hedyorg / hedy

Hedy is a gradual programming language to teach children programming. Gradual languages use different language levels, where each level adds new concepts and syntactic complexity. At the end of the Hedy level sequence, kids master a subset of syntactically valid Python.
https://www.hedy.org
European Union Public License 1.2
1.33k stars 290 forks source link

📜 Trigger survey-email to teacher #3995

Open woutccc opened 1 year ago

woutccc commented 1 year ago
  1. GOAL: retrieving feedback (for improvement purposes) from teachers on why they drop out of the program on 3 different points in the teacher journey:
  1. Survey link to be sent to teacher via email through Mailchimp
woutccc commented 1 year ago

@noelgolding Hi Noel, it was you how was volunteering last Tuesday for helping our with this issue right? What do you need from me to get this moved forward? Note I have created just 1 survey in Mailchimp and no invitation email or jourbney yet (need to figure out how, but dont expect this to be too difficult)

woutccc commented 1 year ago

@Felienne is there a reason why I cant assign myself (and Noel) to this issue?

Felienne commented 1 year ago

Yes, you have to be members of the org first! I will invite you tomorrow

is there a reason why I cant assign myself (and Noel) to this issue?

noelgolding commented 1 year ago

@woutccc

What do you need from me to get this moved forward?

You will need to create an email template with the survey link, or embedded survey. I don't think you will need to create a journey or welcome email for this survey.

I will create a new campaign via the mailchimp api, each time the survey is to be sent to a new group

I will need the mailchimp api credentials and email template id. we can sync up on those items outside of this conversation.

@Felienne we may need to update the db schema to either have a join table or new field to identify the users we have already sent a survey to so we don't spam them.

Felienne commented 1 year ago

Yes, you have to be members of the org first! I will invite you tomorrow

I have assigned you now @woutccc! (Noel was already there)

noelgolding commented 1 year ago

@Felienne based on the db schema, I don't think we have the necessary data to formulate the queries for scenarios 2 and 3. Please let me know if there is an updated schema that exposes the level for a given class and whether or not the course was completed for a given class.

Felienne commented 1 year ago

@Felienne based on the db schema, I don't think we have the necessary data to formulate the queries for scenarios 2 and 3. Please let me know if there is an updated schema that exposes the level for a given class and whether or not the course was completed for a given class.

Hi @noelgolding!

Firstly, I am not 100% sure the schema is up to date (maybe @jpelay can chip in?). But I think based on the information we have, we can do something for scenarios 2 and 3:

We can look at the class, fetch all students in the class, and then fetch their programs and see if the students reached a certain level (and for performance reasons, we can store that max_level on the class so we don't have to do that expensive query multiple times).

The question is what "reaching level 3" means. One student? All students? The majority of students? But that is a definition question and not a technical limitation. I am inclined to say something like 80%?

Similarly for TEACHER_DropOut_3, the question is what it means to "follow for a whole year". Some schools might just have , say, 8 weeks for programming courses and might fill them with Hedy entirely. Is that a drop out or not? They might plan to do the same next year. So maybe for scenario 3 we want something like more than a year of no activity? That means that they did not start at the same time (But for now, I think we can do these manually and should focus first on 1 and 2).

Hope this helps!

noelgolding commented 1 year ago

would we want to limit scenarios 2 and 3 to teachers with just 1 class? or do we want to find any class that matches the criteria?

Felienne commented 1 year ago

would we want to limit scenarios 2 and 3 to teachers with just 1 class? or do we want to find any class that matches the criteria?

Maybe all classes? Like this:

TEACHER_DropOut_2: Account created, created one or more class but stopped (-> no activity for 2 weeks in any class before having any student in any class reach level 3)?

Felienne commented 1 year ago

But of course we can also take it "gradual" and first start to find teachers with 1 class! that stopped.

jpelay commented 1 year ago

I am not 100% sure the schema is up to date (maybe @jpelay can chip in?)

It most certainly isn't! It's fairly outdated by this point.

For scenario 2, as you say, it depends on the amount of students who reached level 3! Which we can of course do looking at their programs, or also look at their quiz scores right? Like we do for 'Highest level reached' in the view class

woutccc commented 1 year ago

@Felienne @noelgolding we might want to start off with only the first 2 "dropout groups". With the 2nd one like Felienne stated: "Account created, created one or more class but stopped (-> no activity for 2 weeks in any class before having any student in any class reach level 3)". Ultimately we want to have more insight in a) Teacher interested in Hedy but did not manage to start at all. WHY b) Teacher started with Hedy but did not manage to follow through. WHY

We can adjust the trigger parameters along the way, when/if the survey insights point us into other directions...

woutccc commented 1 year ago

Other question, before I set up the email(s) in Mailchimp: should I translate survey (and email) into Dutch first?

Felienne commented 1 year ago

@Felienne @noelgolding we might want to start off with only the first 2 "dropout groups". With the 2nd one like Felienne stated: "Account created, created one or more class but stopped (-> no activity for 2 weeks in any class before having any student in any class reach level 3)". Ultimately we want to have more insight in a) Teacher interested in Hedy but did not manage to start at all. WHY b) Teacher started with Hedy but did not manage to follow through. WHY

We can adjust the trigger parameters along the way, when/if the survey insights point us into other directions...

Yes lovely idea to just start with a simple scenario! Let's discuss tonight who will pick this up!

Felienne commented 1 year ago

For scenario 2, as you say, it depends on the amount of students who reached level 3! Which we can of course do looking at their programs, or also look at their quiz scores right? Like we do for 'Highest level reached' in the view class

I would say: let's do the simplest thing first: less than x% (50?) have reached level 3. Later we can also take other things into account.

Felienne commented 1 year ago

Other question, before I set up the email(s) in Mailchimp: should I translate survey (and email) into Dutch first?

That would be lovely! We have a large part of Dutch teachers in the database.

Felienne commented 1 year ago

I think it is clear now that @noelgolding will not come back to pick this up, this would fit with the community management tasks of @hasan-sh (but no rush!)