Draft Survey for Student Feedback on Content

njlyon0 commented 1 month ago

Summary

The design team met and discussed the benefits of surveying students balanced against the risk of surveying them too much / losing quality of responses. We decided that we could survey at the end of each phase (~4 modules & ~ 2 months) and likely balance the risks and benefits of either approach. For a full set of that discussion see the 10//08/24 notes in the running notes doc for 2024

Sub-Tasks

[x] Draft the 'post-phase I' survey
- Include the following topics (at least):
- Quality of content (i.e., clarity, organization)
- Quality of instruction (i.e., interactivity, pacing)
- Quality of panels (if applicable)
[x] Share with the design team & gather feedback
[x] Send the survey to the students!
Further sub-tasks TBD

Resources

Survey content is in the Shared Drive in the top-level assessment folder

njlyon0 commented 1 month ago

First Draft Complete

Created a draft of the survey (see assessment > 02_during-course > 2024_phase-1-survey in the Shared Drive) that includes--I think--the main points we wanted to include. Need to share this across the design team to solicit feedback

njlyon0 commented 1 month ago

Emailed Design Team

Sent my draft via email to the design team and invited them to either email feedback to me or post it in this issue. We'll try to send out the survey ASAP after the team science module so that everyone's memory is still sharp

ckappel commented 1 month ago

It might be helpful in the intro blurb to tell them that the survey asks for their input on three parts: content, instruction, and panels, so that they save their feedback for the appropriate section.
I like how the instructors and panels sections have a header - can you add one for the Content section, too?
Please add a Q for the panel for the Team Sciences Practices session in section 3

scelmendorf commented 1 month ago

I am in favor of short surveys (so make only the really fast things required, rest optional) and for only asking for actionable content. To that end I would:

label all the free-form fields as optional and
remove the 1-5 ratings on individual instructors on the logic that you probably don't have a roster of backup instructors you can sub in.
if removing 1-5 on instructors, you could add these types of hints to the free-form anwers, e.g. "how can xxx improve their instructional approach for future cohorts?(e.g. communication, organization, speaking pace, etc.) Feel free to also use this space to comment on any elements you thought this instructor did really well!"

As always - these are just suggestions based on my personal tendencies which may not be representative. If a survey has a lot of required long form sections only some of which I have a strong opinion on, I usually ignore it.

gremau commented 1 month ago

Looks very good, but I agree with Sarah that the instruction section is perhaps more fine-grained than will be useful. It could be condensed in the way she suggests so that individual instructors can zero in and address feedback specifically directed at them. Plus, rankings with a range of 5 categories for speaking pace or organization may not be as interpretable as just counting up the number of responses that said "slow down" or provided organizational suggestions. Carrie's suggestion for a little outline in the introduction is also a good idea.

The third question is on ranking the interactivity, but also includes a sub-question on "was it useful to you?". Ranking interactivity gets at how engaging/effective the material is, but I think having students reflect on the usefulness of the material might warrant a separate question.

njlyon0 commented 1 month ago

Thanks all for the feedback! I've made the following changes:

Identified that there are three sections in the intro blurb
Added a section break so the first batch of questions have the same formatting as the 2nd/3rd
Added the ranking / free form questions for the team science practices panel
Labeled all questions as optional
Removed the 1-5 ranking of instructors question
Simplified the speaking pace question for instructors to only have 'too slow', 'just right', and 'too fast' as the only options
- I figure this will make counting up / balancing responses easier (to Greg's point)
Made module usefulness and interactivity questions into separate questions

Is there any other feedback or are we good to send this around to students?

If it is looking good, it might be nice to share it during the Team Science Practices module Zoom so that it is top-of-mind and we get good feedback quickly (if we need to revise more though that absolutely takes precedent)

ckappel commented 1 month ago

Looks great to me. Thanks, Nick!

On Thu, Oct 17, 2024 at 7:54 AM Nick J Lyon @.***> wrote:

Thanks all for the feedback! I've made the following changes:

Identified that there are three sections in the intro blurb

Added a section break so the first batch of questions have the same formatting as the 2nd/3rd

Added the ranking / free form questions for the team science practices panel

Labeled all questions as optional

Removed the 1-5 ranking of instructors question

Simplified the speaking pace question for instructors to only have 'too slow', 'just right', and 'too fast' as options

I figure this will make counting up / balancing responses easier (to Greg's point)

Made module usefulness and interactivity questions into separate questions

Is there any other feedback or are we good to send this around to students?

If it is looking good, it might be nice to share it during the Team Science Practices module Zoom so that it is top-of-mind and we get good feedback quickly (if we need to revise more though that absolutely takes precedent)

— Reply to this email directly, view it on GitHub https://github.com/lter/ssecr/issues/34#issuecomment-2419775337, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACTZXSGQJV3UGFRKQXXOR4DZ37FRRAVCNFSM6AAAAABPTEI7ZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJZG43TKMZTG4 . You are receiving this because you commented.Message ID: @.***>

--

Carrie Kappel, PhD

Independent Consultant, Facilitator, Research Scientist

Preferred Pronouns: she / her / hers

@.***

831.869.1503 m

njlyon0 commented 1 month ago

Added example/description text to all questions about module content (thanks for the rec Li!)

njlyon0 commented 1 month ago

Survey Sent

Survey shared with students so closing this issue. May open a new one for tweaks to the survey for the next batch of modules (a.k.a. "phase")

lter / ssecr