Before developing and modifying this workshop, please read through the presenter and developer protocol and refer back to it regularly as you work.
Summary of presenter and participant feedback (2020-2021)
This workshop received positive feedback last year. Participants felt that the workshop's material was clear and thorough and that they were able to understand how LMMs and GLMMs work theoretically and how to implement them in R. Participants appreciated the challenges and use of breakout rooms in order to test their knowledge of the material. However, we received feedback that this workshop is material-heavy, especially in the first half, and often feels rushed at the end. Presenters also requested more presenter notes to make their job easier in preparing to teach this workshop.
Therefore, this workshop would benefit from further development to ensure that slides are made concise when possible, this includes the editing of unclear sections and the removal of unnecessary material (if any). Currently there are 9 challenges in this workshop, including a mix of long and short form challenges. In order to ensure that this workshop does not run longer than 4 hours including breaks, challenges should be reviewed to ensure time is being used effectively. In addition, presenter notes should be added throughout (This is a major priority!) These proposed corrections are outlined in more detail below.
General issues to be addressed related to the presentation of this workshop
Facilitate the presentation of the slides with presenter notes
Presenter notes are particularly helpful to guide presenters through slides that have been developed by many QCBS contributors over several years. For example, you may add notes to direct presenters to the key message of a complex slide, to remind presenters to spend more (or less) time on a difficult concept, to offer additional information or explanations, to propose check-in questions to seek more active engagement of the participants, to remind presenters to call back to a previous concept in an earlier workshop, and more. To add presenter notes, you can add ??? at the bottom of the slide, and write your presenter note underneath as such:
# Slide title: Coding in R is fun!
R allows us to clean, analyze, and plot our data!
???
Take the time to explain that R allows us to program tasks, so we do not have to manually repeat them.
To engage the participants, ask them if they think R is fun too.
[x] Annotate slides with presenter notes, to help presenters navigate the previously developed workshop material more fluidly. Note: There may already be additional notes within the Workshop 7 Book that could help you with this task.
Adapt exercises to better suit the remote workshop format
Exercises should be informative and engaging, and should build on the material shown during the workshop. Here are some suggestions to achieve this goal:
[x] Review the quantity and structure of exercises. Use exercises efficiently, to avoid overcrowding a workshop with exercises that do not actively build on the concepts and tools shown in the workshop. We generally recommend implementing up to a maximum of 2 longer-form exercises in breakout rooms, to allow ample time to thoroughly go over challenge solutions. Of course, there may be exceptions, and we ultimately leave this decision to our contributors. When adapting or construction exercises, think about the time management required to run an effective exercise in a workshop setting. Please remember to leave time for students to work through their solutions and ask questions, and consider the additional time it takes to use breakout rooms.
[x] Ensure that breakout rooms are used effectively. If breakout rooms are used, provide or add prompts to stimulate collaboration and/or discussion between participants, such as questions to discuss about the challenge or specific elements to compare in the solutions developed by participants in the room. You may remove the recommendation for breakout room use if you do not find them relevant, or propose breakout rooms if they would be beneficial.
[x] Construct or edit exercises to promote active learning. Refer to these guidelines to find suggestions of exercises promoting active learning. You may use additional resources to create the exercises.
Ensure that all changes are reflected in both languages
[x] All changes to workshop should be conducted in both French and English. Final workshops should be identical with neither version lagging behind the other in terms of development and editing.
Specific to slide content, structure, grammar and style
In general, these slides are quite text-heavy. Some of the text can be moved into the presenter notes in order to make the slides more concise without losing important material. Below we've compiled edits that should be made to the slides, but developers may also edit them further to flow better/cut down on text as they see fit.
[x] Add a presenter note at the beginning of the workshop which reminds presenters to communicate to participants that all of the material for these workshops is available on our website, where there is also a book with additional text not on the slides for those who want to work through the material again on their own. Presenters can therefore go a bit faster and know that participants can revisit these concepts later if they miss anything.
[x] Slide 22-25. The concept of random and fixed effects could be more clearly explained here (perhaps by reordering these slides). Presenter notes should be added.
[x] Slide 26. Missing punctuation in text should be fixed. Should separate each sentence into a new line.
[x] Slide 30. Bracketed text should be translated.
[x] Slide 40. The line par(mfrow = c(1, 2), mar = c(4, 4, 1, 1)) should be hidden if possible, as it is not relevant to the participants.
[x] Slide 41. Text is missing a word.
[x] Slide 52-53. Text should read 'into R code'.
[x] Slide 54. Missing 's' on underestimateS.
[x] Slide 55. This slide should be reviewed and edited for clarity.
[x] Slide 57. This slide should be reviewed and edited for clarity (plus presenter notes).
[ ] The entire 'Model selection' section could use enhanced clarity regarding varying intercepts vs. slopes.
[x] Challenge 5, add presenter notes to this challenge to enhance efficiency. Could call on participants to explain their solution(s).
[x] Slide 76. par() line should be hidden from slide.
[x] Slides 76-78. These slides could be reordered/edited to flow better with presenter notes.
[x] Slide 100. Text here should be edited (LMM is not a separate workshop).
[x] GLMMs starting at slide 100. This section should be edited for flow and presenter notes should be added.
[x] Slide 101. Explain the dataset more thoroughly/clearly. Add more info on slides or in presenter notes.
[x] Slides 118-120 should be reviewed and edited for clarity.
[x] Slide 126. Maybe split into two slides (1 with model creation, 1 with model selection methods)
Remaining issues to resolve from previous contributors:
[x] The badges inside the presentation linking to the PDF slides and to the script do not work.
[x] A small refresher slide about overdispersion could be useful before slide 104 as this concept was defined in the previous version of the GLM workshop.
[ ] A dedicated workshop on LMMs was held at UQAM a couple of years ago (https://wiki.qcbs.ca/lmm.workshop). If possible, material from that workshop could be included into this one. Or at least mention and include the link to the other workshop as a reference in case someone wants to increase their knowledge on this topic (see issue #4 ) .
Before developing and modifying this workshop, please read through the presenter and developer protocol and refer back to it regularly as you work.
Summary of presenter and participant feedback (2020-2021)
This workshop received positive feedback last year. Participants felt that the workshop's material was clear and thorough and that they were able to understand how LMMs and GLMMs work theoretically and how to implement them in R. Participants appreciated the challenges and use of breakout rooms in order to test their knowledge of the material. However, we received feedback that this workshop is material-heavy, especially in the first half, and often feels rushed at the end. Presenters also requested more presenter notes to make their job easier in preparing to teach this workshop.
Therefore, this workshop would benefit from further development to ensure that slides are made concise when possible, this includes the editing of unclear sections and the removal of unnecessary material (if any). Currently there are 9 challenges in this workshop, including a mix of long and short form challenges. In order to ensure that this workshop does not run longer than 4 hours including breaks, challenges should be reviewed to ensure time is being used effectively. In addition, presenter notes should be added throughout (This is a major priority!) These proposed corrections are outlined in more detail below.
General issues to be addressed related to the presentation of this workshop
Facilitate the presentation of the slides with presenter notes
Presenter notes are particularly helpful to guide presenters through slides that have been developed by many QCBS contributors over several years. For example, you may add notes to direct presenters to the key message of a complex slide, to remind presenters to spend more (or less) time on a difficult concept, to offer additional information or explanations, to propose check-in questions to seek more active engagement of the participants, to remind presenters to call back to a previous concept in an earlier workshop, and more. To add presenter notes, you can add
???
at the bottom of the slide, and write your presenter note underneath as such:Adapt exercises to better suit the remote workshop format
Exercises should be informative and engaging, and should build on the material shown during the workshop. Here are some suggestions to achieve this goal:
[x] Review the quantity and structure of exercises. Use exercises efficiently, to avoid overcrowding a workshop with exercises that do not actively build on the concepts and tools shown in the workshop. We generally recommend implementing up to a maximum of 2 longer-form exercises in breakout rooms, to allow ample time to thoroughly go over challenge solutions. Of course, there may be exceptions, and we ultimately leave this decision to our contributors. When adapting or construction exercises, think about the time management required to run an effective exercise in a workshop setting. Please remember to leave time for students to work through their solutions and ask questions, and consider the additional time it takes to use breakout rooms.
[x] Ensure that breakout rooms are used effectively. If breakout rooms are used, provide or add prompts to stimulate collaboration and/or discussion between participants, such as questions to discuss about the challenge or specific elements to compare in the solutions developed by participants in the room. You may remove the recommendation for breakout room use if you do not find them relevant, or propose breakout rooms if they would be beneficial.
[x] Construct or edit exercises to promote active learning. Refer to these guidelines to find suggestions of exercises promoting active learning. You may use additional resources to create the exercises.
Ensure that all changes are reflected in both languages
Specific to slide content, structure, grammar and style
In general, these slides are quite text-heavy. Some of the text can be moved into the presenter notes in order to make the slides more concise without losing important material. Below we've compiled edits that should be made to the slides, but developers may also edit them further to flow better/cut down on text as they see fit.
[x] Add a presenter note at the beginning of the workshop which reminds presenters to communicate to participants that all of the material for these workshops is available on our website, where there is also a book with additional text not on the slides for those who want to work through the material again on their own. Presenters can therefore go a bit faster and know that participants can revisit these concepts later if they miss anything.
[x] Slide 22-25. The concept of random and fixed effects could be more clearly explained here (perhaps by reordering these slides). Presenter notes should be added.
[x] Slide 26. Missing punctuation in text should be fixed. Should separate each sentence into a new line.
[x] Slide 30. Bracketed text should be translated.
[x] Slide 40. The line
par(mfrow = c(1, 2), mar = c(4, 4, 1, 1))
should be hidden if possible, as it is not relevant to the participants.[x] Slide 41. Text is missing a word.
[x] Slide 52-53. Text should read 'into R code'.
[x] Slide 54. Missing 's' on underestimateS.
[x] Slide 55. This slide should be reviewed and edited for clarity.
[x] Slide 57. This slide should be reviewed and edited for clarity (plus presenter notes).
[ ] The entire 'Model selection' section could use enhanced clarity regarding varying intercepts vs. slopes.
[x] Challenge 5, add presenter notes to this challenge to enhance efficiency. Could call on participants to explain their solution(s).
[x] Slide 76.
par()
line should be hidden from slide.[x] Slides 76-78. These slides could be reordered/edited to flow better with presenter notes.
[x] Slide 100. Text here should be edited (LMM is not a separate workshop).
[x] GLMMs starting at slide 100. This section should be edited for flow and presenter notes should be added.
[x] Slide 101. Explain the dataset more thoroughly/clearly. Add more info on slides or in presenter notes.
[x] Slides 118-120 should be reviewed and edited for clarity.
[x] Slide 126. Maybe split into two slides (1 with model creation, 1 with model selection methods)
Remaining issues to resolve from previous contributors:
[x] The badges inside the presentation linking to the PDF slides and to the script do not work.
[x] A small refresher slide about overdispersion could be useful before slide 104 as this concept was defined in the previous version of the GLM workshop.
[ ] A dedicated workshop on LMMs was held at UQAM a couple of years ago (https://wiki.qcbs.ca/lmm.workshop). If possible, material from that workshop could be included into this one. Or at least mention and include the link to the other workshop as a reference in case someone wants to increase their knowledge on this topic (see issue #4 ) .