CarmenTamayo
commented
8 months ago - • I think it’s necessary to including links to external resources when concepts are mentioned but not explained, and also to include them in the glossary, e.g., ODE, C++, contact matrix, model parameters (what each of them represent and how to interpret them), SEIR, ODE solver, etc.
- • I like that there is a disclaimer of the pre-requisites of the episodes, but the phrasing "concept dependencies" might be confusing, are these concepts that the reader is expected to already be familiar with? Or concepts that the reader should familiarise themselves with before/during the study of the episode? I would clarify this
- • Overall, the headings that say "generating trajectories" I think should be replaced with something that is more understandable by readers, such as "simulating disease spread", something that is more indicative and related to the title of the episode.
- • "Models can be used to evaluate the implementation of non-pharmaceutical and pharmaceutical interventions while accounting for factors such as age." <- I don't understand the purpose of this sentence being added here in the context of this episode. Maybe mention how models can be used to simulate transmission instead.
- • Building on the last point, I think there should be an introduction of the reasoning behind why one might want to simulate transmission, with some examples so that the content is relevant to learners. This is the kind of information that would be very valuable and promote the usability of these tutorials. There are many resources out there to learn modelling of infectious diseases, but I think we can offer the context of why and when, which is often what students don't fully grasp, especially in the beginning of the learning process. 5.1.- "By the end of this tutorial, you will be able to generate the trajectory below showing the number of infectious individuals in different age categories through time" <-A learner could ask "so what?" How is this useful for the different learning profiles that are targeted with these tutorials?
- • "In this tutorial, we will use the R package {epidemics}" add link to the package when mentioning it so people can learn more about it if they're interested.
- • "The first step is to install the R packages epidemics." <- this is placed in the introduction as the first step that the students should follow, but straight after there is another theorical section about model structures, and the package is not actually used until the "model parameters" section- if there is no special reason for this, I would move the step 1 to be in the model parameter section with the subsequent practical steps. Unless the point is for students to play around with the package and check out the different models in the section "model structures", in which case I would explicitly say so to encourage readers to do so.
- • "In this tutorial, we will use the default epidemic model, model_default_cpp() which is described in the next section" <- I'd add where this model comes from
- • "There is a library of models to choose from in epidemics. " <- I feel there's a disconnect between the package and the contents of the tutorial. There should be a demo here of how to select models from the library in epidemics, and how to check their parameters, etc. The tutorial proceeds to explain this model and its parameters in detail below, but how can the user actually check these characteristics when choosing this model (or a different one) from the library? Is this explained in another tutorial? If so, it should be cross-referenced here.
- • Similarly to the point above, there is a section on checking model assumptions, that I'm not really sure refers in general or when using {epidemics}. For the former, if this is not within the scope of the tutorials, I would add some reference to a resource that describes how to correctly check model assumptions. Someone new to modelling won't know how to do this and, while it's good to remind them that doing so is important, they might be frustrated that this is not explained. Concepts such as indirect transmission are referenced and included in the glossary, which are quite basic concepts that readers will more likely be familiar with than with concepts such as model assumptions and how to check them. "There can be subtle differences in model structures for the same infection or outbreak type which can be missed without studying the equations." here, it's important to add a sentence that highlights what the implications of this would be, and how this is relevant for the reader in answering their research questions. Conversely, if the point of this section is to teach readers how to check model equations using {epidemics}, the info is missing on how one might do so when using the package.
- • "We want to generate disease trajectories of an influenza strain with pandemic potential. " this is slightly differently phrased from the aim described in the introduction. I think there should be an "aim of this episode's analysis" section at the beginning that should be cohesive throughout the episode.
- • "an age-structured SEIR model described by a system of ordinary differential equations" <- this is a very important point as it defined the model that will be used, however there is no further explanation of why this model is used, or references for the concepts of SEIR or ordinary differential equations.
- • When describing the model parameters and equations, I think that either the equations are explained to some degree, or that they should be removed. This tutorial seems to be targeted to learners that have little to no knowledge of ID modelling, e.g., starts by explaining what "S" in "SEIR" means, so if the tutorial goes from that explanation to a series of equations and mentions of parameters, that leap is probably too big for new learners. This section should at least cover what each equation represents, how they relate to the parameters and, most importantly, spell out how they are relevant to the purpose of simulating transmission.
- • "from the model structure, we see that the contact matrix allows for heterogeneity in contacts and that there is no loss of immunity"- how can the reader see this? Could this be explained better? And include why/how this is relevant to the exercise conducted in the tutorial? The same for the sentence "This model also has the functionality to include vaccination and tracks the number of vaccinated individuals through time. We will cover the use of interventions in future tutorials.", which I believe should probably be removed.
- • "To generate trajectories using our model, we need the following :" <- I feel that this should be highlighted more, why do we need these and how are they related to the previously defined model? Also the wording "generate trajectories" could be changed to something like "simulate disease spread" to ease readers into the materials. Building on this point, each of these elements that are needed should have a subsequent dedicated section, which headings at the same level, right now some of them do, but some of them are subheadings of other sections, and I also think the same wording should be used in the heading as in this section where they are being introduced, otherwise readers might be lost when following the steps. 15.1. - "We will learn how to specify the contact matrix 𝐶 in the next section." <- this wouldn't be necessary if the above point was fixed.
- • The contact matrix section is well explained and has references, the same format should be followed for the rest of the episode (e.g., when introducing the model used). 16.1. - "The result is a square matrix with rows and columns for each age group." - if this is included here, the introductory section can be simplified as, at the moment, the content about the contact matrix is pretty heavy and condensed, especially compared to other elements included in the same section that are barely explained. 16.2. - "Contact matrices can be loaded from other sources, but they must be in the correct format to be used in epidemics." <- If this is mentioned, then it's important to either say what the format actually is, or to add a link to a vignette in {epidemics} or to a resource that explains this format and how one might change the format of a contact matrix to fit into the expected format.
- • "We have prepared our parameter values, contact matrix and demography vector" <- this is the first time in the episode that a demography vector is mentioned. Readers will be confused as to when they prepared this and why they would need it. It's also not included in the list of requirements specified in the previous section (it says demographic structure- it'd be important to clarify what the vector represents and how it's related to the demographic structure of the population). Further, the vector is not created until 2 sections below, in "running the model", so this should be removed from here.
- • "Now we must set the initial conditions, prepare the population and run the model." <- I believe the tutorials will really benefit from a schematic of the steps followed to achieve the episode's aim, as well as what elements are required to achieve each step. Right now it's a bit messy to figure out what the steps to follow are (would also be a good idea to add numbers perhaps) and how to separate them from the theory bits in the content of the episode.
- • For the initial conditions, it's easy to get lost between the point where the proportion of SEIR are and how these are related to the different age categories. I would split the code chunk that starts with "# build for all age groups" into 2, and add a heading about how the second command is used to assign the specified proportions to the selected age groups.
- • The explanation about time increments is good, however I would include links to further reading as it is not fully clear how readers can use this argument just by reading this section.
- • The "accounting for uncertainty" section has a clear structure with numbered steps to follow, that should be applied throughout the episode. This section would also benefit from links to further reading. 21.1. - This is the only time that uncertainty is mentioned in the episode, but it appears as a main component of it. This should be added to the objectives or questions outlined at the beginning of the episode.
- • "In this tutorial, we have learnt how to generate disease trajectories using a mathematical model" - again I believe some other wording relating to the simulation of transmission would be better (see 3rd point).
- • Lastly, I think the key points could be improved, for instance the last key point just says "Include uncertainty in model trajectories"- what is the key point? Is it that it is important to include uncertainty? Or that the tutorial demonstrates how to do so? Also "contact matrices can be estimated from different sources" doesn't sound like a very strong key point for learners.
amanda-minter