Peer Review from Group 17

[JuliaJLee]

Opening statement summary

The following is a peer review of this paper on forecasting the 2024 US Presidential election by Group 7.

Strong positive points

The repository is set-up the way it should be, and the paper itself is organized. There is a clear outline of how this paper will come together in the end. As the figures provided in the paper hint at a compelling analysis and model, I believe that this paper will be a strong one!

Critical improvements needed:

• Complete a written first draft of the paper by expanding on the outline that has already been created.
• Make sure to cite all software/R packages (e.g. ggplot2) as in-text citations and in the reference list in addition to R itself.
• Add meaningful descriptions for all commits.
• Complete the R scripts/code for simulation, tests, and downloading the data to the repo. Make sure that the code is appropriately styled (e.g. code using styler).
• Add new sketches that relate to the analysis of this paper to the sketches folder and remove the example sketches (from an in-class reflection exercise) that are in the folder.
• As there is a statement in the README on LLM usage, please make sure to add chat histories to the appropriate folder (e.g. other/llm_usage)
• Add cross-references throughout the paper to any sections, figures, or tables.
• Save the analysis data as a Parquet file.

Suggestions for improvement:

• The title of the paper clearly outlines what is being analyzed and how. It may be helpful to add a brief description of what is found (i.e. what the linear model predicted about the election) as a subtitle.
• The abstract addresses what was done in the analysis, however, it is missing what was found and why the main findings are important (their implications). It may be helpful to add one sentence for each (i.e. one for findings and another for why the findings are important/relevant) before the sentence that says pollster methodology and an idealized survey are included.
• The introduction provides some broader context, some detail about what the paper is about, and a brief outline for the rest of the paper. To this, adding the main findings, their implications, and cross-references to each section of the paper that follows the introduction may be helpful. It may also be helpful to consider expanding on the description of what was done in the analysis (e.g. what variables/relationships did the linear model consider?). This might help readers understand how the main findings were found while reading the introduction.
• There is a section in the introduction allocated for describing the Estimand, so I would just suggest making sure that this section has a sentence that explicitly states what the estimand is.
• Similar to the previous point, it may be helpful to fill in the sections you have created in the Data section while keeping in mind that all variables should be thoroughly explained (e.g. what they represent, whether they relate to other variables, etc.) within the context of the dataset.
• For measurement, it can be helpful to organize this section of the paper and outline how the dataset as a whole, each predictor variable, and response variable were measured for a clear, detailed discussion.
• While you complete the model section of the paper, it may be helpful to write about why certain decisions were made and how those decisions impact the analysis (e.g. potential limitations due to certain decisions). Also, while writing this section, it would be helpful to double check that the variables match those described in the data section and expand on why those variables were considered in the model. As the rubric mentions to provide evidence of model validation and checking, I also suggest adding this as a sub-section within this part of the paper.
• For the results section, a table that summarizes the model is included (Table 1), and it would be helpful to include a description of what the table shows. It would also be helpful to also include graphs of the data to better illustrate the results of the model (e.g. relationships between variables or the predicted values).
• The discussion section can be set-up to provide a summary of the analysis (i.e. what was done and the main findings), a detailed account of the findings' implications, some points on limitations, and future directions.
• As the draft is not yet complete, for prose, I suggest going over the paper - once fully written - and reading it out loud to ensure that it is free of typos or grammatical errors.
• Graphs are included in the paper (size and colour are appropriately used), however, their purpose is a bit unclear. For this, it would be helpful to add titles and captions to make them more self-contained.
• Sections in the Appendix for the "Idealized Methodology and Survey" and "Pollster Methodology" are not included. To complete these parts, it may be helpful to go over the paper instructions and consider other resources (e.g. url links in the original dataset).
• While there is a detailed README, the paper and analysis is currently not reproducible. It would be helpful to ensure that all R scripts and code are organized with comments. I would also suggest removing any unnecessary files (e.g. the literature folder) from the repo.

Evaluation & Estimated Mark:

1/1 point for R Citation 1/1 point for Class Paper 1/1 point for LLM Statement 1/2 points for Title 2/2 points for Author, Date, and Repo 1/4 points for Abstract 2/4 points for Introduction 0/1 point for Estimand 0/10 points for Data 0/4 points for Measurement 0/10 points for Model 0/10 points for Results 0/10 points for Discussion 0/6 points for Prose 0/1 points for Cross-References 0/2 points for Captions 1/4 points for Graphs/Tables/etc 0/10 points for Idealized Methodology 0/4 points for Idealized Survey 0/10 points for Pollster Methodology Overview and Evaluation 1/4 points for Referencing 0/2 points for Commits 0/2 points for Sketches 0/4 points for Simulation 0/4 points for Test-simulation 0/4 points for Test-actual 0/1 point for Parquet 1/4 for Reproducible Workflow 0/1 for Code Style 0/3 for General Excellence

Estimated overall mark: 12/126

Any other comments:

There are comments in the "Critical improvements needed" and "Suggestions for improvement" sections that correspond to the categories in the rubric with partial/no marks. The comments include suggestions that highlight requirements from each section in the rubric.

[Kylie309]

Thank you very much for your long comment Julia! It is very helpful. We have read your comment carefully and expanded the context of the paper in multiple sections. We also revised citations and codings to the better versions, and added sketches, llm, etc.