Submission: GROUP 10: CadioPredict

Submitting authors: @sandygross Sandy Gross @MDSFusionist Doris Wang @hema2022ubc He Ma @joeywwwu Joey Wu

Repository: https://github.com/UBC-MDS/CardioPredict Report link: https://ubc-mds.github.io/CardioPredict/heart_analysis_report.html Abstract/executive summary: Cardiovascular disease (CVD) remains a leading cause of mortality globally, necessitating the development of accurate predictive tools for early detection and intervention. This study utilizes a practice dataset from the renowned Framingham Heart Study (FHS) comprising clinical, demographic, and behavioral variables from patients at risk of CVD. The study employed a methodological approach centered on hyperparameter optimization of the k-Nearest Neighbors (kNN) algorithm, supplemented by an oversampling technique to address class imbalances and improve model sensitivity. Despite modest levels of accuracy (0.623) and recall (0.552), our model underscores the significance of cholesterol levels and smoking habits as substantial contributors to cardiovascular disease risk, alongside established factors such as age and systolic blood pressure. These insights pave the way for future investigations into the complex interplay of causal factors, intending to refine the predictive accuracy and clinical utility of the model.

Editor: @MDSFusionist Doris Wang Reviewer: <@carrieyanyi Yan Carris> <@Rachel0619 Rachel LI> <@shawnhu444 Shawn Hu> <@sungg888 Ruocong Sun>

[x] I agree to abide by MDS's Code of Conduct during the review process and in maintaining my package should it be accepted.

Data analysis review checklist

Reviewer:

Conflict of interest

[ ✓] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[✓ ] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[✓ ] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[ ✓] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[ ✓] Installation instructions: Is there a clearly stated list of dependencies?
[✓ ] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[ ✓] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[✓ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[✓ ] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[ ✓] Style guidelides: Does the code adhere to well known language style guides?
[✓ ] Modularity: Is the code suitably abstracted into scripts and functions?
[ ✓] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[✓ ] Data: Is the raw data archived somewhere? Is it accessible?
[✓ ] Computational methods: Is all the source code required for the data analysis available?
[✓ ] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[ ✓] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[ ✓] Authors: Does the report include a list of authors with their affiliations?
[✓ ] What is the question: Do the authors clearly state the research question being asked?
[✓ ] Importance: Do the authors clearly state the importance for this research question?
[✓ ] Background: Do the authors provide sufficient background information so that readers can understand the report?
[ ✓] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[ ✓] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[✓ ] Conclusions: Are the conclusions presented by the authors correct?
[✓ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[✓ ] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing:

Review Comments:

This project exemplifies a highly standardized approach to data analysis and software development, aligning with best practices in several key areas:

High code quality, functionality documentation and follow the community guidelines. These aspects are vital for maintaining and scaling the software efficiently, ensuring its long-term viability.
The code submitted ensures data accessibility by providing a complete computational methods and putting details, functions. These features significantly enhance the reproducibility and reliability of the research, which are cornerstones of scientific rigor.
Reporting: The comprehensive approach to reporting, including clear articulation of research questions, background, functions, and coupled with high-quality writing and complete referencing .This demonstrates an exemplary standard in code communication.

In summary, this project stands out for comprehensive documentation, code quality, reproducibility, and thorough analysis reporting.

Data analysis review checklist

Reviewer: sungg888

Conflict of interest

[✓] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[✓] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[✓] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[✓] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[✓] Installation instructions: Is there a clearly stated list of dependencies?
[✓] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[✓] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[✓] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[✓] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[✓] Style guidelides: Does the code adhere to well known language style guides?
[✓] Modularity: Is the code suitably abstracted into scripts and functions?
[✓] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[✓ ] Data: Is the raw data archived somewhere? Is it accessible?
[✓] Computational methods: Is all the source code required for the data analysis available?
[✓ ] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[ ✓] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[ ✓] Authors: Does the report include a list of authors with their affiliations?
[✓] What is the question: Do the authors clearly state the research question being asked?
[✓] Importance: Do the authors clearly state the importance for this research question?
[✓] Background: Do the authors provide sufficient background information so that readers can understand the report?
[✓] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[✓] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[✓] Conclusions: Are the conclusions presented by the authors correct?
[✓] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[✓] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing:

1.5 hours

Review Comments:

Documentation: The project is good with its detailed documentation, easy installation, and usage examples. The inclusion of community guidelines significantly aids collaboration and user support.
Code Quality: The code is well-organized, readable, and adheres to style guidelines, enhancing its maintainability. The project also have clear and readable robust testing framework to test its reliability.
Reproducibility: The project excels in reproducibility, providing accessible raw data, comprehensive computational methods, and clear steps in Readme.
Analysis Report: The analysis report is clear and detailed, effectively communicating the research question, methods, and results. The high-quality writing makes the report engaging and informative. The report covers all points in the rubric. Overall, the project is a high-quality, reproducible research. And it meet the standards of review checklist.
Attribution

This was derived from the JOSE review checklist and the ROpenSci review checklist.

Data analysis review checklist

Reviewer: @Rachel0619

Conflict of interest

[x] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[x] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[x] Installation instructions: Is there a clearly stated list of dependencies?
[x] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[x] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[x] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[x] Style guidelides: Does the code adhere to well known language style guides?
[x] Modularity: Is the code suitably abstracted into scripts and functions?
[x] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[x] Data: Is the raw data archived somewhere? Is it accessible?
[x] Computational methods: Is all the source code required for the data analysis available?
[x] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[x] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[x] Authors: Does the report include a list of authors with their affiliations?
[x] What is the question: Do the authors clearly state the research question being asked?
[x] Importance: Do the authors clearly state the importance for this research question?
[x] Background: Do the authors provide sufficient background information so that readers can understand the report?
[x] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[x] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[x] Conclusions: Are the conclusions presented by the authors correct?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[x] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing: 1 hour

Review Comments:

Overall, the project is conducted in high-quality and I'm deeply impressed by your hard work.

I really appreciate the plotting part of the report. The figures are well made and organized.
Low recall might be an issue in the context of early detection and intervention a disease (if this is the research question of this project). You might want to try some other classification models to see if you can get higher recall if you have time and really want to make it perfect. But what it is now is good enough.
The src file is a little bit messy now with different type of files in it.

Attribution

This was derived from the JOSE review checklist and the ROpenSci review checklist.

Data analysis review checklist

Reviewer: @carrieyanyi

Conflict of interest

[x] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[x] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[x] Installation instructions: Is there a clearly stated list of dependencies?
[x] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[x] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[x] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[x] Style guidelides: Does the code adhere to well known language style guides?
[x] Modularity: Is the code suitably abstracted into scripts and functions?
[x] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[x] Data: Is the raw data archived somewhere? Is it accessible?
[ ] Computational methods: Is all the source code required for the data analysis available?
[x] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[x] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[x] Authors: Does the report include a list of authors with their affiliations?
[x] What is the question: Do the authors clearly state the research question being asked?
[x] Importance: Do the authors clearly state the importance for this research question?
[x] Background: Do the authors provide sufficient background information so that readers can understand the report?
[x] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[x] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[x] Conclusions: Are the conclusions presented by the authors correct?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[x] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing:

2hrs

Review Comments:

Please provide more detailed feedback here on what was done particularly well, and what could be improved. It is especially important to elaborate on items that you were not able to check off in the list above.

I really enjoy going through this project and gain some knowledge about Cardiovascular Disease. The overall structure of the project is well organized. Both figures and models have sufficient descriptions.

Installation: Some issues were encountered when i runned the script. EDA.py: "ValueError: Saving charts in 'png' format requires the vl-convert-python or altair_saver package: see http://github.com/altair-viz/altair_saver/"
Reference: In the Data section, the citation for Tsao201 is mis-spelling, it should be Tsao2015 instead.
Visualization: It would be more obvious to add legend explanation in figure 5 to different what does disease 0 and 1 mean.
Abbreviation: A lot of terminology are used in the report, it would be great if maintaining consistency in the use of abbreviations, rather than alternating with their full names.

Attribution

This was derived from the JOSE review checklist and the ROpenSci review checklist.

UBC-MDS / data-analysis-review-2023