Submission: GROUP_3: Predicting direction of stock price from interest rate and inflation rate

Submitting authors: @allan8392 @carrieyanyi @LilyTao0531 @andyzhangstat Repository: https://github.com/UBC-MDS/stock_price_direction_prediction_from_interest_and_inflation_rate Report link: https://ubc-mds.github.io/stock_price_direction_prediction_from_interest_and_inflation_rate/predicting_direction_of_stock_price_from_inflation_rate_and_interest_rate.html Abstract/executive summary: Inflation and interest rate often takes the headline of financial news and with more than 50% of American households owning stocks, our team is curious to find out how inflation and interest rate affect stock returns. We ask the question: given inflation rate and interest rate data, can we predict whether we will profit if we invest in a stock market index and hold for 1 year.

The data we use including the Standard & Poors 500 Index (S&P500) as stock market proxy and the inflation data obtained from calculating the change of consumer price index (CPI) from the Federal Reserve Economic Data website. Also being included is the target interest rate -- Federal funds rate, which is set by the Federal reserve for commercial banks to lend and borrow overnight.

The model training is designed such that it follows the Golden Rule. The test data of dummy regression yields an accuracy of 75.9%. The accuracy of dummy regression is better then the accuracy of logistic regression. The model's performance may be sensitive to hyperparameter settings, therefore it can be helpful to improve the model performance to experiment with different configurations through hyperparameter tuning.

Editor: @LilyTao0531 Reviewer: @arturoboquin , @riyaeliza123 , @sean-m-mckay , @monazhu

[x] I agree to abide by MDS's Code of Conduct during the review process and in maintaining my package should it be accepted.

Data analysis review checklist

Reviewer: @riyaeliza123

Conflict of interest

[✓] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[✓] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[✓] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[✓] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[ x ] Installation instructions: Is there a clearly stated list of dependencies?
[✓] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[✓] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[✓] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[✓] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[✓] Style guidelides: Does the code adhere to well known language style guides?
[✓] Modularity: Is the code suitably abstracted into scripts and functions?
[✓] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[✓] Data: Is the raw data archived somewhere? Is it accessible?
[✓] Computational methods: Is all the source code required for the data analysis available?
[ x ] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[✓] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[✓] Authors: Does the report include a list of authors with their affiliations?
[✓] What is the question: Do the authors clearly state the research question being asked?
[✓] Importance: Do the authors clearly state the importance for this research question?
[✓] Background: Do the authors provide sufficient background information so that readers can understand the report?
[✓] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[✓] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[✓] Conclusions: Are the conclusions presented by the authors correct?
[ x ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[✓] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing: 3 hours

Review Comments:

Please provide more detailed feedback here on what was done particularly well, and what could be improved. It is especially important to elaborate on items that you were not able to check off in the list above.

Report:

The overall flow of the report is concise and accurate. I appreciate how full context as well as motivation for the project is well written. This gave me enough context to understand the report.
However I would suggest re-visiting the layout of the report. Just focus on making sure that all main headings are properly formatted (right now, everything after EDA is rendered as a sub-topic, let’s avoid that).
I also suggest that rounding the final values (logistic_regression_score 0 0.753086% to 75.3%) will be more appealing to a reader.
Additionally, mention DOI for each paper that is referenced.

READ.ME (instructions):

I am a windows user and the command “docker compose up jupyter-lab” did not work for me. When debugging, I realized there is no “jupyter-lab” in “services” of the docker-compose.yml file. That can be changed in the Read.me to “docker compose up”
The references are not rendered well in the Read.me, try to change the formatting.
Add your dependencies in the Read.me for easy reproducibility.

Attribution

This was derived from the JOSE review checklist and the ROpenSci review checklist.

Data analysis review checklist

Reviewer: @monazhu

Conflict of interest

[x] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[X] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[X] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[X] Installation instructions: Is there a clearly stated list of dependencies?
[X] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[X] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[X] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[X] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[X] Style guidelides: Does the code adhere to well known language style guides?
[X] Modularity: Is the code suitably abstracted into scripts and functions?
[X] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[X] Data: Is the raw data archived somewhere? Is it accessible?
[X] Computational methods: Is all the source code required for the data analysis available?
[X] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[ ] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[X] Authors: Does the report include a list of authors with their affiliations?
[X] What is the question: Do the authors clearly state the research question being asked?
[X] Importance: Do the authors clearly state the importance for this research question?
[ ] Background: Do the authors provide sufficient background information so that readers can understand the report?
[X] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[X] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[X] Conclusions: Are the conclusions presented by the authors correct?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[X] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing: 2 hrs

Review Comments:

Overall comment:

I enjoyed going through the report and the question, and I think it's an interesting question to be posing. However, as a reader, the report left me feeling a bit unsatisfied (there really wasn't great explanation for the conclusion for why the results are the way that they are; the authors posed some possibilities, but did not sufficiently address how or why these potential solutions help to address the issue).

Specific things that could be improved upon:

The overall organization of the folder is mostly clear, but some aspects can be improved (e.g., the image can go into the results folder for better organization)
In addition to having a MIT license (for the software), you might want to specify a license for the report (e.g., a Creative Common license)
In README, under Running the analysis, you might want to specify that you're running the command in the root folder of the project file that was cloned for clarity
the command for running docker in the README didn't work for me (Windows 11) - I needed to enter docker compose up without the jupyter-lab component
I followed the instructions and was able to replicate all the file creation in jupyter lab's terminal, but wasn't able to render the jupyter book the error I got said No such kernel named conda-env-522Group3-py - the project I did was in R so I'm actually not familiar with whether I needed to install and activate the conda environment inside jupyter lab's terminal
In the report, the authors could provide a bit more literature search or context (the link between the introduction and methods is a bit tenuous - for instance, why does this specific dataset help to address the question? This could be spelled out a bit more for the audience)
For a report, I think there is maybe too much space dedicated to EDA (the audience needs to know enough about the methods to draw a conclusion but not so much detail that it becomes overly technical for a report)
There should be a bit more discussion as to why the LR model is actually performing worse than the dummy model in the discussion
The writing is concise and to the point, but sometimes the narrative can jump around a bit (e.g., the data section after the section titled model should really be shown earlier in the method section)
Links/DOIs missing for software citation

Attribution

This was derived from the JOSE review checklist and the ROpenSci review checklist.

Data analysis review checklist

Reviewer: sean-m-mckay

Conflict of interest

[x] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[x] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[x] Installation instructions: Is there a clearly stated list of dependencies?
[x] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[x] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[x] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[x] Style guidelides: Does the code adhere to well known language style guides?
[x] Modularity: Is the code suitably abstracted into scripts and functions?
[x] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[x] Data: Is the raw data archived somewhere? Is it accessible?
[x] Computational methods: Is all the source code required for the data analysis available?
[x] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[x] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[x] Authors: Does the report include a list of authors with their affiliations?
[x] What is the question: Do the authors clearly state the research question being asked?
[x] Importance: Do the authors clearly state the importance for this research question?
[x] Background: Do the authors provide sufficient background information so that readers can understand the report?
[x] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[x] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[x] Conclusions: Are the conclusions presented by the authors correct?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[x] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing:

Review Comments:

Overall:
- Well posed and interesting problem statement that requires combining data from multiple sources - Very well done
- Great job including developer Notes for how to add new packages
Tests: test coverage is a bit low, with no tests for the data cleaning, feature extraction or classification source files. Additionally the tests for EDA only check if the returned plot is the correct type - It might be helpful to check axis labels / other plot details
Extraneous files: There are some files in the repo that could be removed (the notebook under EDA folder, and the duplicate data files under the Raw folder, the train_df and test_df files and the .DS_store file)
Modularization: There is no separation between source and script files - it might be helpful to have a separate folder for scripts which takes arguments and call the functions under src files
EDA: The histogram could benefit from having a label for the target variable, and it would be better to have axis labels for X and Y rather than just the variable names
Reproducability:
- The README mentions to launch the service using docker compose up jupyter-lab but the service name should be t1 not jupyter-lab
- Was able to run all the analysis scripts and reproduce all the output files
- It would be good to have more discussion on why the Dummy classifier has worse accuracy than the Logistic Regression model, and the motivation behind picking the Logistic Regression model - Also would help to explain the motivation behind the preprocessing that was done (e.g. Because the features were all numeric, we applied a StandardScaler to ensure X)
- Some discussion using metrics other than accuracy would be helpful, especially since the report mentions there is a class imbalance (e.g. F1 score, precision, recall)
- Some subheadings could be combined or reordered (e.g. the "Read Data from the Web" and "Raw Data" seem to be referring to the same step, and should probably come before "Derived" and "Analysis" sections)
- The environment yaml and the installed packages in the Dockerfile do not match - It might be helpful to pare down the environment yaml

Attribution

This was derived from the JOSE review checklist and the ROpenSci review checklist.

Data analysis review checklist

Reviewer: @arturoboquin

Conflict of interest

[x] As the reviewer I confirm that I have no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the MDS code of conduct.

General checks

[x] Repository: Is the source code for this data analysis available? Is the repository well organized and easy to navigate?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

Documentation

[ ] Installation instructions: Is there a clearly stated list of dependencies?
[x] Example usage: Do the authors include examples of how to use the software to reproduce the data analysis?
[x] Functionality documentation: Is the core functionality of the data analysis software documented to a satisfactory level?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Code quality

[x] Readability: Are scripts, functions, objects, etc., well named? Is it relatively easy to understand the code?
[x] Style guidelides: Does the code adhere to well known language style guides?
[x] Modularity: Is the code suitably abstracted into scripts and functions?
[x] Tests: Are there automated tests or manual steps described so that the function of the software can be verified? Are they of sufficient quality to ensure software robsutness?

Reproducibility

[x] Data: Is the raw data archived somewhere? Is it accessible?
[x] Computational methods: Is all the source code required for the data analysis available?
[x] Conditions: Is there a record of the necessary conditions (software dependencies) needed to reproduce the analysis? Does there exist an easy way to obtain the computational environment needed to reproduce the analysis?
[ ] Automation: Can someone other than the authors easily reproduce the entire data analysis?

Analysis report

[x] Authors: Does the report include a list of authors with their affiliations?
[x] What is the question: Do the authors clearly state the research question being asked?
[x] Importance: Do the authors clearly state the importance for this research question?
[x] Background: Do the authors provide sufficient background information so that readers can understand the report?
[x] Methods: Do the authors clearly describe and justify the methodology used in the data analysis? Do the authors communicate any assumptions or limitations of their methodologies?
[x] Results: Do the authors clearly communicate their findings through writing, tables and figures?
[x] Conclusions: Are the conclusions presented by the authors correct?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[x] Writing quality: Is the writing of good quality, concise, engaging?

Estimated hours spent reviewing: 3.5 hours

Review Comments:

Project Overview:

The background and historical context provided for the project are both informative and engaging, enhancing its relevance for future readers.
The project demonstrates a comprehensive approach to trying to predict stock price direction based on interest and inflation rates. The repository is well-documented and organized, making it easy to understand and reproduce the analysis. Overall, this project is a well-executed example of data analysis in practice, even if the results are a little discouraging.

Specific Concerns:

In the report, 'Fig. 1 Present the head of the processed data' seems to have rendering issues, making it not align properly with the report dimensions. Adjusting the image size or format might resolve this.
Formatting and Clarity: In the model section, the formatting of 'dummy_model_score 0 0.759259%' and 'dummy_model_score logistic_regression_score 0 0.759259 0.753086%' makes it challenging to read and understand. Clarifying these sections and using standard formatting for percentages would enhance readability.
Audience Understanding: In the discussion section, it would be beneficial to consider the audience's familiarity with terms like 'the Golden Rule.' Perhaps a brief explanation or a reference link could be included for clarity.
Technical Recommendations: I recommend introducing a threshold to ensure that the return on investment surpasses inflation or a standard benchmark, like the U.S. 10 Year Treasury yield.
Readme File and Reproducibility: I noticed some formatting issues in the references section of the readme file. A review and correction of this section would be beneficial.

Technical Issues:

The README could be more user-friendly by specifying the need to be in the project directory in the terminal for successful setup.
Docker Command Issue: I encountered an issue with the 'docker compose up jupyter-lab' command on my Mac, receiving a 'no such service: jupyter-lab' error. Using just 'docker compose' worked for me. Updating the README to reflect this might help future users.

Attribution

This was derived from the JOSE review checklist and the ROpenSci review checklist.

UBC-MDS / data-analysis-review-2023