One big notebook

[kaedonkers]

Aim:

One notebook to recreate the analysis and plots in the paper draft. Keep an eye on how big it gets and split out into different notebooks (or Quarto notebook) if gets too big (new issue).

Linked to PR #10

TODO:

Whole notebook

• [x] Hyperlink sections in header text
• [x] Move comprehensive notebook to Met Office repo
• [x] Jets CSV into data dir
• [ ] Test notebook in Binder -> R Kernel broken in Binder in general
• [x] Correct MJO name
• [x] Verify results of notebook with paper results

Section 1

• [x] Regional temperature data
• [x] Climate index data
• [x] Jet data
• [x] Absolute temperature data (for Section 6)
• [x] DePreSys data

Section 2

• [x] Critical correlation threshold

Section 3

• [x] Results from Boruta feature selection
• [x] Significant predictors selected by Boruta
• [x] Top 10 strongest predictors
• [x] Selected predictors in each region
• [x] Feature selection of jets

Section 4

• [x] Create Bayesian network
• [x] Learn structure of BN
• [x] Learn strength of links
• [x] Fit BN
• [x] Visualise BN
• [ ] Figure 7: Climate index cross correlation matrix
• [ ] Figure 8: Predictors vs temperature PCA
• [ ] Linear Mixed Effects model

Section 6

• [x] Build linear regression model to predict provincial temperatures
• [x] Bootstrap linear model
• [x] Bootstrap DePreSys
• [x] Plot ERA5 vs linear model vs DePreSys
  • [x] Plot absolute temperature rather than anomaly
• [x] LOOCV
• [ ] 5-fold CV of linear model
• [x] 5-fold CV of Bayesian network (china_indices_corr_v2.R#1744:1750)

[kaedonkers]

@nathan962 Please leave comments here as you go so we can work on this asynchronously 👍

[kaedonkers]

All sections in this notebook are mostly complete and working. However, Section 5 (Dynamics plots) hasn't been done, and the results I am getting by running this notebook don't match those in Nathan's latest notebook or Ed's code (or the paper for that matter), so this notebook on GitHub will need to be verified against results we expect