ph-kev
commented
1 month ago Would it be possible to split the
leaderboard.jlfile in such a way that if one wants to add a new variable they would have to only change one file? (Essentially, moving the loaders to a separate file)
I moved all the code involving data and preprocessing to data_sources.jl.
closes #948 , closes #956
This PR refactors the leaderboard code using the new features from ClimaAnalysis. This commit also deletes the old leaderboard code and the tests for it. Also, there is an off by one month issue when handling the dates and seasons. This issue arises because the first day of each month in the simulation data represents the monthly average of the previous month. This issue only affect the plots involving seasonal data. This is fixed in this commit.
The commit also moves the code to leaderboard.jl and a line is added to the pipeline to run the script.
One significant difference is the leaderboard which now plot the best and worst single model using only annual rather than averaging the error over the annual and seasonal data.
The leaderboard now looks like this:
To compare, the previous leaderboard looks like this:
The errors for the seasonal data change slightly because of the off by one month issue. The new bias plot for MAM looks like this:
The old bias plot for MAM looks this:
In addition to the plots above, there is now a plot with all the seasons for a subset of the variables (see
compare_vars_biases_groupsfor which variables are plotted together in data_sources.jl).