ledm commented 2 years ago

We need to figure out a good user interface. This could be a long issue, so get comfortable. Obviously, this is very much open to discussion and not final in any way.

In BGC-val-public, we provided a single yaml which included all of this for a single run. It was way too much, and hard to set up.

In BGC-val-public, the global settings included mostly global stuff, but also some stuff that was run specific:z

[Global]
jobID       : r1i1p1 Unique run/simulation/job identifiers
model       : HadGEM2-ES ; 
;models          : HadGEM2-ES HadGEM2-CC NorESM1-ME GFDL-ESM2G GFDL-ESM2M CESM1-BGC MPI-ESM-MR MPI-ESM-LR IPSL-CM5A-MR IPSL-CM5B-LR CMCC-CESM  GISS-E2-R-CC GISS-E2-H-CC ; MIROC-ESM MIROC-ESM-CHEM CanESM2
scenarios   : historical ;  The climate scenario
year        : 2000
timerange   : 1750 2101     ; The time range

; -------------------------------
; location to put the global html report
reportdir   : reports/CMIP5/CMIP5-$SCENARIO-$JOBID-chl_debug

; -------------------------------
; Boolean flags
clean       : False ;           ; Boolean flag to make a run from scratch.
makeTS      : True ;            ; Boolean flag to make the time series plots.
makeProfiles    : True ;            ; Boolean flag to make the 3D profile.
makeP2P     : False ;           ; Boolean flag to make the P2P plots.
makeComp    : True ;            ; Boolean flag to make the Comparison of multiple jobs.
makeReport  : True ;            ; Boolean flag to make the global report.
makeCSV     : False ;           ; Boolean flag to make the CSV files.

; -------------------------------
; Base directories  - so the base directory path doesn't need to be repeated every time
basedir_model   : /badc/cmip5/data/cmip5/output1/       ; To replace $BASEDIR_MODEL
basedir_obs : /group_workspaces/jasmin/esmeval/example_data/bgc/        ; To replace $BASEDIR_OBS
basedir_images  : /group_workspaces/jasmin2/ukesm/      ; To replace $BASEDIR_IMAGES
basedir_work    : /group_workspaces/jasmin2/ukesm/      ; To replace $BASEDIR_WORK

; -------------------------------
; Default model files
modelFiles  : $BASEDIR_MODEL/*/$MODEL/$SCENARIO/yr/ocean/Oyr/$JOBID/latest/$NAME/$NAME*_Oyr_$MODEL_*$SCENARIO*$JOBID*.nc
modelFiles_p2p  : $BASEDIR_MODEL/*/$MODEL/$SCENARIO/yr/ocean/Oyr/$JOBID/latest/$NAME/$NAME*_Oyr_$MODEL_*$SCENARIO*$JOBID*.nc

;------------------------------
; Default Output Images folders
images_ts   : $BASEDIR_IMAGES/$USERNAME/CMIP5/images/$MODEL/$SCENARIO-$JOBID/timeseries/$NAME
images_pro      : $BASEDIR_IMAGES/$USERNAME/CMIP5/images/$MODEL/$SCENARIO-$JOBID/profiles/$NAME
images_p2p  : $BASEDIR_IMAGES/$USERNAME/CMIP5/images/$MODEL/$SCENARIO-$JOBID/p2p/$NAME-$YEAR
images_comp : $BASEDIR_IMAGES/$USERNAME/CMIP5/images/comparisons/CMIP5-$MODELS-$SCENARIOS-$JOBIDS/

; -------------------------------
; Default Working directories
postproc_ts : $BASEDIR_WORK/$USERNAME/CMIP5/work/$MODEL/$SCENARIO-$JOBID/timeseries/$NAME
postproc_pro    : $BASEDIR_WORK/$USERNAME/CMIP5/work/$MODEL/$SCENARIO-$JOBID/profiles/$NAME
postproc_p2p    : $BASEDIR_WORK/$USERNAME/CMIP5/work/$MODEL/$SCENARIO-$JOBID/p2p/$NAME-$YEAR
postproc_csv    : $BASEDIR_WORK/$USERNAME/CMIP5/work/csv/CMIP5/CMIP5-$SCENARIOS

; -------------------------------
; These are the default model coordinates (CMIP5 conventions on JASMIN)
model_t     : time          ; model time dimension
model_cal   : auto          ; model calendar
model_z     : lev           ; model depth dimension
model_lat   : lat           ; model latitude dimension
model_lon   : lon           ; model latitude dimension

; -------------------------------
; Default model grid files
modelgrid   : CMIP5-$MODEL-grid
gridFile    : Cmip5Grids/$MODEL_gridfile.nc

; -------------------------------
; The default observational data details. 
; Left empty as observational dataset do not
; usually have standardised coordinates.
data_t      :
data_cal    :
data_z      :
data_lat    :
data_lon    :
data_tdict  :
dataFile    :

Which is a lot, and also several weird bits. And I didn't know how to use yaml anchors at the time, so there's a bit of repetition. But the dynamic paths are pretty handy, and it would be nice to have that in BGCval2.

The timeseries and p2p analyses

In bgc-val public, we provided a yaml section for each variables, but this still had too much detail:

[DIC]
name            : DIC
units           : mmol C/m^3
datasource      : GLODAPv2
dimensions      : 3

; The filenames
modelFiles      : $BASEDIR_MODEL/$JOBID/medusa*_1y_*_ptrc-T.nc
modelFile_p2p   : $BASEDIR_MODEL/$JOBID/medusa*_1y_*$YEAR????_ptrc-T.nc
dataFile        : $BASEDIR_OBS/GLODAPv2/GLODAPv2_Mapped_Climatologies/GLODAPv2.tco2.historic.nc

; Model coordinates/dimension names
model_vars      : DIC
model_convert   : NoChange

; Data coordinates names
data_t          : index_t
data_cal        : standard
data_z          : Pressure
data_lat        : lat
data_lon        : lon
data_vars       : tco2
data_tdict      : ZeroToZero
data_convert    : multiplyBy
data_convert_factor : 1.027
layers          : Surface
regions         : Global

A lot of this could be hidden in a pre-defined yaml, and the user simply requests DIC in their runtime. You may also notice that this is both a timeseries and a p2p analysis together.

Compare report settings

This is where nearly 100% of the BGC-val v1 use case was for the UKESM core group. If we could get a simple interface to make a comparison report, my life would be so much easier.

We'd need it to:

download files on mass-cli1.
run several analysis_timeseries and single model reports
generate a compare report
copy comparison report to public facing website path (on jasmin)

To do this, the user input would need to be the list of details in #8:

keys (kmf, physics, bgc, etc)
line colours, thicknesses and linestyles
descriptions
year differences (this is a function, but it should be changed to a dictionary.)

In an ideal world, I'd want a single yaml to do all of this:

analysis_name: hotsauce
keys:
    level1
jobIDs : 
   u-ab123: 
       description: job number 1
        colour: green
        linestyle: -
        linewidth: 1.2
        year_diff: none
   u-ab122: 
       description: job 1 with extra hot sauce
        colour: red
        linestyle: -
        linewidth: 1.2
        year_diff: +75
   u-ab121: 
       description:  job 1 hold the hot sauce
        colour: blue
        linestyle: -
        linewidth: 1.2
        year_diff: -1850

Anyway, just a way to kick start a discussion, or at least figure something out.

ledm commented 2 years ago

So, there's clearly several things here:

[ ] global settings - being worked on in PR #9.
[ ] straighten out the time series analysis input requirements
[x] Sort out the comparison report interface:
- [x] Step one: change year0 to a dict.
- [x] Step 2: change analysis_compare.py to take an input yaml, instead of hard-wiring everything.

I'm going to skip straight to the comparison report stuff first.

ledm commented 2 years ago

So the analysis_compare part is done now( #24), but I may still change the other bits of the interface for the timeseries and p2p analyses..

valeriupredoi / bgcval2

Thoughts on user interface - especially for comparison reports. #15

The timeseries and p2p analyses

Compare report settings