search

cedadev / padocc

Other
0 stars 0 forks source link

CMIP6 Path Ids #2

Closed dwest77a closed 7 months ago

dwest77a commented 10 months ago

Current progress: Recreated some filesets, need to check if any contain 'archvol' which is not correct

dwest77a commented 9 months ago

Fixed using os.readlink instead of os.path.realpath at 'init' level. CMIP6 datasets reinitialised, must redo computation step. Future datasets no longer at risk

agstephens commented 9 months ago

Hi @dwest77a, just chipping in here. Just to be sure, we want to provide users with the links that have the proper /badc/cmip6/data paths and also the version directories (e.g. v20210515) and NOT the latest version directories (for true traceability). So, the following would be a GOOD path:

/badc/cmip6/data/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/day/tas/gn/v20190624/tas_day_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_19500101-20141230.nc
dwest77a commented 7 months ago

Solved with proper use of python os library to determine 'latest' if specified. If latest version is not the target, this should be specified in the input csv file by using a specific version in the 'pattern' field