search

NOAA-EMC / godas

7 stars 4 forks source link

Archive Xiao's 30 member ensemble experiment #473

Open guillaumevernieres opened 1 year ago

guillaumevernieres commented 1 year ago

@ShastriPaturi , we need to archive @liuxiao37k 's experiment (without removing it from the Gaea fs) on HPSS.

ShastriPaturi commented 1 year ago

@guillaumevernieres I will have @hyunchul386 do it.

liuxiao37k commented 1 year ago

The path is here: gaea: /lustre/f2/dev/ncep/Xiao.Liu/sandbox/09aug2022/cpld_BGCv2/ens2019010600***

Please note that there are other experiments located under the same directory, but only the ones named ens2019010600_ ( = 001 to 030, n=30) are from this experiment.

Most of the 30 ensemble members were completed for 17 months from 01/01/2019 to 05/30/2020, but some were terminated a bit earlier in 05/2020. We have the full set at least for 16 months from Jan 2019 to Apr 2020.

The total size for /rst files is about 60T so I recommend we slim them by removing the 3D variables that won't be used (e.g. the ocean BGC ones) if archiving them becomes an issue. The /bkg files are much smaller (except the obgc.bkg.nc files because I saved more digits for BGC bkg variables), less than 1T, but @guillaumevernieres suggested that we would need more digits than what we saved for the ocn and ice* variables under /bkg.

Let me know if anyone has questions.

ShastriPaturi commented 1 year ago

@hyunchul386 please let me know if and when can you work on this? If yes, a timeline is needed.

liuxiao37k commented 1 year ago

Please let me know if I can be of further assistance in addressing this issue beside the information provided above. Thanks!

ShastriPaturi commented 1 year ago

@liuxiao37k you are good

hyunchul386 commented 1 year ago

I have looked at and let you know if there is question for that. Thanks

liuxiao37k commented 1 year ago

@hyunchul386 I thought you posted a question here earlier today (I received a GitHub notification) but could not locate that post. Please ignore the /static/bump directory for these forecasts. They were not used. I linked the /static directory only to skip the prep.soca step.

hyunchul386 commented 1 year ago

@liuxiao37k yes it was about the bump.

hyunchul386 commented 1 year ago

Archive is going on and the archive directory is /5year/NCEPDEV/marineda/Ensem_xiao/. If the status of RDTN keeps working as usual, it is expected that the archiving process will be completed in the next week.

hyunchul386 commented 1 year ago

The data are archived at HPSS /5year/NCEPDEV/marineda/Ensem_xiao/: ens2019010600_001/ ens2019010600_007/ ens2019010600_013/ ens2019010600_019/ ens2019010600_025/ ens2019010600_002/ ens2019010600_008/ ens2019010600_014/ ens2019010600_020/ ens2019010600_026/ ens2019010600_003/ ens2019010600_009/ ens2019010600_015/ ens2019010600_021/ ens2019010600_027/ ens2019010600_004/ ens2019010600_010/ ens2019010600_016/ ens2019010600_022/ ens2019010600_028/ ens2019010600_005/ ens2019010600_011/ ens2019010600_017/ ens2019010600_023/ ens2019010600_029/ ens2019010600_006/ ens2019010600_012/ ens2019010600_018/ ens2019010600_024/ ens2019010600_030/