search

awslabs / open-data-registry

A registry of publicly available datasets on AWS
https://registry.opendata.aws
Apache License 2.0
1.4k stars 894 forks source link

GEFS v12 AWS missing data #1773

Open lzlarryli opened 1 year ago

lzlarryli commented 1 year ago

The following files seem to be missing:

s3://noaa-gefs-pds/gefs.20210211/00/atmos/pgrb2bp5/gec00.t00z.pgrb2b.0p50.f000
s3://noaa-gefs-pds/gefs.20220226/00/atmos/pgrb2bp5/gep18.t00z.pgrb2b.0p50.f426
s3://noaa-gefs-pds/gefs.20220226/00/atmos/pgrb2bp5/gep20.t00z.pgrb2b.0p50.f444
s3://noaa-gefs-pds/gefs.20220226/00/atmos/pgrb2bp5/gep24.t00z.pgrb2b.0p50.f474
s3://noaa-gefs-pds/gefs.20220505/00/atmos/pgrb2bp5/gep20.t00z.pgrb2b.0p50.f408
s3://noaa-gefs-pds/gefs.20220505/00/atmos/pgrb2bp5/gep23.t00z.pgrb2b.0p50.f408
s3://noaa-gefs-pds/gefs.20220722/00/atmos/pgrb2bp5/gep09.t00z.pgrb2b.0p50.f402
s3://noaa-gefs-pds/gefs.20220722/00/atmos/pgrb2bp5/gep10.t00z.pgrb2b.0p50.f396
s3://noaa-gefs-pds/gefs.20220722/00/atmos/pgrb2bp5/gep15.t00z.pgrb2b.0p50.f402

Is there a way to recover these? Thanks.

lzlarryli commented 1 year ago

In additional to the missing files above, I found some further issues in the existing files.

These 2 files are in the bucket but are of size 0.

s3://noaa-gefs-pds/gefs.20201112/00/atmos/pgrb2bp5/gep03.t00z.pgrb2b.0p50.f588
s3://noaa-gefs-pds/gefs.20201008/00/atmos/pgrb2bp5/gep05.t00z.pgrb2b.0p50.f492

These 3 files have at least one variable missing.

s3://noaa-gefs-pds/gefs.20211004/00/atmos/pgrb2ap5/gep15.t00z.pgrb2a.0p50.f588 t2m
s3://noaa-gefs-pds/gefs.20220720/00/atmos/pgrb2bp5/gep11.t00z.pgrb2b.0p50.f210 d2m
s3://noaa-gefs-pds/gefs.20201112/00/atmos/pgrb2bp5/gep08.t00z.pgrb2b.0p50.f612 d2m
Patrick-Keown commented 1 year ago

Thank you for pointing this out. We will look into this and your original note.