search

geoschem / geos-chem-cloud

Run GEOS-Chem easily on AWS cloud
http://cloud.geos-chem.org
MIT License
39 stars 9 forks source link

[BUG/ISSUE] Some data files in the S3 Input Repository could be corrupted. #41

Closed neyranaz closed 1 year ago

neyranaz commented 4 years ago

GC 12.7.0 run in the Tutorial AMI.

We already solved this issue but wanted to bring it to your attention. There are some files in the S3 gcgrid input repository that perhaps got corrupted when uploaded to the S3. For instance we were getting an error when GC was trying to pull offline_dust for an Aerosol Only run. We then found that this error was due to a specific file that was damaged.

/gcgrid/data/ExtData/HEMCO/OFFLINE_DUST/v2019-01/0.25x0.3125/2014/02/dust_emissions_025.20140202.nc Which weighted 173 Kb in the gcgrid input repository.

The same file in Compute Canada weighted 2.6MBs /ExtData/HEMCO/OFFLINE_DUST/v2019-01/0.25x0.3125/2014/02/dust_emissions_025.20140202.nc

We were able to get the run going by deleting the file that was failing, and by downloading the missing file using wget from Compute Canada. But it is important to mention that we weren't getting this error two weeks ago, so we believe there might have been a change in the S3://gcgrid input repository.

This error was a solution to 

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
HEMCO ERROR: Wrong dimensions: $ROOT/OFFLINE_DUST/v2019-01/0.25x0.3125/$YYYY/$MM/dust_emissions_025.$YYYY$MM$DD.nc
ERROR LOCATION: HCOIO_READ_STD (hcoio_read_std_mod.F90)
ERROR LOCATION: HCOIO_DataRead (hcoio_dataread_mod.F90)
ERROR LOCATION: ReadList_Fill (hco_readlist_mod.F90)
ERROR LOCATION: ReadList_Read (hco_readlist_mod.F90)
ERROR LOCATION: HCO_RUN (hco_driver_mod.F90)
yantosca commented 1 year ago

Closing this issue.