geoschem / geos-chem-cloud

Run GEOS-Chem easily on AWS cloud
http://cloud.geos-chem.org
MIT License
39 stars 9 forks source link

[BUG/ISSUE] DUST_DEAD/v2018-04 moved to AFCID/; breaks pre-12.5.0 model #33

Closed JiaweiZhuang closed 4 years ago

JiaweiZhuang commented 4 years ago

In HEMCO_data_directories I notice that

In GEOS-Chem 12.5.0 AFCID was separated from the DEAD dust emissions extension to allow for use with offline dust emissions as well.

Now DUST_DEAD/v2018-04 no longer exist in S3 bucket:

$ aws s3 ls --request-payer=requester s3://gcgrid/HEMCO/DUST_DEAD/
                           PRE v2014-07/
                           PRE v2019-06/

This is fine for 12.5.0+ that does not read from DUST_DEAD/v2018-04, but older version still reads it

On Odyssey, it is linked to ../AFCID/v2018-04

$ cd /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/gcdata/ExtData/HEMCO/DUST_DEAD
$ ls -l v2018-04
lrwxrwxrwx 1 msulprizio jacob_gcst 17 2019-08-12 10:36 v2018-04 -> ../AFCID/v2018-04/

However S3 doesn't support symlinks. We can fix it just like fixing GMI files

DATA_ROOT=$HOME/ExtData
aws s3 cp  --request-payer=requester --recursive s3://gcgrid/HEMCO/AFCID/v2018-04/ $DATA_ROOT/HEMCO/AFCID/v2018-04/

cd $DATA_ROOT/HEMCO/DUST_DEAD
rmdir v2018-04
ln -s ../AFCID/v2018-04 ./

Note that this issue breaks the HEMCO parser script suggested at https://github.com/geoschem/geos-chem-cloud/issues/25#issuecomment-541446027. The generated script will not download from AFCID/v2018-04/as it is not shown in the log file.

yantosca commented 4 years ago

We recommend people update to a more recent version than 12.9.0. I will close out this issue now.