yantosca
commented
4 years ago Thanks for writing. I agree, the nested-grid documentation is somewhat out of date. We are going to be rewriting our documentation for the GEOS-Chem 13.0.0 release, which will happen later this year. Also in 13.0.0, the run directory generation will happen from the source code directory instead of from the unit tester (which will be retired).
In the meantime, I will try to answer your questions the best I can:
Creating the Boundary Conditions in geosfp_4x5_aerosol
1.- Creating the boundary conditions: First we create a Directory from /UT/perl/CopyRunDirs.Input for Offline Aerosol _(or should we create the BCs by it activating geosfp standard?
The global simulation from which you will save out the boundary conditions for the nested run has to be the same type of simulation as the nested run. In this case, you'll want to set up a geosfp_4x5_aerosol run directory and use that to save out the boundary conditions. You should start the global run at least 1 day before the start of the nested run.
_ by activating GEOSFP Aerosol only 4x5 aerosol (or is 2x2.5 better?), for the date in which we plan to generate our boundary conditions. This date has to be one day before the date of our nested run.
You could also set up a geosfp_2x25_aerosol simulation for the boundary conditions but that will of course take longer to run and the boundary condition files will be larger. So I think the 4x5 is fine for the purpose of boundary conditions.
As noted, you should start the global run to save out BC's at least one day before the start of the nested run.
2.-Once the directory is generated, we go there, and modify HISTORY.rc, there we turn on 'Restart', 'SpeciesConc', and 'BoundaryConditions' (or should we turn on/off something else?),
If you are only saving out boundary conditions then you can omit saving out the SpeciesConc diagnostic. That will save disk space and time.
then we modify BoundaryConditions.frequency: 00000000 030000 to 1 hour, so that we can address the hourly issue specified by @msulprizio.
That is correct.
4.- Then in input.geos we modify the Latitude and Longitude to the area of interest, for example, lat and lon for california, (_or is it for North America or is it globally, (leaving it as it was)_). Here, should we turn Nested grid simulation on, or should we leave it off? Also, If we want to run, a Nested Grid for California, should we run the BC for a global area (-180 180; -90 90) for NA (-140 -40; 10 70), or can we do it for a Custom Area (Lets say, western US) ? Do nested and global dimensions have to agree, or can we do global for North America and nested for California?).
For the global simulation you will need these specifications:
%%% GRID MENU %%% :
Grid resolution : 4.0x5.0
Longitude min/max : -180.0 180.0
Latitude min/max : -90.0 90.0
Half-sized polar boxes?: T
Number of levels : 72
Nested grid simulation? : F
Buffer zone (N S E W ) : 0 0 0 05.- Then compile, using
make realclean ; make -j4 build NC_DIAG=y BPCH_DIAG=n TIMERS=1, (Or is it something else, as @FeiYao was doing for version 12.2?).
This should work. But you can also compile using CMake](http://wiki.seas.harvard.edu/geos-chem/index.php/Compiling_with_CMake), as described here:
5.1.- Download data, using the dry run, and execute ./geos to get the boundary conditions in OutputDir
Correct.
Creating the nested grid directory in geosfp_4x5_aerosol_na
6.- Once we have created the BC, we move again to /UT/perl/CopyRunDirs.Input and create a new directory by activating GEOSFP Aerosol only 4x5 aerosol and modifying it to nested as @FeiYao-Edinburgh did (or would it be advisable to modify it from the tropchem nested grid?)
- -We modify the generated directory (either geosfp_4x5_aerosol or geosfp_025x03125_tropchemna). How do we do this? What are we supposed to change to make this happen?_ . This is one of the most important questions, but it was not entirely clear how Fei-Yao approached it here
8.-Once the geosfp_4x5_aerosol_na directory is set up, and we modify everything to make it run (How does input.geos look?). Can we select the lat and longitud of our interest (P.ex to Los Angeles).
Right now, the GEOS-Chem Unit Tester does not have a nested-aerosol option. I think the quick thing to do would be to create a new geosfp_4x5_aerosol run directory (the same way as before) and rename it. Then copy the boundary condition files from the 4x5 run directory to the 025x03125 run directory:
cp -R geosfp_4x5_aerosol geosfp_025x3125_aerosol_na
cd geosfp_025x3125_aerosol_na
mkdir BC_4x5
cp ../ geosfp_4x5_aerosol/OutputDir/GEOSChem.BoundaryConditions* ./BC_4x5/ALSO NOTE: You might need to concatenate boundary condition files together (you can do this with nco ncrcat) so that each file has 24 time points.
Settings for geosfp_025x03125_aerosol_na/input.geos
`%%% GRID MENU %%% :
Grid resolution : 025x03125
Longitude min/max : 114.0 119.0
Latitude min/max : 38.0 42.0
Half-sized polar boxes?: F
Number of levels : 72
Nested grid simulation? : T
Buffer zone (N S E W ) : 3 3 3 3You may want to go a little bit further in E/W longitude and N/S latitude to make sure that the area of interest does not fall within the buffer zone.
Settings for geosfp_025x03125_aerosol_na/HISTORY.rc
ALSO VERY IMPORTANT!!! In the geosfp_025x3125_aerosol_na/HISTORY.rc file, make sure you turn off the BoundaryConditions collection.
At a minimum you'll want Restart and SpeciesConc. Any other diagnostics are up to you.
COLLECTIONS: 'Restart',
'SpeciesConc',
#'Budget',
#'AerosolMass',
#'Aerosols',
#'CloudConvFlux',
#'DryDep',
#'LevelEdgeDiags',
#'ProdLoss',
#'StateChm',
#'StateMet',
#'WetLossConv',
#'WetLossLS',
#'Transport',
#'BoundaryConditions',Settings for geosfp_025x03125_aerosol_na/HEMCO_Config.rc
9.- We modify HEMCO_config.rc, just as we did in step 3, to be able to download the data, and we also modify the Metdir, to be able to read _NA data.
METDIR: /project/data/ExtData/GEOS_0.5x0.625_NA/GEOSFPin the same file, we also modify the path to our boundary conditions just as @msulprizio Explained (For 12.7 solved for later versions).
That's correct. Since my instructions above have the boundary condition files in BC_4x5, then this is the path to use:
`#==============================================================================
# --- GEOS-Chem boundary condition file ---
#==============================================================================
(((GC_BCs
* BC_ ./BC_4x5/GEOSChem.BoundaryConditions.$YYYY$MM$DD_$HH$MNz.nc4 SpeciesBC_?ADV? 1980-2019/1-12/1-31/* RFY xyz 1 * - 1 1 (Note that we substituted 1-23 for *)
)))GC_BCsThat is all correct. The HEMCO_Config.rc file should now point to the BC_4x5 subdirectory (where we copied the boundary conditions above).
Also from the wiki
`# ExtNr ExtName on/off Species 0 Base : on * # ----- RESTART FIELDS ---------------------- --> GC_RESTART : true --> GC_BCs : true --> HEMCO_RESTART : true `These are the correct settings.
10.- _Do we modify HISTORY.rc t_o ask for boundary conditions?
No, this is only necessary from the global simulation.
11.- We compile using this (Or is it something else? @FeiYao-Edinburgh)
make realclean make -j4 build NC_DIAG=y BPCH_DIAG=n TIMERS=1`12.- We download files from dry run, and modify some file extensions manually like ".NA.nc" to ".nc" in ExtData to be able to pull them
I'm not sure you need to rename files. Try it first.
13.- We then run ./geos, and then get our .nc file in Output Dir?
Yes, that's it!
Also tagging @msulprizio who has done a lot of work with FlexGrid and the nested simulations.
Hope this helps!!!
neyranaz
Model version: 12.7.0 AMI Name: GEOSChem_12.7.0_tutorial_20200205 Instance type: c5.18xlarge (To prevent the EC2 from being killed or running out of memory)
Hello, we've been trying to set up an Aerosol Only Nested Run for North America with a special focus on western U.S., but we haven't been able to identify the complete steps to accomplish this. We've decided to use GC12.7 because of this issue. Also, we've read in depth the following links: Nested Grid Wiki, Nested Model Clinic, Flex Grid Simulation Error, 3 hour boundary conditions for nested grid, and @FeiYao-Edinburgh steps to run a nested simulation for Complex SOA. We have also troubleshooted with the preconfigured nested grid (tropchem), to identify some errors, but we haven't succeeded at running a nested grid for the specialty of our interest (Aerosol). Because of this, we would like to have your input on the steps to tackle this.
Here is our understanding of the way to process it, along with some questions (In italics) associated to it, :
I don't want to confuse anyone so please take these steps as our notion of what might have to be done rather than what should actually be done.
Creating the Boundary Conditions in geosfp_4x5_aerosol
1.- Creating the boundary conditions: First we create a Directory from /UT/perl/CopyRunDirs.Input for Offline Aerosol (or should we create the BCs by it activating geosfp standard?) by activating GEOSFP Aerosol only 4x5 aerosol (or is 2x2.5 better?), for the date in which we plan to generate our boundary conditions. This date has to be one day before the date of our nested run.
2.-Once the directory is generated, we go there, and modify HISTORY.rc, there we turn on 'Restart', 'SpeciesConc', and 'BoundaryConditions' (or should we turn on/off something else?), then we modify BoundaryConditions.frequency: 00000000 030000 to 1 hour, so that we can address the hourly issue specified by @msulprizio.
3.-In the HEMCO_config.rc file, we modify the paths to the data as it was [explained] (https://github.com/geoschem/geos-chem-cloud/issues/39#issuecomment-665073840) by Bob, so that we are able to download them once we do the dryrun as explained by @yantosca.
4.- Then in input.geos we modify the Latitude and Longitude to the area of interest, for example, lat and lon for california, (_or is it for North America or is it globally, (leaving it as it was)_). Here, should we turn Nested grid simulation on, or should we leave it off? Also, If we want to run, a Nested Grid for California, should we run the BC for a global area (-180 180; -90 90) for NA (-140 -40; 10 70), or can we do it for a Custom Area (Lets say, western US) ? Do nested and global dimensions have to agree, or can we do global for North America and nested for California?).
5.- Then compile, using
make realclean ; make -j4 build NC_DIAG=y BPCH_DIAG=n TIMERS=1, (Or is it something else, as @FeiYao was doing for version 12.2?).5.1.- Download data, using the dry run, and execute ./geos to get the boundary conditions in OutputDir
Creating the nested grid directory in geosfp_4x5_aerosol_na
6.- Once we have created the BC, we move again to /UT/perl/CopyRunDirs.Input and create a new directory by activating GEOSFP Aerosol only 4x5 aerosol and modifying it to nested as @FeiYao-Edinburgh did (or would it be advisable to modify it from the tropchem nested grid?)
We want:
7.-We modify the generated directory (either geosfp_4x5_aerosol or geosfp_025x03125_tropchemna). How do we do this? What are we supposed to change to make this happen?_ . This is one of the most important questions, but it was not entirely clear how Fei-Yao approached it here
8.-Once the geosfp_4x5_aerosol_na directory is set up, and we modify everything to make it run (How does input.geos look?). Can we select the lat and longitud of our interest (P.ex to Los Angeles).
9.- We modify HEMCO_config.rc, just as we did in step 3, to be able to download the data, and we also modify the Metdir, to be able to read _NA data.
METDIR: /project/data/ExtData/GEOS_0.5x0.625_NA/GEOSFPin the same file, we also modify the path to our boundary conditions just as @msulprizio Explained (For 12.7 solved for later versions).
Also from the wiki
10.- _Do we modify HISTORY.rc t_o ask for boundary conditions? Or we don't really need to do this__. If we need to, should we also change BoundaryConditions.frequency: 00000000 030000 to 1 hour, just as we did in step 2 or is it not really necessary? Also, in HISTORY.rc we will request 'Restart', as output. (This might bring an error later on).
11.- We compile using this (Or is it something else? @FeiYao-Edinburgh)
make realclean make -j4 build NC_DIAG=y BPCH_DIAG=n TIMERS=112.- We download files from dry run, and modify some file extensions manually like ".NA.nc" to ".nc" in ExtData to be able to pull them
13.- We then run ./geos, and then get our .nc file in Output Dir?
I should be clearly missing many steps, or this might be entirely wrong so your input would be very valuable
Once we have some input on how to approach this, we will be sharing with you a Public ami, to troubleshoot some of the errors that we expect to see, also we will share the following files (once we make them): GC_log.txt, HEMCO.log.txt, input.geos.txt, HISTORY.rc.txt, HEMCO_Config.rc.txt