Closed vkolatschek closed 3 years ago
Hi Vanessa,
The file aero_coarse_1849.nc is created
How should this file be created? Or is it one of the files that are normally in the pool directory, and then it is not copied correctly?
No, you are right. It was not copied correctly from the pool directory. I just compared T127_aeropt_kinne_sw_b14_coa.nc with aero_coarse_1850.nc, and they are the same. So there is just an error in naming the aerocoarse*.nc file?
Likely yes. Does it need to have the year before and after? If yes, have a look at the ozone forcing files, there is something called "additional info, needs before and after" (or similar, I can't remember off the top of my head). It might be that this is missing.
I don't quit know, which files you mean.
(I have three files for aerocoarse*.nc, namely for 1850, 1851 and 1852. But they are needed for 1849, 1850 and 1851. For ozone, I also have three files for the same years, but I think they are the right years.)
OK, so the missing files are then:
I'll reproduce the bug now, maybe then I'll understand what is going wrong
Hi Vanessa,
sorry this took me so long to get to. I am able to reproduce that the aero_1849 file is missing. This is, I guess, because you are starting your run in 1851, and the logic is programmed to need one year before and one year after that. Can you figure out why ECHAM also needs 2 years before? I am a bit confused by that, as this has never been the case previously.
By the way, I am not sure if this is because I have a "read permission" error, or something else, but I get a bunch of missing files when trying to branch off from your run:
========================================================
MISSING FILES:
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/echam//greenhouse_gases.nc:
--> /pool/data/ECHAM6/input/r0007/T127/greenhouse_historical.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/input/echam//unit.23:
--> /pool/data/ECHAM6/input/r0007/T127/T127L47_jan_spec.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_echam.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_echam.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_accw.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_accw.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_co2.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_co2.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3bid.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_g3bid.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3bim.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_g3bim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3bday.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_g3bday.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3b1hi.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_g3b1hi.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_glday.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_glday.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_aclcim.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_aclcim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_sp6h.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_sp6h.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_glim.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_glim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_spim.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/echam//restart_hist1_18501231_spim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/fesom//fesom.1850.oce.diag.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/fesom//fesom.18501231.oce.diag.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/fesom//fesom.1850.oce.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/fesom//fesom.18501231.oce.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/fesom//fesom.1850.ice.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/fesom//fesom.18501231.ice.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/jsbach//landuse_harvest.1851.nc:
--> /pool/data/JSBACH/input/r0010/T127/New_Hampshire_LCC/hist_harvest/LUH_harvest_T127_1851.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/jsbach//landuse_harvest.1850.nc:
--> /pool/data/JSBACH/input/r0010/T127/New_Hampshire_LCC/hist_harvest/LUH_harvest_T127_1850.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/jsbach//cover_fract.1851.nc:
--> /pool/data/AWICM/FESOM1/MESHES/glob//tarfilesT127/input/jsbach/jsbach_T127GLOB_11tiles_1851.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_jsbach.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/jsbach//restart_hist1_18501231000000_jsbach.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_jsbid.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/jsbach//restart_hist1_18501231000000_jsbid.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_yasso.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/jsbach//restart_hist1_18501231000000_yasso.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_surf.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/jsbach//restart_hist1_18501231000000_surf.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_veg.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/jsbach//restart_hist1_18501231000000_veg.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/hdmodel//hdrestart.nc:
--> /work/bk0988/awicm/a270062/AWICM/hist1/restart/hdmodel//restart_hist1_18501231000000_hdrestart.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//greenhouse_gases.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/echam//greenhouse_gases.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//unit.23:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/input/echam//unit.23
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_echam.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_echam.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_accw.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_accw.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_co2.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_co2.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_g3bid.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3bid.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_g3bim.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3bim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_g3bday.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3bday.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_g3b1hi.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_g3b1hi.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_glday.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_glday.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_aclcim.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_aclcim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_sp6h.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_sp6h.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_glim.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_glim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_spim.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/echam//restart_vanessa_000_spim.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//fesom.1850.oce.diag.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/fesom//fesom.1850.oce.diag.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//fesom.1850.oce.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/fesom//fesom.1850.oce.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//fesom.1850.ice.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/fesom//fesom.1850.ice.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//landuse_harvest.1851.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/jsbach//landuse_harvest.1851.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//landuse_harvest.1850.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/jsbach//landuse_harvest.1850.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//cover_fract.1851.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/forcing/jsbach//cover_fract.1851.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_jsbach.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_jsbach.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_jsbid.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_jsbid.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_yasso.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_yasso.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_surf.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_surf.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//restart_vanessa_000_veg.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/jsbach//restart_vanessa_000_veg.nc
-- /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/work//hdrestart.nc:
--> /work/ba1066/a270077/vanessa-bug-213/experiments//vanessa_000/run_18510101-18510131/restart/hdmodel//hdrestart.nc
========================================================
The missing files are probably because of missing files. I had to copy the restart files for echam (and other) to an other directory, because they were in different places than expected. Missing permission is probably the reason there are so many. Only missing files I still get when starting the model are the landuse_* stuff, but the model should not use them anyway. I will try to change permissions.
The problem with the aerosol files:
There are different kind of aerosol concentrations used. In the case of the historical CMIP6 scenario (1850-2014), the aerosol concentration is combined from anthropogenic aerosols modelled in simple plumes together with natural background aerosols of 1850 according to S. Kinne and volcanic aerosols after G. Stenchikov (from echam6 userguide).
The natural background aerosols are split in three files for the years 1849, 1850, and 1851, because the calculations always use three years to calculate an average. This are the files aero_coarse_*
, aero_farir_*
, and aero_fine_*
. Independent on which year in the historical scenario you want to simulate, they should always stay the same three files.
The anthropogenic aerosols (MAC-SP.nc) and volcanic (stratospheric) aerosols (strat_aerosol_sw_1851.nc, strat_aerosol_ir_1851.nc) are added to that. They should change with the simulated year.
Aha, ok, I think the lightbulb just went on for me. So -- in a historic case, you always need 1849-1851 and then on top of that, whatever year you are currently in, plus/minus one. Is that also the case for the RCP scenarios? Might as well fix it all in one go...
In the echam6 userguide, it says that the aerosols of anthropogenic origin is included in aero_fine_*.nc
. That seems to be the case:
From echam.yaml:
RCP26:
greenhouse_dir: "${forcing_dir}"
forcing_files:
aerocoarse: piaerocoarse
aerofarir: piaerofarir
aerofin: rcp26aerofin
ozone: rcp26ozone
greenhouse: rcp26greenh
volcanir: histvolcanir
volcsw: histvolcsw
swflux: histswflux
add_input_files:
MAC-SP: MAC-SP
From echam.datasets.yaml:
"rcp26aerofin":
"${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_fin_1865.nc":
to: 1864
"${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_fin_@YEAR@.nc":
from: 1865
to: 2000
"${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_fin_rcp26_@YEAR@.nc":
from: 2001
to: 2014
you will need to add another class of files. Maybe call it aerofinprehuman
or something. That then needs to point to piaerofin
, and there needs to somewhere be a block in that sets that sets areofinprehuman
to use the files for 1849 to 1851.
I just looked which datasets it takes for histaerofin, and that looks right to me.
"histaerofin":
"${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_fin_1865.nc":
to: 1864
"${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_fin_@YEAR@.nc":
from: 1865
to: 2000
"piaerofin":
"${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_fin_1865.nc"
The userguide states that aero_fine_*
is year dependent, because it also contains some anthropogenic aerosols... Then aero_fine_*
should also change with the year?
I just realized that the error might just be in the naming of the copied files. Where does that happen?
you will need to add another class of files. Maybe call it aerofinprehuman or something. That then needs to point to piaerofin, and there needs to somewhere be a block in that sets that sets areofinprehuman to use the files for 1849 to 1851.
How do I put a block in? That does not happen in echam.yaml or echam.datasets.yaml.
The idea was know to add the files to the run script in a way that they will be added to the work directory. But that is not working. Here is what I added to the run script:
echam:
add_forcing_files:
piaerocoarse1849: piaerocoarse1849
add_forcing_sources:
"piaerocoarse1849": "${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_coa.nc"
add_forcing_in_work:
piaerocoarse1849: "aero_coarse_1849.nc"
Do I need to add something else?
When I added this
add_forcing_files:
piaerocoarse1849: piaerocoarse1849
add_forcing_sources:
"piaerocoarse1849": "${forcing_dir}/aero/${resolution}_aeropt_kinne_sw_b14_coa.nc"
add_forcing_in_work:
piaerocoarse1849: "aero_coarse_1849.nc"
to my run script, it did not copy the year 1849, but after sorting the lines into echam.yaml and echam.datasets.yaml, it now copies the files aero_coarse_1849.nc into the work directory. I do not think, it is an ideal solution, because I have to do it for each year separately (so, three times), but for now it seems to work.
Describe the problem I am using AWI-CM (CMIP6) to branch of an existing historical scenario run of the CMIP6 simulations. I start in 1851, hist starts in 1850. The optical properties of coarse mode aerosols are extracted form T127_aeropt_kinne_sw_b14_coa.nc. This creates the files aero_coarse_1850.nc, aero_coarse_1851.nc, and aero_coarse_1852.nc.
But because these are aerosols of natural origin, the model needs the "background" aerosols for the years 1849, 1850 and 1851. The file is therefore aero_coarse_1849.nc missing.
To Reproduce Steps to reproduce the behavior:
My runscript (awicm-mistral_hist1_trial.yaml):
Expected behavior The file aero_coarse_1849.nc is created and the aerosol properties are read from the file.
System (please complete the following information):