avaldebe commented 3 years ago

OpenSource rv4.36 (202011)

@metno/emep The countdown to the release is ticking, please leave a comment if we need to pause and redo some of the following tasks

[x] update source code
[x] new tag rv4_36
[x] release draft
[x] reference runs
[x] update catalog
[x] update docs
[x] close pending issues: #69 #62
[x] add CONTRIBUTORS.md, see https://github.com/metno/emep-mscw/issues/193#issuecomment-718460477
~~add CHANGELOG.md, see https://github.com/metno/emep-mscw/issues/193#issuecomment-718460477~~
[x] update README.md

mifads commented 3 years ago

clock is ticking, but when is the zero-hour?

avaldebe commented 3 years ago

clock is ticking, but when is the zero-hour?

when all tasks are done

avaldebe commented 3 years ago

git commit messages is do not contain enough info for a meaningful changelog. Skip for this release...

avaldebe commented 3 years ago

@metno/emep

The source code and reference runs are ready for the next release. All that is missing is the release itself. You can see the draft text at: https://github.com/metno/emep-ctm/releases

Let me know if there is any reason to wait with the release.

Cheers, Á

mifads commented 3 years ago

Just had a (very) quick look and it looks fine, though I would change rv4_36 to rv4.36. The git describe also uses the . notation rather than the underscore.

avaldebe commented 3 years ago

Just had a (very) quick look and it looks fine, though I would change rv4_36 to rv4.36.

All other tags on this repo use underscores.

# nebula:~sm_alvva/emep-ctm/source
git tag -l -n

rv3             OpenSource rv3 (200802)
rv4_0           OpenSource rv4.0 (201209)
rv4_10          add README
rv4_10_eEMEP_ASH eEMEP ASH version
rv4_15          update from research repo
rv4_17          OpenSource rv4.17 (2018029)
rv4_3           OpenSource rv4.0 (201209)
rv4_32          OpenSource rv4.32 (201904)
rv4_33          OpenSource rv4.33 (201906)
rv4_34          OpenSource rv4.34 (202001)
rv4_36          OpenSource rv4.36 (202011)
rv4_4           OpenSource rv4.4 (201309)
rv4_5           OpenSource rv4.5 (201409)
rv4_8           OpenSource rv4.8 (201510)
ug4_10          user guide for rv4_10
ug4_15          rv4_15 user guide
ug4_17          rv4_17 user guide
ug4_33          rv4_33 user guide
v201106         OpenSource v.2011-06 (201108)

The git describe also uses the . notation rather than the underscore.

Git describe will use any tag (with _ or '.') as long as the tag was created with a message.

# nebula:~sm_alvva/emep-ctm/source
git describe

rv4_36

avaldebe commented 3 years ago

@mifads

I'll sit on the release until tomorrow morning, just incase.

mifads commented 3 years ago

I can't remember why we started with _ and not ., but all other software uses ., and that is also github's recommendation (x.y.z style for version numbering). Why not use it? This ain't very important, but I'd like us to move to more standardised notation. It is also much faster to say rv4 dot 36 than rv4 underscore 36 ;-)

mifads commented 3 years ago

Hi @avaldebe, cc @mifapw Just by coincidence, I tried to use emissplits on nebula from

/nobackup/forsk/sm_petwi/Data/ZCM_DIRS/ZCM_EmChem19/emissplits_snap/CAMS-REG-AP_v2.2.1_2015_REF2/emissplit_specials_nox.csv

and the model crashes. Not surprisingly, since that file contains garbage from around line 497 onwards! very odd that nobody has hit this problem, but this is likely due to everyone using gnfr and not snap recently. I will edit this file, but you should to if part of the OS.

avaldebe commented 3 years ago

/nobackup/forsk/sm_petwi/Data/ZCM_DIRS/ZCM_EmChem19/emissplits_snap/CAMS-REG-AP_v2.2.1_2015_REF2/emissplit_specials_nox.csv
and the model crashes. Not surprisingly, since that file contains garbage from around line 497 onwards! very odd that nobody has hit this problem, but this is likely due to everyone using gnfr and not snap recently. I will edit this file, but you should to if part of the OS.

This is not part of the the release, we are not releasing new emissions.

In case we want to keep 2015 as emission year, your original link is the correct one. We did not get recalculated emissions for previous years this year, so that one is still the most up-to-date emission data for 2015.

Originally posted by @agnesny in https://github.com/metno/emep-mscw/issues/192#issuecomment-717114325

I forgot to list the emission splits on metno/emep-mscw#192. I updated the list, here are the relevant entries.

/nobackup/forsk/sm_petwi/Data/emis01deg_EMEP/GNFREmis_EMEP01_2015_22072019.nc
/nobackup/forsk/sm_petwi/Data/ZCM_EmChem19/emissplit_run/emissplit.defaults.*
/nobackup/forsk/sm_petwi/Data/ZCM_EmChem19/emissplit_run/emissplit.specials.*

The splits and emissions are the same as on the last release (rv4.34 202001).

mifads commented 3 years ago

Hi @avaldebe That setup mixes GNFR emissions and SNAP emissplits! I don't know how that worked last time around, but these days emep-dev uses the new style .csv files, which would here be from

 pwd/nobackup/forsk/sm_petwi/Data/ZCM_DIRS/ZCM_EmChem19/emissplits_gnfr/CAMS-REG-AP_v2.2.1_2015_REF1

avaldebe commented 3 years ago

I can't remember why we started with _ and not ., but all other software uses ., and that is also github's recommendation (x.y.z style for version numbering). Why not use it?

This ain't very important, but I'd like us to move to more standardised notation. It is also much faster to say rv4 dot 36 than rv4 underscore 36 ;-)

Thing is x.y.z style versioning (semantic versioning, or semver) has a meaning and best practices associated with. Will try to to find a good article about this, but in short x.y.z means major.minor.patch:

patch: bug fixes
minor: non breaking changes, such as new features and deprecate features (warn about features that will be removed in the next major release)
major: breaking changes, such as remove features and change default options

We do not follow this style of development, so we should not use semver.

If we want to use a more modern style of versioning, I suggest ubuntu style versioning (calendar versioning, or calver): YY.MM.patch. With this stile the next release would be 21.02 and 21.02.1 if we release a bugfix.

EDIT Here is a good article about versioning best practices in python https://emptysqua.re/blog/api-evolution-the-right-way/

avaldebe commented 3 years ago

Hi @mifads , @gitpeterwind, @agnesny and @hildefagerli

I'm pausing the release until we figure this one out.

I forgot to list the emission splits on metno/emep-mscw#192. I updated the list, here are the relevant entries.
/nobackup/forsk/sm_petwi/Data/emis01deg_EMEP/GNFREmis_EMEP01_2015_22072019.nc
/nobackup/forsk/sm_petwi/Data/ZCM_EmChem19/emissplit_run/emissplit.defaults.*
/nobackup/forsk/sm_petwi/Data/ZCM_EmChem19/emissplit_run/emissplit.specials.*
That setup mixes GNFR emissions and SNAP emissplits! I don't know how that worked last time around, but these days emep-dev uses the new style .csv files, which would here be from
 pwd/nobackup/forsk/sm_petwi/Data/ZCM_DIRS/ZCM_EmChem19/emissplits_gnfr/CAMS-REG-AP_v2.2.1_2015_REF1

The reference runs went trough without any warnings on this respect. Here are the relevant parts of the logfile, nebula:/nobackup/forsk/sm_alvva/EMEP_MSC-W_model.rv4.36.OpenSource/Base/emepctm.out:

[....]
 Emission source number            1 from
 Emissions:DataDir/GNFRemis_EMEP01_2015.nc
original emission name DataDir/GNFRemis_EMEP01_2015.nc
filename redefined as: ../input/GNFRemis_EMEP01_2015.nc
 Switching sector categories to GNFR
[...]
Splitting sox emissions into SO2 SO4 using ../input/ZCM_EmChem19/emissplit.defaults.sox
Splitting sox emissions into SO2 SO4 using ../input/ZCM_EmChem19/emissplit.specials.sox
Splitting nox emissions into NO NO2 shipNOx using ../input/ZCM_EmChem19/emissplit.defaults.nox
Splitting nox emissions into NO NO2 shipNOx using ../input/ZCM_EmChem19/emissplit.specials.nox
Splitting co emissions into CO using ../input/ZCM_EmChem19/emissplit.defaults.co
Splitting co emissions into CO using ../input/ZCM_EmChem19/emissplit.specials.co
Splitting voc emissions into NC4H10 OXYL C2H4 C3H6 HCHO MEK TOLUENE CH3OH C2H6 CH3CHO C2H5OH
BENZENE C5H8 APINENE GLYOX MGLYOX UNREAC using ../input/ZCM_EmChem19/emissplit.defaults.voc
 emis_split: no specials for:voc
Splitting nh3 emissions into NH3 using ../input/ZCM_EmChem19/emissplit.defaults.nh3
Splitting nh3 emissions into NH3 using ../input/ZCM_EmChem19/emissplit.specials.nh3
Splitting pm25 emissions into POM_f_ffuel EC_f_ffuel_new EC_f_ffuel_age POM_f_wood EC_f_wood_new EC_f_wood_age remPPM25 using ../input/ZCM_EmChem19/emissplit.defaults.pm25
Splitting pm25 emissions into POM_f_ffuel EC_f_ffuel_new EC_f_ffuel_age POM_f_wood EC_f_wood_new EC_f_wood_age remPPM25 using ../input/ZCM_EmChem19/emissplit.specials.pm25
Splitting pmco emissions into POM_c_ffuel EC_c_ffuel EC_c_wood remPPM_c using ../input/ZCM_EmChem19/emissplit.defaults.pmco
Splitting pmco emissions into POM_c_ffuel EC_c_ffuel EC_c_wood remPPM_c using ../input/ZCM_EmChem19/emissplit.specials.pmco
Emissions: reading emis_inputlist    1../input/GNFRemis_EMEP01_2015.nc
#EMTAB Total emissions by countries for ../input/GNFRemis_EMEP01_2015.nc (Gg)
[....]

We have been providing this combo since rv4.15 (201709). In my optionion it should continue to work or stop the simulation with a clear error message.

avaldebe commented 3 years ago

let's move this discussion to metno/emep-mscw#192

avaldebe commented 3 years ago

TL;DR

The current set of emissions and splits (in use since rv4.15) are fine as they are. If we release a new emission dataset, e.g. next release, we need to release new split files and document SNAP/GNFR user configurable options.

metno / emep-ctm

tasks before new release #79

OpenSource rv4.36 (202011)