catalog.py fail to download some meteo-EMEP01 files (Python 2.7)

[JessMBemep]

Hi, I'm trying to download meteorological files EMEP01 by using the recommended python command line: ./catalog.py -Y 2015 --meteo --met-domain EMEP01

The code line successes to download the majority of the data set, but it unexpectdly fails with the following errors: Traceback (most recent call last): File "./catalog.py", line 518, in main(parse_arguments()[0]) File "./catalog.py", line 512, in main x.download(opts.verbose) File "./catalog.py", line 303, in download url = urlopen(self.src) File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 410, in open response = meth(req, response) File "/usr/lib/python2.7/urllib2.py", line 523, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.7/urllib2.py", line 448, in error return self._call_chain(args) File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(args) File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 404:

I have enough space in the computer, so that, I do not think it is a problem of memory space. I launched the same command line twice, without success. The following files are missing after the python error: January days: 12 13 16 17 18 23 24 25 February days: 1 March days: 7 8 11 16 18 19 20 21 24 25 26 27 April days: 1 3 7 8 12 15 17 22 26 27 28 May days: 7 16 19 24 29 June days: 1 11 12 14 16 19 22 28 July days: 1 6 13 14 22 25 August days: 1 5 8 10 14 21 22 26 September days: 4 10 26 October days: 3 4 12 13 15 23 28 29 November days: 3 6 7 12 13 17 20 26 27 December days: 2 6 14 20 21

Had you get previuous issues concerning this error? Any idea of what is happening? Thanks in advance

[avaldebe]

Can you try with Python3? The error your'e getting is from the urllib2 module used on Python2. Python3 uses a different module (urllib.request).

[avaldebe]

If that does not work, you can download the missing meteo from directly the thredds catalog

[JessMBemep]

Hi, Thanks for these advises I have checked the thredds catalog, I think that meteo20150524.nc is missing. Could you provide it please? Thanks

[avaldebe]

@andagit I can not find meteo20150524.nc on the thredds catalog for EMEP01/2015 Can you put it there?

[andagit]

Hm, strange! But does this mean that not many people has downloaded these data? Well, the file is now in place at : /lustre/storeB/project/fou/kl/emep/Meteorology/OpenSource/EMEP01_2015/ -rw-rw-r-- 1 annac emepgrp 1763043513 Nov 16 12:27 meteo20150524.nc

Hilsen Anna

[JessMBemep]

Thanks!

[martinsaliba]

Hi,

I am installing the EMEP_MSC-W model version 4.15. However some input files are missing. The missing files are listed below: Data Missing file Format Landuse LanduseGLC.nc Forest Fire FINN_ForestFireEmis_YYYY.nc Emissions emislist.POLL (7 files, EMEP 50km PS grid) ASCII4 Emis_TNO7.nc (regional, 0.125 × 0.0625 lon-lat) netCDF3 Emis_GLOB_05.nc (global, 0.5 × 0.5 lon-lat) netCDF3 Vertical level Vertical_levels.txt ASCII Emission heights EmisHeights.txt ASCII Natural SO2 natso2MM.dat (12 files) ASCII1 Emissions speciation emissplit.defaults.POLL ASCII4 emissplit.specials.POLL ASCII43

Photo-dissociation rates jcl1.jun jcl3.jun

Any guidance how to get these files?

Regards Martin

[avaldebe]

Each model release includes the files needed to run the model. The following catalog.py command will download the the inputs for rv4_15:

./catalog.py --revision=rv4_15 --input

After downloading and integrity check, the input files will be placed on ./EMEP_MSC-W_model.rv4.15.OpenSource/input/

Data Missing file Format Landuse LanduseGLC.nc

The last 2 releases rv4_15 and rv4_17 released this file under a different name glc2000mCLM.nc, and you can still use Landuse_PS_5km_LC.nc. The configuration files included together with the example outputs shows how to use them.

Forest Fire FINN_ForestFireEmis_YYYY.nc

We can not redistribute the FINN forest fire files, see #19 . Fortunately they are not needed to run the model.

Emissions emislist.POLL (7 files, EMEP 50km PS grid) ASCII4

grided emissions for the EECCA domain are located at ./EMEP_MSC-W_model.rv4.15.OpenSource/input/EECCA/ and for the EMEP01 domain at ./EMEP_MSC-W_model.rv4.15.OpenSource/input/EMEP01/

Emis_TNO7.nc (regional, 0.125 × 0.0625 lon-lat) netCDF3 Emis_GLOB_05.nc (global, 0.5 × 0.5 lon-lat) netCDF3

This are were not included on the latest model releases. You can use the provided grided emissions (for EECCA or EMEP01 domains) or make your own.

Vertical level Vertical_levels.txt ASCII Emission heights EmisHeights.txt ASCII

2 versions of each file are included. For a run on the standard/traditional 20 levels use ./EMEP_MSC-W_model.rv4.15.OpenSource/input/20lev/Vertical_levels.txt ./EMEP_MSC-W_model.rv4.15.OpenSource/input/20lev/EmisHeights.txt and for the newer/experimental 34 levels use ./EMEP_MSC-W_model.rv4.15.OpenSource/input/34lev/Vertical_levels.txt ./EMEP_MSC-W_model.rv4.15.OpenSource/input/34lev/EmisHeights.txt

Natural SO2 natso2MM.dat (12 files) ASCII1

as far as I can tell those files are no longer needed

Emissions speciation emissplit.defaults.POLL ASCII4 emissplit.specials.POLL ASCII43cd ..

are available on ./EMEP_MSC-W_model.rv4.15.OpenSource/input/

Photo-dissociation rates jcl1.jun jcl3.jun

as far as I can tell those files are no longer needed

[avaldebe]

Hi @JessMBemep and @martinsaliba

Looks like your questions have been answered, so I will close the issue. Please re-open it if that was not the case.

If you have a new question, please open a new issue.

[martinsaliba]

HI,

Yes this issue is solved.

Thanks for your support