Downloading meteo data for EMEP01 grid not working with the catalog tool

[rosagierens]

Hey, I tried to get the meteo data for the EMEP01 grid (as instructed) with : ./catalog.py -Y 2015 -m --met-domain EMEP01 but get the following error:

  File "./catalog.py", line 514, in <module>
    main(parse_arguments(sys.argv[1:] or ['-h'])[0])
  File "./catalog.py", line 508, in main
    x.download(opts.verbose)
  File "./catalog.py", line 299, in download
    url = urlopen(self.src)
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib64/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 421: Misdirected Request

In another issue it was suggested to download data directly form the thredds catalog, however the link (http://thredds.met.no/thredds/catalog/data/EMEP/Meteo/EMEP01/2015/catalog.html) does not work.

[avaldebe]

I get the same error, will have deeper a look

[avaldebe]

@andagit

Looks like our meteorology is no longer available via thredds

I get nothing at https://thredds.met.no/thredds/catalog/data/EMEP/Meteo/EMEP01/2015/catalog.html

[avaldebe]

@rosagierens

Seems the meteorological dataset you requested disappeared from storage at some point last September. We are working to get the dataset back.

We'll let you know when it is available again. In the meantime, we updated the catalogue tool with a new URL for the dataset. You can get the latest version catalog.py and catalog.csv at https://github.com/metno/emep-ctm/tree/tools

[rosagierens]

@avaldebe Thank you for the quick reply! I'll stay tuned.

[avaldebe]

@rosagierens

Had to update catalog.py and catalog.csv again,. On the bright side, the files are coming back. We have about 4 months already available. You can see the full list at:

https://thredds.met.no/thredds/catalog/data/EMEP/OpenSource/EMEP01/2015/catalog.html

[andagit]

@rosagierens Finally all data in place at thredds now - a bit painful from my homeoffice, but please let me know if anything is missing now. At least good that Alvaro has already tested that script and new links are working fine!

[rosagierens]

It took some time, but I'm happy to report that I was successful in downloading the data (the --cleanup option turned out to be important) and the simulation for a full year just completed without issues. Thank you again for your assistance!