Access to the ECMWF ftp server for TC forecast tracks does not work anymore

[manniepmkam]

The ftp server (ftp://wmo:essential@dissemination.ecmwf.int) where TCForecast.fetch_ecmwf() fetches the latest TC tracks does not work anymore since around half a year ago. Currently I do not know where ECMWF would put their data and how to access it.

Potential work to solve the issues:

• Find out where the latest TC forecast tracks (bufr format) is stored.
• Possibly re-route the script in climada_petals.hazard.TCForecast to fetch the BUFR files
• Contact ECMWF if we couldn't find out where they store the data

[peanutfun]

I executed this code on my machine (while inside the ETH network) and it downloads the data as it should:

from climada_petals.hazard import TCForecast
fcast = TCForecast()
fcast.fetch_ecmwf()

Maybe a probnlem with your particular setup, @manniepmkam?

[mmyrte]

I can actually reproduce this on macOS 13 and 11; I could only get it to not immediately drop with HTTP. @peanutfun, you're on some bare metal Linux, right? I thought using a virtual machine (docker desktop with an alpine image) might help, but I get the same response.

~ curl --verbose "ftp://dissemination.ecmwf.int"
*   Trying 136.156.176.1:21...
* Connected to dissemination.ecmwf.int (136.156.176.1) port 21 (#0)
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
~ curl --verbose "ftp://wmo:essential@dissemination.ecmwf.int"
*   Trying 136.156.176.1:21...
* Connected to dissemination.ecmwf.int (136.156.176.1) port 21 (#0)
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
~ curl --verbose "http://wmo:essential@dissemination.ecmwf.int"
*   Trying 136.156.176.1:80...
* Connected to dissemination.ecmwf.int (127.0.0.1) port 80 (#0)
* Server auth using Basic with user 'wmo'
> GET / HTTP/1.1
> Host: dissemination.ecmwf.int
> Authorization: Basic d21vOmVzc2VudGlhbA==
> User-Agent: curl/7.87.0
> Accept: */*
>
^C
~ curl --verbose "https://wmo:essential@dissemination.ecmwf.int"
*   Trying 136.156.176.1:443...
* Connected to dissemination.ecmwf.int (127.0.0.1) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Client hello (1):
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dissemination.ecmwf.int:443
* Closing connection 0
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dissemination.ecmwf.int:443

[mmyrte]

Actually, it seems that ECMWF is now linking to the new domain diss.ecmwf.int, see https://www.ecmwf.int/en/forecasts/datasets/wmo-essential which seems to hold the data. @manniepmkam can you take it from here?

[peanutfun]

@manniepmkam I'm on macOS 13.3.1 😕

[manniepmkam]

Actually, it seems that ECMWF is now linking to the new domain diss.ecmwf.int, see https://www.ecmwf.int/en/forecasts/datasets/wmo-essential which seems to hold the data. @manniepmkam can you take it from here?

@mmyrte Thanks for pointing out where the data is! However, currently this is not on my priority list. Please feel free to propose an update:)

[peanutfun]

According to the suggestion by @mmyrte, it looks to me like one only needs to change the FTP server definition in this line: https://github.com/CLIMADA-project/climada_petals/blob/3d716f1d5b3e5496c66aec636de4856badbdd015/climada_petals/hazard/tc_tracks_forecast.py#L52

@manniepmkam, it would be great if you could raise a PR