search

regro / cf-scripts

Flagship repo for cf-regro-autotick-bot
Other
52 stars 74 forks source link

Bot missed version updates for ECMWF Magics and Metview #942

Closed dtip closed 4 years ago

dtip commented 4 years ago

Metview went from v5.8.2 to v5.8.3. Magics went from v4.3.0 to v4.3.1. Bot missed them both.

metview: new release not noticed after 24 hours conda-forge/metview-feedstock#25 magics: new released not noticed after at least 24 hours conda-forge/magics-feedstock#67

Opening at the request of @beckermr to investigate later. Thanks :)

iainrussell commented 4 years ago

Thanks @dtip for opening this

iainrussell commented 4 years ago

I've been told that if it's possible to know the IP address of the machine that's scanning for new tarballs, we can check if we're blocking it.

beckermr commented 4 years ago

The bot finds the url fine and starts the download. However, the tarballs are huge and exceed our 120-second timeout. Is there a reason your source tarballs are so big?

beckermr commented 4 years ago

Also the download speeds from this server are really bad (at least for me). ~2 mins for ~60 MB.

beckermr commented 4 years ago

As this is not an issue with the bot per say, I am going to close this for now. Feel free to reopen!

iainrussell commented 4 years ago

Thanks for the @beckermr that's very interesting. Some of our tarballs do contain some large data files that are essential to them, which accounts for the large size. As for speed of download, I guess this is quite variable - I got Magics (60MB) in about a minute from home, and about one second from a browser running at ECMWF :)