search

cta-observatory / ctapipe

Low-level data processing pipeline software for CTAO or similar arrays of Imaging Atmospheric Cherenkov Telescopes
https://ctapipe.readthedocs.org
BSD 3-Clause "New" or "Revised" License
64 stars 268 forks source link

Unit tests fail to download files from dataserver when in homeoffice #2491

Open LukasBeiske opened 8 months ago

LukasBeiske commented 8 months ago

When I try to run the unit tests while in homeoffice the download of the needed files from the dataserver is very slow to the point of failing to download some files resulting in multiple errors like:

ERROR ctapipe/io/tests/test_eventseeker.py - requests.exceptions.ConnectionError: HTTPConnectionPool(host='cccta-dataserver.in2p3.fr', port=80): Read timed out.

Apparently this is related to the IP address trying to download the data, since this does not happen when I am in the office (using the same machine).

maxnoe commented 8 months ago

I also encountered this issue in the past and could only solve it by using a VPN.

@bregeon couldn't help if I remember correctly, since he doesn't have control over the external firewall at in2p3 or whatever else is causing these issues

kosack commented 8 months ago

We actually have a new data server being set up at DESY, so we can at some point drop CCin2p3 and have more control... For now probably there is nothing we can do. Maybe there is some blocklist set up there now, as for me at home it works fine.

maxnoe commented 8 months ago

You're in France...

kosack commented 8 months ago

You're in France...

Though it works from GitHub's CI, which probably isn't. But yes, it's likely they block some IPs (GitHub is likely an exception)

maxnoe commented 8 months ago

it's likely they block some IPs

It's not completely blocked, just very very slow, I get something like 20 kB/s at home without VPN, and 4 MB/s with VPN.

My home is ~2 km away from the university...