Unable to load timescales from outside USA

[enritoomey]

I was unable to load the timescales as I have no access to http://maia.usno.navy.mil/ser7/deltat.data or http://maia.usno.navy.mil/ser7/deltat.preds. Only USA proxies can access this sites. I was able to solve the problem by changing my proxy settings, but it would be nice to provide alternative sources that do not require USA proxies.

[brandon-rhodes]

Thank you for this big report! Which proxy should I try using if I want to see the error that you were encountering? I want to understand why proxies outside of the United States are having problems connecting, since I had thought that the HTTP connection would work the same regardless of the country in which a proxy is located. So, thanks for any additional details you can provide!

[enritoomey]

I'm connecting from argetina. Try using Proxy Id = 138.59.247.3 with proxy Port=8080.

[ckuethe]

Linking to #89, because this is also relevant to using skyfield in a degraded-/no-network situation

[tritium21]

I just checked a few open proxies inside the US vs. a few outside of the US. It turns out the United States military, especially the departments that have cyber-warfare capabilities (namely the air force and the navy -- this data is on a US Navy server), is sensitive to open (requiring no authentication) proxies outside of the United States accessing their systems. Baring a massive change in the world geo-political status-quo... other arrangements will need to be made.

[brandon-rhodes]

Interesting! What is the advantage of using a proxy from other countries, as compared to simply connecting directly to the site to download the files?

If some people cannot download these files directly from their source, then we should next look for any kind of official mirrors — though the danger would be that they would not be as carefully kept up to date. We should do some Google searches for snippets from the files and see if we can find other copies of them up at other organizations’ web sites that could be trusted to keep them up to date. As a backup measure — I think we still want to pull from the official site as a default.

[ckuethe]

There are lots of ways and reasons a user might have a degraded network connection:

• Some networks ban some paste sites to prevent data leakage but while allowing others.
• Some networks ban ftp because it's not firewall friendly.
• Some communication providers filter protocols for economic reasons, maybe so they can inject ads or can upsell a higher level of service ("email/social" class of service vs. "internet").
• Some network operators think they have an obligation to refuse communication with certain countries. (OFAC list)
• Some network operators just don't like the traffic profile they see coming from certain parts of the net (Tor, Russia, China, whoever is in trade negotiations this week...)
• Some people might have a legitimate cause to try bypass filtering technologies

http://asa.usno.navy.mil/SecK/DeltaT.html suggests that the same quantities may be found in various IERS bulletins... https://www.iers.org/IERS/EN/DataProducts/EarthOrientationData/eop.html

[brandon-rhodes]

@ckuethe — If you find the same files over at the IERS site, please let me know — I would be happy to switch to that official source if the same information was available! Meanwhile, let me know if the proxies do allow access to that site, or whether it seems to have the same annoyance with open proxies. Thanks!

@tritium21 — Thanks for researching the military's policy regarding open proxies! At least now we know why those .mil machines are not accepting some proxied connections.

[tritium21]

@brandon-rhodes My research of policy involves trial and error through about 3 dozen open proxies and a phone call to a former CPO... so... don't take it as official, but it fits with what I was told and what I saw.

[zdomjus60]

Hi Brandon, thank you for your excellent work. I was googling to find a solution to this issue and looking at IERS home page I found this link just at the front page: https://datacenter.iers.org/web/guest/eop/-/somos/5Rgv/latest/6 which says:

Please note that daily and Bulletin A EOP data can be obtained from the primary Earth Orientation (EO) servers at:
http://maia.usno.navy.mil ftp://maia.usno.navy.mil
and from the backup EO server at:
http://toshi.nofs.navy.mil ftp://toshi.nofs.navy.mil

From Italy, the country I write from, both the cited backup server links http://toshi.nofs.navy.mil/ser7/ and ftp://toshi.nofs.navy.mil/ser7/
are accessible without restrictions. I hope this can be useful to anyone

[tritium21]

@zdomjus60 Can you access it from an open proxy in your country?

[zdomjus60]

Hi tritium21. Try it, click on link, it should work from everywhere

[tritium21]

@zdomjus60 Direct access was never the issue. The problem is that some users have to use proxies to have reliable access to overseas websites, and that the united states military blocks access to a class of those proxies.

[zdomjus60]

@tritium21 After your last message, I have tried to connect to the three repositories (maia.usno.navy.mi, toshi.nofs.navy.mil (both html and ftp) and ftp://cddis.gsfc.nasa.gov/pub/products/iers) using Orbot on Android, to test proxy connections from a large number of locations (ExitNodes) in the USA and in various countries in the world, obtaining inconstant results. Sometimes connections failed even from the USA, sometimes succeeded from Eastern Europe. So maybe the problem is not the military or governative nature of the servers, but simply the fact that anonymous connections are refused because they are considered potentially malicious. Even large private organizations like Wikipedia have a full list of banned IPs. In my modest opinion, proxy chaining is not a good way to go. A public open source software like Skyfield, after unsuccessful trials, shoud provide a hardcoded solution based on historical data and formulas for forecasting, giving the user a specific alert for this. Curiously enough, today the main repository of US navy accepts connections from Italy, first time after several days of denials.