geopython / GeoHealthCheck

Service Status and QoS Checker for OGC Web Services
https://geohealthcheck.org
MIT License
83 stars 71 forks source link

update jspark download location #447

Closed tomkralidis closed 1 year ago

tomkralidis commented 1 year ago

As per https://github.com/geopython/GeoHealthCheck/pull/446#issuecomment-1595283162

tomkralidis commented 1 year ago

CI continues to error out based on this dependency download. @justb4 perhaps we can just include as part of the source code (stable)?

justb4 commented 1 year ago

Yes. Relying on a personal URL seems brittle.

justb4 commented 1 year ago

Though it could be a HTTP/2 problem, some kind of negotiation mismatch which urlopen() can't handle. Seen this with MapServer WFS from pygeoapi as well: https://lists.osgeo.org/pipermail/mapserver-users/2023-March/083084.html

justb4 commented 1 year ago

See e.g.

>>> import urllib.request
>>> urllib.request.urlopen('http://python.org/')
<http.client.HTTPResponse object at 0x104960a58>
>>> urllib.request.urlopen('https://johnresig.com/files/jspark.js')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/just/.pyenv/versions/3.7.1/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/just/.pyenv/versions/3.7.1/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/Users/just/.pyenv/versions/3.7.1/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Users/just/.pyenv/versions/3.7.1/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/Users/just/.pyenv/versions/3.7.1/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/Users/just/.pyenv/versions/3.7.1/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
justb4 commented 1 year ago

Problem of the 403 here is not sending a UserAgent HTTP header. This works:

url='https://johnresig.com/files/jspark.js'
request_site = Request(url, headers={"User-Agent": "Mozilla/5.0"})
urlopen(request_site).read() 
justb4 commented 1 year ago

Still for this case placing jspark.js directly under static/lib is better option. Most other URL fetches are GitHub and CDNs, thus more stable. We did a similar thing with JQuery UI.