Error when i try to fetch data

[darvinrivera]

Fetching CSSE CSV... 2020/04/01...2020/03/31...2020/03/30...2020/03/29...2020/03/28... 2020/03/27...2020/03/26...2020/03/25...2020/03/24...2020/03/23... 2020/03/22...2020/03/21...2020/03/20...2020/03/19...2020/03/18... 2020/03/17...2020/03/16...2020/03/15...2020/03/14...2020/03/13... 2020/03/12...2020/03/11...2020/03/10...2020/03/09...2020/03/08... 2020/03/07...2020/03/06...2020/03/05...2020/03/04...2020/03/03... 2020/03/02...2020/03/01...2020/02/29...2020/02/28...2020/02/27... 2020/02/26...2020/02/25...2020/02/24...2020/02/23...2020/02/22... 2020/02/21...2020/02/20...2020/02/19...2020/02/18...2020/02/17... 2020/02/16...2020/02/15...2020/02/14...2020/02/13...2020/02/12... 2020/02/11...2020/02/10...2020/02/09...2020/02/08...2020/02/07... 2020/02/06...2020/02/05...2020/02/04...2020/02/03...2020/02/02... 2020/02/01...2020/01/31...2020/01/30...2020/01/29...2020/01/28... 2020/01/27...2020/01/26...2020/01/25...2020/01/24...2020/01/23... 2020/01/22... Fetching CSSE CSV completed Fetching CSSE REST... Traceback (most recent call last): File "fetch_data.py", line 1215, in fetch_csse_rest() File "fetch_data.py", line 379, in fetch_csse_rest features = fetch_all_features(features_url) File "fetch_data.py", line 367, in fetch_all_features res = json.loads(res.content.decode()) File "/usr/lib/python3.7/json/init.py", line 348, in loads return _default_decoder.decode(s) File "/usr/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[4rund3v]

I think i can contribute to this project, if there are issues or enhancements that can be done; Please let me know. Thanks!

[HuidaeCho]

res = json.loads(res.content.decode())

I cannot reproduce it right now. An exception in this line implies that JHU's response is not valid JSON. I think it was a temporary issue caused by JHU's REST server. Maybe, data was being updated on their side? Are you still having this issue?

[HuidaeCho]

I think i can contribute to this project, if there are issues or enhancements that can be done; Please let me know. Thanks!

Sure, contributions are always welcome! Especially, I'm interested in automatic data fetching so I can be less dependent on JHU's data errors.

[javierconcha]

I think i can contribute to this project, if there are issues or enhancements that can be done; Please let me know. Thanks!

It would be nice to have a way to switch the CFR plot to other plots, like daily increases, for instance. Just an idea. Something that would be useful for my adaptation of @HuidaeCho's code: https://github.com/javierconcha/covid-19-Chile

[HuidaeCho]

I think i can contribute to this project, if there are issues or enhancements that can be done; Please let me know. Thanks!

It would be nice to have a way to switch the CFR plot to other plots, like daily increases, for instance. Just an idea. Something that would be useful for my adaptation of @HuidaeCho's code: https://github.com/javierconcha/covid-19-Chile

That's a good idea. Please create a new ticket with your request if you don't mind. Daily increase. Country-to-country comparisons starting all from 0, etc.

[darvinrivera]

I make Clean instalation.. again... Same Error...

[HuidaeCho]

I think I know what it is. Do you have your config.py? You need to first copy config-example.py to config.py and set it up. For example,

bing_maps_key = 'your_bing_maps_key'
bing_maps_referer = 'https://your_covid19_app_url'
app_url = 'https://your_covid19_app_url'
use_local_data_only = False
countries_to_display = ()

JHU's REST server returns an error if you don't set a referer (app_url setting above). You need to put your web map's URL there. In my case, it's https://app.isnew.info/covid-19.

Also, if you need geocoding using Bing Maps, you need to set bing_maps_key and bing_maps_referer. bing_maps_referer protects your key from being used from anywhere else than your URL. You can set it up in the Bing Maps Dev Center (https://www.bingmapsportal.com). Account => May keys => Security. If you don't ever need geocoding, you don't need those two settings.

[darvinrivera]

I have set my config.py with all fields When i open "features_url" var in the browser:

403 ERROR The request could not be satisfied. Request blocked. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner. If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.

Generated by cloudfront (CloudFront)

[HuidaeCho]

Yes, that's because when you open the URL directly, the referer is not set. I get the same error message if I directly type the URL in the browser. If you have wget, you can try:

wget 'https://services9.arcgis.com/N9p5hsImWXAccRNI/arcgis/rest/services/Nc2JKvYFoAEOFCG5JSI6/FeatureServer/1/query?where=1%3D1&outFields=*&f=json'

and

wget --referer=https://example.com 'https://services9.arcgis.com/N9p5hsImWXAccRNI/arcgis/rest/services/Nc2JKvYFoAEOFCG5JSI6/FeatureServer/1/query?where=1%3D1&outFields=*&f=json'

[HuidaeCho]

Are you still having this problem with app_url set in config.py?

[darvinrivera]

Yes, i still have the problem...

[HuidaeCho]

Yes, i still have the problem...

Can I see your app_url? I want to try it.