JSONDecodeError: Expecting value for some indicators

[jyslee]

To recreate the error

import wbdata wbdata.get_data('SL.TLF.TOTL.IN', country='all'

Error

`Traceback (most recent call last):

File "", line 1, in wbdata.get_data('SL.TLF.TOTL.IN', country='all')

File "C:\Users\joshua.lee\Anaconda3\lib\site-packages\wbdata\api.py", line 289, in get_data data = fetcher.fetch(query_url, args, cache=cache)

File "C:\Users\joshua.lee\Anaconda3\lib\site-packages\wbdata\fetcher.py", line 122, in fetch response = get_response(url, args, cache=cache)

File "C:\Users\joshua.lee\Anaconda3\lib\site-packages\wbdata\fetcher.py", line 100, in get_response return json.loads(response)

File "C:\Users\joshua.lee\Anaconda3\lib\json__init__.py", line 357, in loads return _default_decoder.decode(s)

File "C:\Users\joshua.lee\Anaconda3\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File "C:\Users\joshua.lee\Anaconda3\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Expecting value`

Thoughts

I have tried tracing through to the fetcher so see if I can understand the cause of the issue but am struggling. When I put the query_url into a browser (https://api.worldbank.org/v2/countries/all/indicators/SL.TLF.TOTL.IN), the output is largely as expected to the naked eye.

For other indicators, this issue only arises if I specify a start / end date. It's a bit of a mystery to me!

Any help gratefully appreciated.

[edwardtilley]

I have the same problem with NY.GDS.TOTL.CN , NY.GNP.MKTP.KD , NY.GNS.ICTR.GN.ZS , SE.SEC.TCHR.FE , SE.TER.CUAT.ST.MA.ZS , SP.POP.0004.MA.5Y ... did you ever get to the bottom of this problem? wbdata is also erroring-out with a non-standard python exception, to catch this you have to import sys, traceback and then try: except Exception: traceback.print_exc(file=sys.stdout) . Its a different process than except (IndexError, RuntimeError) as e: but at least it's possible to "catch" programmatically

[jyslee]

The issue seems to have been resolved for me with a more recent version of wbdata. Have you installed the latest version?

[edwardtilley]

Thanks for responding....

I'll take a look at the update shortly. I notice that wbdata is a considerable resource hog and much slower than a simple api url call as well, is there a better way to collect the same data by simply using the same json data?

Edward

http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avg.com http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Dec 14, 2020 at 4:50 AM reevesnmortimer notifications@github.com wrote:

The issue seems to have been resolved for me with a more recent version of wbdata. Have you installed the latest version?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OliverSherouse/wbdata/issues/61#issuecomment-744320180, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANZ245TZMWHIHEUVS6FC27LSUXNVJANCNFSM4QGHBKZA .

-- Edward

Edward Tilley, Direct: +1(289) 834-0072

Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential, and may be privileged. If you are not the intended recipient, you are hereby notified that any review, retransmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email, and delete this message and any attachments from your system. Thank-you!

[jyslee]

I use wbdata for the convenience of getting the World Bank data straight into pandas.

You could access World Bank's API directly and parse the JSON response yourself if you prefer (https://datahelpdesk.worldbank.org/knowledgebase/articles/898581)