datadotworld / data.world-py

Python package for data.world
https://data.world/integrations/python
Apache License 2.0
101 stars 30 forks source link

String literal 'NONE' gets mapped to Python None value in result sets #68

Open bryonjacob opened 6 years ago

bryonjacob commented 6 years ago

In a SQL or SPARQL response, the literal string value 'NONE' (case-insensitive) is converted into the python value None:

>>> datadotworld.query('bryon/odin-2015-2016', 'SELECT ?value WHERE{ BIND("NONE" AS ?value)}', query_type='sparql').table
[OrderedDict([('value', None)])]
>>> datadotworld.query('bryon/odin-2015-2016', 'SELECT ?value WHERE{ BIND("ABCD" AS ?value)}', query_type='sparql').table
[OrderedDict([('value', 'ABCD')])]
>>> datadotworld.query('bryon/odin-2015-2016', 'SELECT "NONE" AS value').table
[OrderedDict([('value', None)])]
>>> datadotworld.query('bryon/odin-2015-2016', 'SELECT "ABCD" AS value').table
[OrderedDict([('value', 'ABCD')])]
bryonjacob commented 6 years ago

UGH. so, this is because we are depending on the datapackage library which is hardcoded to treat any of these strings:

['null', 'none', 'nil', 'nan', '-']

as None when they occur as a value in a StringValue-typed column. In the context of returning query results from our servers, this is always wrong - we have the capacity to handle null results just fine explicitly.

rflprr commented 6 years ago

Some work here is still necessary after Bryon's hotfix to ensure that there is no broader impact and, possibly, a simpler and more generalized solution.

rflprr commented 6 years ago

Py: Small bugs