CSIRO-enviro-informatics / loci-integration-api

This application provides a convenient interface for executing common functions across the whole LOCI system.
0 stars 3 forks source link

Inconsistent JSON response (NaN != nan) #71

Open shaneseaton opened 4 years ago

shaneseaton commented 4 years ago

The api is returning two different values of NaN which is, if nothing else, confusing. We should create a constant and use that.

{
  meta: { count: 1, offset: 0, featureArea: 'NaN' }
  overlaps:[{
    "uri":"http:\\/\\/linked.data.gov.au\\/dataset\\/asgs2016\\/meshblock\\/50055290000",
    "intersectionArea":"nan",
    "featureArea":"37700.00000000",
    "forwardPercentage":"nan",
    "reversePercentage":"nan"
  }]
}
jyucsiro commented 4 years ago

@shaneseaton should we use 'NaN' everywhere then?

shaneseaton commented 4 years ago

If we HAVE to use one or the other I would suggest NaN as it is how I have seen it used most of the time, and it what is used in javascript as a constant.

However, I am pretty sure the JSON spec doesn't account for NaN as a response. Note, we don't use NaN we use it as a string "NaN", which is perhaps even worse.

I think if it were up to me, if there was a NaN value in python, I would export it in my JSON as null:

{
  meta: { count: 1, offset: 0, featureArea: null }
  overlaps:[{
    "uri":"http:\\/\\/linked.data.gov.au\\/dataset\\/asgs2016\\/meshblock\\/50055290000",
    "intersectionArea": null,
    "featureArea":"37700.00000000",
    "forwardPercentage": null,
    "reversePercentage": null
  }]
}

or I would remove the properties from the output all together:

{
  meta: { count: 1, offset: 0 }
  overlaps:[{
    "uri":"http:\\/\\/linked.data.gov.au\\/dataset\\/asgs2016\\/meshblock\\/50055290000",
    "featureArea":"37700.00000000",
  }]
}

I have seen both used, and don't mind either, so long as consistent. The use of null is perhaps a bit easier to look at, and to consume from other languages than JS.

shaneseaton commented 4 years ago

Having a quick look, this post seams to support my musings above.

https://stackoverflow.com/questions/6601812/sending-nan-in-json/6602204

shaneseaton commented 4 years ago

Also, I just noticed the numbers are also quoted in the JSON output suggesting they are strings!!! Drop the quotes on the numbers too. (also reduces payload size)

Perhaps this done because of precision? There must be a json.dumps or something in python that does this stuff without too much of a headache.

jyucsiro commented 4 years ago

the use of null is more consistent with JSON.

On the numbers, we had this issue with the decimals/floats in the encoding and hit an issue with float/decimal handling. The decision was to preserve the decimal using strings. See also https://stackoverflow.com/questions/35709595/why-would-you-use-a-string-in-json-to-represent-a-decimal-number