Incorrect serialization of infinite values: Floats to 'null', Integers to '-2147483648'

[kryaksy]

Bug Report

Specifically, the problem arises during the serialization of this data using the to_json and to_columns methods. Notably, the serialization appears correct when using the to_csv method. The issue is demonstrated in the provided reproduction link.

Steps to Reproduce:

Please refer to the codeSandbox link for a self-explanatory code sample demonstrating the issue.

Expected Result:

-/+Infinity values should be correctly serialized, maintaining their integrity across different methods.

Actual Result:

• For float data types, infinity values are serialized as 'null'.
• For integer data types, infinity values are serialized as '-2147483648'.

Environment:

• perspective@2.7.1
• codesandbox

[texodus]

Thanks for the report, this is not a bug.

• "integer" columns cannot represent infinity or nan, in Perspective or otherwise.
• "float" columns replace NaN, -Infinity and Infinity, with null intentionally - as asserted by our test suite. We made this decision intentionally in the very first Perspective release for two reasons:
  • These values are generally not relevant in the context of data visualization, and their presence among other false-y value makes other operations in the UI more cumbersome (like filters and groups with null).
  • JSON does not support these values, meaning we need to add validation and encoding/decoding anywhere the data may be stringified as JSON, leading to performance and significant complexity cost.

It's possible we could elaborate on support for these values for "float" types only, but discussion around this should go in its own feature request if anyone cares to elaborate on what the real use case for this would be.

[kryaksy]

Thanks for the prompt response, @texodus .

I understand your points regarding integers. However, for floats, while these values may be irrelevant and possibly undesirable for visualization libraries outside of data grids, there are many scenarios where data grids are more frequently used and where handling these values becomes necessary. In such contexts, there can be significant semantic differences between NaN, +/-Infinity, and null values. I am confident that in my specific situation, acknowledging these distinctions is crucial.

Considering this, I wonder if it's feasible to accurately serialize these values with a method akin to to_json. Would it be suitable to open a feature request for this particular issue?