search

josdejong / lossless-json

Parse JSON without risk of losing numeric information
MIT License
412 stars 29 forks source link

readme

lossless-json

Parse JSON without risk of losing numeric information.

import { parse, stringify } from 'lossless-json'

const text = '{"decimal":2.370,"long":9123372036854000123,"big":2.3e+500}'

// JSON.parse will lose some digits and a whole number:
console.log(JSON.stringify(JSON.parse(text)))
// '{"decimal":2.37,"long":9123372036854000000,"big":null}'
// WHOOPS!!!

// LosslessJSON.parse will preserve all numbers and even the formatting:
console.log(stringify(parse(text)))
// '{"decimal":2.370,"long":9123372036854000123,"big":2.3e+500}'

The following in-depth article explains what happens there: Why does JSON.parse corrupt large numbers and how to solve this?

How does it work? The library works exactly the same as the native JSON.parse and JSON.stringify. The difference is that lossless-json preserves information of big numbers. lossless-json parses numeric values not as a regular number but as a LosslessNumber, a lightweight class which stores the numeric value as a string. One can perform regular operations with a LosslessNumber, and it will throw an error when this would result in losing information.

When to use? If you have to deal with JSON data that contains long values for example, coming from an application like C++, Java, or C#. The trade-off is that lossless-json is slower than the native JSON.parse and JSON.stringify functions, so be careful when performance is a bottleneck for you.

Features:

Install

Install via npm:

npm install lossless-json

Use

Parse and stringify

Parsing and stringification works as you're used to:

import { parse, stringify } from 'lossless-json'

const json = parse('{"foo":"bar"}') // {foo: 'bar'}
const text = stringify(json) // '{"foo":"bar"}'

LosslessNumbers

Numbers are parsed into a LosslessNumber, which can be used like a regular number in numeric operations. Converting to a number will throw an error when this would result in losing information due to truncation, overflow, or underflow.

import { parse } from 'lossless-json'

const text = '{"normal":2.3,"long":123456789012345678901,"big":2.3e+500}'
const json = parse(text)

console.log(json.normal.isLosslessNumber) // true
console.log(json.normal.valueOf()) // number, 2.3

// LosslessNumbers can be used as regular numbers
console.log(json.normal + 2) // number, 4.3

// but the following operation will throw an error as it would result in information loss
console.log(json.long + 1)
// throws Error: Cannot safely convert LosslessNumber to number:
//   "123456789012345678901" will be parsed as 123456789012345680000 and lose information

BigInt

JavaScript natively supports bigint: big integers that can hold a large number of digits, instead of the about 15 digits that a regular number can hold. It is a typical use case to want to parse integer numbers into a bigint, and all other values into a regular number. This can be achieved with a custom numberParser:

import { parse, isInteger } from 'lossless-json'

// parse integer values into a bigint, and use a regular number otherwise
export function customNumberParser(value) {
  return isInteger(value) ? BigInt(value) : parseFloat(value)
}

const text = '[123456789123456789123456789, 2.3, 123]'
const json = parse(text, null, customNumberParser)
// output:
// [
//   123456789123456789123456789n, // bigint
//   2.3, // number
//   123n // bigint
// ]

You can adjust the logic to your liking, using utility functions like isInteger, isNumber, isSafeNumber. The number parser shown above is included in the library and is named parseNumberAndBigInt.

Validate safe numbers

If you want parse a json string into an object with regular numbers, but want to validate that no numeric information is lost, you write your own number parser and use isSafeNumber to validate the numbers:

import { parse, isSafeNumber } from 'lossless-json'

function parseAndValidateNumber(value) {
  if (!isSafeNumber(value)) {
    throw new Error(`Cannot safely convert value '${value}' into a number`)
  }

  return parseFloat(value)
}

// will parse with success if all values can be represented with a number
let json = parse('[1,2,3]', undefined, parseAndValidateNumber)
console.log(json) // [1, 2, 3] (regular numbers)

// will throw an error when some of the values are too large to represent correctly as number
try {
  let json = parse('[1,2e+500,3]', undefined, parseAndValidateNumber)
} catch (err) {
  console.log(err) // throws Error 'Cannot safely convert value '2e+500' into a number'
}

BigNumbers

To use the library in conjunction with your favorite BigNumber library, for example decimal.js. You have to define a custom number parser and stringifier:

import { parse, stringify } from 'lossless-json'
import Decimal from 'decimal.js'

const parseDecimal = (value) => new Decimal(value)

const decimalStringifier = {
  test: (value) => Decimal.isDecimal(value),
  stringify: (value) => value.toString()
}

// parse JSON, operate on a Decimal value, then stringify again
const text = '{"value":2.3e500}'
const json = parse(text, undefined, parseDecimal) // {value: new Decimal('2.3e500')}
const output = {
  // {result: new Decimal('4.6e500')}
  result: json.value.times(2)
}
const str = stringify(output, undefined, undefined, [decimalStringifier])
// '{"result":4.6e500}'

Reviver and replacer

The library is compatible with the native JSON.parse and JSON.stringify, and also comes with the optional reviver and replacer arguments that allow you to serialize for example data classes in a custom way. Here is an example demonstrating how you can stringify a Date in a different way than the built-in reviveDate utility function.

The following example stringifies a Date as an object with a $date key instead of a string, so it is uniquely recognizable when parsing the structure:

import { parse, stringify } from 'lossless-json'

// stringify a Date as a unique object with a key '$date', so it is recognizable
function customDateReplacer(key, value) {
  if (value instanceof Date) {
    return {
      $date: value.toISOString()
    }
  }

  return value
}

function isJSONDateObject(value) {
  return value && typeof value === 'object' && typeof value.$date === 'string'
}

function customDateReviver(key, value) {
  if (isJSONDateObject(value)) {
    return new Date(value.$date)
  }

  return value
}

const record = {
  message: 'Hello World',
  timestamp: new Date('2022-08-30T09:00:00Z')
}

const text = stringify(record, customDateReplacer)
console.log(text)
// output:
//   '{"message":"Hello World","timestamp":{"$date":"2022-08-30T09:00:00.000Z"}}'

const parsed = parse(text, customDateReviver)
console.log(parsed)
// output:
//   {
//     action: 'create',
//     timestamp: new Date('2022-08-30T09:00:00.000Z')
//   }

API

parse(text [, reviver [, parseNumber]])

The LosslessJSON.parse() function parses a string as JSON, optionally transforming the value produced by parsing.

stringify(value [, replacer [, space [, numberStringifiers]]])

The LosslessJSON.stringify() function converts a JavaScript value to a JSON string, optionally replacing values if a replacer function is specified, or optionally including only the specified properties if a replacer array is specified.

LosslessNumber

Construction

new LosslessNumber(value: number | string) : LosslessNumber

Methods

Properties

Utility functions

Alternatives

Similar libraries:

Test

To test the library, first install dependencies once:

npm install

To run the unit tests:

npm test

To build the library and run the unit tests and integration tests:

npm run build-and-test

Lint

Run linting:

npm run lint

Fix linting issues automatically:

npm run format

Benchmark

To run a benchmark to compare the performance with the native JSON parser:

npm run benchmark

(Spoiler: lossless-json is much slower than native)

Build

To build a bundled and minified library (ES5), first install the dependencies once:

npm install

Then bundle the code:

npm run build

This will generate an ES module output and an UMD bundle in the folder ./.lib which can be executed in browsers and node.js and used in the browser.

Release

To release a new version:

$ npm run release

This will:

To try the build and see the change list without actually publishing:

$ npm run release-dry-run

License

Released under the MIT license.