revisit: Support JSON serialisation of BigInt values

[kaizhu256]

https://github.com/tc39/proposal-bigint/issues/24 mainly discussed literal JSON-serialization (and how it would break JSON-spec).

what about serializing to string-from, like Date? yes its one-way, but it provides needed guidance, and i'm skeptical we'll come up with a better solution 5 years from now.

JSON.stringify({aa:1234n}) // '{"aa":"1234"}'
// or
JSON.stringify({aa:1234n}) // '{"aa":"1234n"}'

[littledan]

This proposal exposed a hook specifically for this purpose: BigInt.prototype.toJSON. Just set that to BigInt.prototype.toString to turn on this behavior.

I think this option should be used with caution since it doesn't round-trip: JSON.parse of the serialized form will give you a Number. It's an important goal of this proposal to not implicitly round BigInt to Number, but permit explicit conversion.

When you want to include BigInt in JSON, I would recommend using a library to do the serialization and deserialization, such as granola.

[kaizhu256]

no, JSON.parse would just return a string (copying behavior of Date):

JSON.parse(JSON.stringify(new Date())) // '2018-07-28T09:41:47.519Z'
JSON.parse(JSON.stringify(1234n)) // '1234' or '1234n'

this is good-enough guidance imo (and let user figure out how to revive BigInt, just like Date). i don't see a simpler, easier-to-use solution than Date-like behavior in the forseeable future.

i just played with granola. seems to have bugs stringifying Set and Map. and not sure how to run it in the browser (where i can test BigInt).

[kaizhu256]

oh, figured out problem with granola (top-level must be object). still, i think having guidance to copy Date-like JSON-serialization behavior for BigInt makes better sense.

[littledan]

JSON.parse(JSON.stringify(1234n)) // '1234' or '1234n'

Not sure what you mean by this. Under what conditions would the answer be 1234n?

It seems like we're coming down to a difference in goals: I have been trying to avoid implicit precision-loss, while permitting JSON serialization, and it sounds like you're OK with some implicit conversions but are more bothered by the JSON serialization behavior not being present by default. Is that accurate?

[kaizhu256]

Not sure what you mean by this. Under what conditions would the answer be 1234n?

i'll remove the confusing n-suffix proposal (don't care too much about it). maybe this will clarify?

var aa;

// we copy JSON-behavior of Date
aa = 12345678901234567890n // <bigint primitive>
aa = JSON.stringify(aa) // '"12345678901234567890"'
aa = JSON.parse(aa) // '12345678901234567890'
aa = BigInt(aa) // <bigint primitive>

aa = new Date() // <Date object>
aa = JSON.stringify(aa) // '"2018-07-28T09:41:47.519Z"'
aa = JSON.parse(aa) // '2018-07-28T09:41:47.519Z'
aa = new Date(aa) // <Date object>

[littledan]

OK, I see what you mean. I still would prefer not to go with those semantics by default, for the reason explained in https://github.com/tc39/proposal-bigint/issues/162#issuecomment-408594787 : Date will "round-trip" through that process accurately, whereas BigInt will be rounded, based on the implicit conversion to a number and back.

[kaizhu256]

i'm confused now by what you mean by rounded? there's no implicit conversion to number (or implicit precision-loss) for JSON.parse. it remains a string (with full-precision). and JSON-spec remains unchanged.

[kaizhu256]

just to clarify further:

var aa;

// we copy JSON-behavior of Date
aa = 12345678901234567890n // <bigint primitive>
aa = JSON.stringify(aa) // '"12345678901234567890"' (escaped string)
aa = JSON.parse(aa) // '12345678901234567890' (un-escaped string)
aa = BigInt(aa) // <bigint primitive> (no precision-loss)

aa = new Date() // <Date object>
aa = JSON.stringify(aa) // '"2018-07-28T09:41:47.519Z"'' (escaped string)
aa = JSON.parse(aa) // '2018-07-28T09:41:47.519Z' (un-escaped string)
aa = new Date(aa) // <Date object>

[littledan]

Ah, I see, you are proposing that it be a string that includes the quotes. Interesting idea. How about we consider this as a follow-on proposal, and experiment in userspace for now by overriding BigInt.prototype.toJSON?

[kaizhu256]

no. i dislike the current behavior (from web-integration perspective), and believe it should be a last-resort option, only if no palatable default-serialization solution is found. and i think Date's JSON-behavior is a palatable solution that is least offensive to everyone, unless someone can think of serious technical-faults for it.

people are already familiar with Date's JSON-behavior, so it won't surprise anyone (or at least less-surprising than throwing an error).

i agree that for the forseeable future, ecma-404 and JSON.parse should not change for stability-reasons. with that constraint, this is the best possible outcome i can think of for both BigInt and possible future primitives like BigDecimal.

[cyberphone]

To put it differently: Would anybody (freed from legacy etc) seriously design a typed general purpose format for data interchange utilizing a single type for everything from a byte to BigNumber? I guess not, which is why using the JSON Number type outside of its already firmly established (JavaScript) scope seems like an unnecessary difficult path although the JSON specification admits it.

There is also a bunch of IETF standards using JSON structures like JOSE. As far as I know, none of them are incompatible with JSON.parse() or JSON.stringify().

[tjcrowder]

I think better to get this in and standarized now, rather than as a follow-on proposal. But not with a raw number string, with a number string with n suffix. Quoting myself from the es-discuss thread.

I agree with you and Anders that this should be sorted out now, not as a follow-on proposal. If it's left to later, people will supply their own BigInt.prototype.toJSON and cause themselves compatibility problems down-the-line.

Don't like the raw number in quotes ("conservative" option) at all. There should be some indication that this is a bigint, just as pattern matching tells us the default Date serialization is a date. This could be achieved with just the lower-case n at the end, as in some early examples in the github issue. And to support easy two-way, BigInt(string) should support the lower-case n at the end.

aa = 12345678901234567890n; // BigInt primitive
aa = JSON.stringify(aa);    // '"12345678901234567890n"'
aa = JSON.parse(aa);        // '12345678901234567890n'
aa = BigInt(aa);            // BigInt primitive

^\d+n$ isn't much of a pattern, but it's a pattern. (Earlier in this thread I suggested /BitInt(123)/ to match Microsoft's /Date(123)/ which is unambiguous and offers a path to extendibility, but Date already goes a different way...)

This parallels Date handling and seems a reasonable stop-gap until the next thing after JSON.

I don't like to say things like "X should" without offering to help where possible, so if I can help, ping me to tell me how.

[cyberphone]

@tjcrowder In this particular case tc39 is faced with a bunch of already established solutions so the choice(s) boils down to "best practices" rather than innovation or standardization.

[kaizhu256]

i also originally wanted @tjcrowder's slightly-modified form, but there was pushback from the champion on issue 160.

[kaizhu256]

i would be happy with either - anything is better than the current awful-behavior

[cyberphone]

If we stick to compatibility with existing practices (why shouldn't we?), a JSONNumber wrapper object as suggested by Michael Theriot on the es-discuss@mozilla.org list could be a way to reduce pushback to a minimum.

That is, the default/standard BigInt serialization method could be the "" model like for Date, while those who insist that JSON needs to be used to its fullest, can get their thing as well. The latter requires some changes to the ES6 JSON object but they seem fairly reasonable.

[cyberphone]

Replaced by: https://github.com/cyberphone/es6-bigint-json-support#summary-of-changes

Something along these lines could probably work:

Serializing:

// Standard mode
JSON.stringify({big: 555555555555555555555555555555n}); 

Expected result: '{"big":"555555555555555555555555555555"}'    

// RFC mode
// Note: this could be achieved by an option forcing mandatory JSON Number notation
JSON.stringify({big: 555555555555555555555555555555n}, 
  (k, v) => typeof v === 'bigint' ? JSONNumber(v) : v
);

Expected result: '{"big":555555555555555555555555555555}'  

Parsing:

// Standard mode
JSON.parse('{"big": "555555555555555555555555555555"}',
  (k, v) => k == 'big' ? BigInt(v) : v  
);

Expected result: {"big":555555555555555555555555555555n}    

// RFC mode
// Requires a new method (or option) which returns all numbers as JSONNumber
JSON.parse2('{"big": 555555555555555555555555555555}',
  (k, v) => typeof v === 'jsonnumber' ? k == 'big' ? v.getBigInt() : v.getNumber() : v
);

Expected result: {"big":555555555555555555555555555555n}    

The JSONNumber object would only hold a string in proper JSON Number notation.

[robjtede]

Looks like the conversation is sort of moving here so I'd like to chip in my latest reply in the es-discuss thread. https://esdiscuss.org/topic/json-support-for-bigint-in-chrome-v8#content-44

+1 on getting this sorted before stage 4

As people have said before, JSON already supports BigInts. We of course need to preserve backwards compatibility with JS and try not to break how other languages use JSON.

I think the best way to satisfy everyone is to give ourselves some flexibility going forwards and JSONifying BigInts as strings doesn’t allow JSON to do what it can do. That and it breaks the idea that JSON.parse(JSON.stringify(x)) is a deep clone.

So, flexibility: there are two options in my head for this.

1. An object passed to JSON.parse with options for how to deal with integers larger than Number can support. The current behaviour of misrepresenting them is the default. But other options could include converting to BigInts (or strings) if needed and always converting to BigInts (since they are non-interoperable currently). This approach allows the programmer to decide the behavior and plan for it in their code.

2. It seems like the already suggested JSONNumber object could offer the same flexibility and guarantees though. A valueOf that returns the misrepresented Number for interop with existing numbers. A toString that gives a well represented string conversion. (Im not 100% certain of the order these two functions are used in auto-casting. On mobile or else would check spec.) A toBigInt / unwrap (monad-ish terminology) to get the true value as and when the programmer wants.

Hope these ideas seem reasonable. Scrutiny welcome.

[MichaelTheriot]

I'll continue to push for a more compliant JSON.parse alternative being introduced (perhaps its own proposal?). Parsing a JSON number as a JS number has always been a convenient lie and now we are paying the toll. JSONNumber would also pave the way for parsing BigDecimal as well (userland or not). Anyone currently dealing with big numbers serialized in environments with arbitrary precision (Python 3) and without the flexibility to receive them as strings has ditched JSON.parse for a smarter userland solution.

As for serialization, BigInt should serialize just how a JSON number would (at least in a strict RFC mode). The fact that JSON.parse does not support it should be irrelevant, and a bigger sign JSONNumber parsing is needed. Not serializing at all is just avoiding the issue which is still present.

[cyberphone]

Parsing a JSON number as a JS number has always been a convenient lie and now we are paying the toll.

That's a bit of a simplification. There is no other typed information exchange format that uses a single notation for everything that smells like a number, be it a bit or BigDecimal. That's simply put a pretty broken idea. By constraining JSON Number = JS Number = IEEE Double Precision it works quite satisfactory anyway since it effectively boils down to a single data type.

There are (AFAIK) no IETF standards using JSON structures exploiting JSON Number outside of JS Number.

JSON.parse(JSON.stringify(x)) does not perform deep cloning for widely used data types like Date and "blobs" either.

I suggested introducing a JSON.parse2() for dealing with JSON in the way it was "intended", making the default behavior the only true issue.

[robjtede]

I should clarify, JSON.parse(JSON.stringify(x)) performs a deep clone for primitive types (including plain objects and arrays). Of course Date is more complicated than that. BigInt should not be treated in the same way as Date. JSON does not support Date objects; it does support BigInts though.

I don't think having both JSON.parse and JSON.parse2 is a good idea. An options object is the JS way for when different behavior is wanted (see something like the Intl API for example.) or else we could end up with JSON.parse3 for other data types (eg. BigDecimal) down the road.

[cyberphone]

The JSON.parse2() proposal is compatible with any other JSON Number addition, but an optional boolean to JSON.parse() can surely do the same job.

I believe developers will have to explicitly deal with non JS Numbers no matter what scheme we come up with. My "problem" with strict JSON is that on top of that, you introduce backward incompatibility with at least JS and with pretty limited gain. Neither Date or "blobs" seem to have been particularly hampered by not having a direct JSON counterpart.

There are some "unwanted side effects" as well: Oracle's JSON support in Java depends on BigDecimal as the core number type in order to deal with dynamic parsing. That doesn't sound like a great solution from a performance point of view. My RFC mode parse proposal doesn't actually convert JSON Number data at all; it puts that burden on the developer. How would your scheme work?

[cyberphone]

Until proven wrong (I'm not infallible 😏), I believe the only point of disagreement is what the default serialization of a BigInt should be, the rest are details of less importance.

[robjtede]

For stringifying:

const numbers = {
  a: 2,
  b: 40000000000000000000000n
}

// returns -> '{ "a": 2, "b": 40000000000000000000000 }'
// NOTE: no "n" suffix
JSON.stringify(numbers, null, null, {
  bigint: 'raw' // default setting
})

// returns -> '{ "a": 2, "b": '40000000000000000000000' }'
// NOTE: no "n" suffix
JSON.stringify(numbers, null, null, {
  bigint: 'string'
})

For parsing:

const numbers = '{ "a": 2, "b": 40000000000000000000000 }'

// returns -> {a: 2, b: 4e+22}
JSON.parse(numbers, null, {
  bigint: 'never' // default setting
})

// returns -> { a: 2, b: 40000000000000000000000n }
JSON.parse(numbers, null, {
  bigint: 'whenNeeded'
})

// returns -> { a: 2n, b: 40000000000000000000000n }
JSON.parse(numbers, null, {
  bigint: 'always'
})

The extra nulls are due to existing optional params I only just found out about.
This kinda makes me prefer the JSONNumber idea but it's not too unsightly.

[robjtede]

Granted this does mean the deep clone idea needs to be changed to:

JSON.parse(JSON.stringify(x), null, { bigint: 'whenNeeded' })

Another reason to support JSONNumber, for me.

[cyberphone]

Does 'whenNeeded' mean that JSON.parse() would return different JS data types depending on numeric value?

I wouldn't take this path. 'always' seems quite awkward as well, why would anybody want all numbers to be BigInt?

[robjtede]

@cyberphone basically, yes. It is a flag for the programmer that any numbers they expect could be large (think twitter IDs) might need a typeof x.id check.

I can't think of many use cases for 'always' either, it's just an option for cross property compatibility. Eg. (x.small + x.large) being guaranteed to be correct and not throw.

[cyberphone]

@robjtede There seems to be multiple issues here. I don't see how you can use a future a BigNumber type at the same time as BigInt since JSON (unlike every other typed data interchange format in existence...) doesn't tell you what's behind a number.

Therefore I stick to my assertion that JSONNumber is the "missing link" in order to support arbitrary JSON Numbers. Then people can certainly argue to death which mode that should be the default although that's not really paramount for success.

[kaizhu256]

@robjtede your suggestion for modifying JSON.parse is too open-ended and high-maintennance. nobody benefits from the tooling-instability caused by a JSON.parse thats under constant maintennance and revisions by tc39 (which your suggestion would open the floodgates to).

[robjtede]

@kaizhu256 Which part is open ended?

Whichever way it works, I'm highly in favor of a literal string-ification (no quotes or "n" suffix) by default with an option to have quotes.

I believe that it should be as transparent as possible to start serializing and parsing BigInts in JS but flexible enough to fit most needs without having to resort to userland libraries. This is a primitive after all, not a class (like Date).

[robjtede]

Following the JSONNumber line, how does this look for a Number/BigInt wrapper returned from parse in terms of backwards compatibility?

interface JSONNumber {
  isBig(): boolean              // true if too large for Number
  getBigInt(): BigInt           // well represented number as BigInt
  getNumber(): Number           // mis-represented number
  valueOf(): Number             // mis-represented number
  [Symbol.toPrimitive](hint): Number | BigInt
  toString(): String            // well represented string
}

[cyberphone]

@robjtede I believe you have to use a delayed number conversion scheme to get anywhere. That is, JSON.parse() with suitable a option would only return a JSONNumber holding a string with proper JSON Number syntax. Then the developer can through suitable methods find out what it actually is (I have no idea how to do that), although most people would as in my example rather use keys to designate content.

Maybe that's what you meant as well with the wrapper? I just lack the string data. It would in my take rather be a class than an interface.

[kaizhu256]

Whichever way it works, I'm highly in favor of a literal string-ification (no quotes or "n" suffix) by default with an option to have quotes.

@robjtede you are beating a dead-horse. literal-stringification was already discussed in [closed-issue] https://github.com/tc39/proposal-bigint/issues/24 by some of the best-and-brightest javascript-experts, and ended in failure.

the failure was largely due to people who do not want the JSON-spec changed, because they believe the instability will do more harm-than-good to industry. i happen to be one of those people, and its unlikely you'll have an argument that would change my mind, now or for the forseeable future.

[closed-issue] https://github.com/tc39/proposal-bigint/issues/24, however, did not discuss Date-like quoted-stringification, which is why this thread was started. it seems to be the optimal, logical-outcome in a reality where you have no choice but to live with people who do not want the JSON-spec changed.

[MichaelTheriot]

JSONNumber should at least include methods to return significant digits and exponent (both as BigInts).

[robjtede]

@kaizhu256 I also do not want the JSON spec changed, nor do I see why it would need to be for what I'm suggesting? Large integers (and indeed decimals) are already valid JSON.

[jakobkummerow]

A fundamental issue is that you cannot just "look at the value" to uniquely determine what type it has: • {"foo": 2} => 2 is obviously a valid Number, but 2n is also a perfectly valid BigInt. • {"foo": 9007199254740994} => this is bigger than Number.MAX_SAFE_INTEGER, but nevertheless can accurately be represented as a Number. Does it count as "too big for a Number"? Neither the the bigint: "whenNeeded" nor the JSONNumber::IsBig() approach solves this problem.

Formatting to string doesn't help either, as has been pointed out before: • {"foo": "2n"} => is this supposed to be a string, or a serialized BigInt?

So in order to faithfully serialize and deserialize, explicit type information has to be transmitted somehow. There are several options for this, e.g.: • implicit knowledge: the deserializing program simply knows that the deserialized object's .foo property must be of type BigInt and can create it as such • an explicit annotation in the serialized format: {"foo": {"type": "BigInt", "value": "2n"}} • your idea here

Any suggestion to somehow auto-detect BigInts can't possibly work in all situations.

[robjtede]

@jakobkummerow There surely can't be scripts out there relying on receiving numbers above MAX_SAFE_INTEGER that just happen to be represented accurately.

What "whenNeeded" or "isBig" do is allow the programmer to make the choice of what type they want to handle. When no choice is made, it does nothing different from the current behaviour which is to parse the number and represent it as well as Number will allow.

[bakkot]

There surely can't be scripts out there relying on receiving numbers above MAX_SAFE_INTEGER that just happen to be represented accurately

Oh jeeze, yes there can.

[MichaelTheriot]

Any suggestion to somehow auto-detect BigInts can't possibly work in all situations.

Convert to Number, compare non-scientific string representation with input? Not that I think detecting BigInt should be a goal of JSONNumber.

[cyberphone]

Just a clarification: neither {"big":555555555555555555555} nor {"big":"555555555555555555555"} changes or violates the JSON specification in any way.

6. Numbers . . This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision.

The question I believe we are sort of wrestling with is the default mode.

Apart from the interoperability issue above, the fact thatJSON.parse() simply cannot be updated to automagically deal with BigInt without breaking lots of code, makes the quoting method more logical as default also for JSON.stringify(). Essentially you end-up with a scheme that mimics Date.

[robjtede]

Yes. I get that argument. JSON.parse cannot magically tell by itself. The programmer can on a use-by-use basis when explicitly telling it to, though.

If not, why not? These are the two why's I can see.

For .stringify: Why does JSON-ifying as a literal (without the n) break existing code (of JS and other languages)? For .parse: Why do the potential solutions we've discussed break existing JS?

I will hapilly drop the case if these get answered, concretely.

[cyberphone]

@robjtede We are still only talking about the default mode, right?

Since '{"big":5555555555555555555555}' is by default incompatible (loses precision) with JSON.parse(), even after the proposed update, it seems less appetizing generating such data by default.

The proposed JSONNumber option gives developers all the power to use JSON as it was "intended".

However, the default for JSON.stringify() is certainly not a "life-or-death" issue for me, I just like the simple rule that a primitive that's not null, true, false, or Number, ends up as a string. I have yet to see a downside with such an arrangement since JSON messages typically are 95% made of strings anyway.

[robjtede]

So you're saying that if there's going to be a way to (de)serialize without quotes, we can't have it happen by default on stringify and not parse? If so, that makes sense as the basis for an argument for serializing with quotes by default as long as there exists a way to do it without quotes (which I believe is not currently possible).

[jakobkummerow]

For .stringify: Why does JSON-ifying as a literal (without the n) break existing code (of JS and other languages)?

Defining .stringify semantics for BigInts, by definition, does not break any existing code which doesn't stringify any BigInts, so I'm not sure why you are asking this question. A problem is that two objects o1 = {foo: 2} and o2 = {foo: 2n} would serialize to the same JSON string '{"foo": 2}', losing the type information. Since Numbers and BigInts are handled very differently in JavaScript (in particular, they cannot be mixed), this makes it very hard to deserialize such a string in a useful way. There is no way to restore the type information; you'd have to transmit it via some other channel. Disallowing JSON-serialization of BigInts is mostly a way to warn people that the round-trip isn't going to work. It is the only place where such a warning can be generated, because this is where BigInt type information is available. (Also, define "break existing code". If you have a producer of data sending '{..., "id": 123, ...}' JSON data to a consumer, and you switch the producer to internally using BigInts for id fields, and the consumer's code is not under your control and expects id values to be unique, then two BigInt values that round to the same Number value will break the consumer. The currently spec'ed exception warns you about this footgun.)

For .parse: Why do the potential solutions we've discussed break existing JS?

Depends on which solution you're talking about. You can trivially define .parse semantics that don't break existing non-BigInt-using code, e.g. by specifying that .parse always returns Numbers. But that doesn't help anyone who wants to serialize and deserialize BigInts. If .parse gives you JSONNumber objects (or any other new kind of thing) instead of Numbers, that's going to break existing code that expects Numbers. (Overriding valueOf and toString mitigates some cases, but not all.) If .parse gives you BigInts in some cases, that's going to break existing code that doesn't expect BigInts. (And even for newly written code that would be hard to work with, see above.)

[robjtede]

A problem is that two objects o1 = {foo: 2} and o2 = {foo: 2n} would serialize to the same JSON string '{"foo": 2}', losing the type information.

Sure but OPs proposal of stringifying with quotes has the same issues JSON.stringify({foo: '2'}) === JSON.stringify({foo: 2n}) in such a case. BigInts are more like numbers than strings so I'd argue more information is lost in that way.

Since Numbers and BigInts are handled very differently in JavaScript.

Yes, but not as differently as Numbers and Strings. This would be where the options object or intermediate JSONNumber comes in, to let the programmer explicitly define the behavior. Serializing as strings doesn't allow that without knowing the specific field to be converted back to BigInt.

Disallowing JSON-serialization of BigInts is mostly a way to warn people that the round-trip isn't going to work. It is the only place where such a warning can be generated, because this is where BigInt type information is available.

Sure, and that's why we're talking about it here, to see if there's a solution 🙂

If you have a producer of data sending '{..., "id": 123, ...}' JSON data to a consumer, and you switch the producer to internally using BigInts for id fields, and the consumer's code is not under your control and expects id values to be unique, then two BigInt values that round to the same Number value will break the consumer.

Producers are already sending BigInts (see Twitter API) along with stringified versions and "breaking consumers". I've encountered this problem myself when trying to cache tweets based with primary keys based on the id field. I'd have loved a way to tell parse to give me a bigint back.

You can trivially define .parse semantics that don't break existing non-BigInt-using code, e.g. by specifying that .parse always returns Numbers.

Yes, this should be default going forward.

But that doesn't help anyone who wants to serialize and deserialize BigInts.

With the possibility for explicit conditional conversion it does.

If .parse gives you JSONNumber objects (or any other new kind of thing) instead of Numbers, that's going to break existing code that expects Numbers. (Overriding valueOf and toString mitigates some cases, but not all.)

Instances of number have 6 prototype methods.

Number.prototype.toExponential()
Number.prototype.toFixed()
Number.prototype.toLocaleString()
Number.prototype.toPrecision()
Number.prototype.toString()
Number.prototype.valueOf()

Just clone them onto the interface of JSONNumber? Looking at the spec, that and defining valueOf as a lossy conversion and toString as a lossless string would have the same semantics for existing code.

If you have an example of code that would break, please share.

If .parse gives you BigInts in some cases, that's going to break existing code that doesn't expect BigInts.

Bug-free JS scripts aren't using unsafe numbers so an explicit conversion simply puts this decision in the programmers hands.

[ljharb]

Twitter's solution is solved by their documentation - changing a specific API's contract is very different than breaking the web.

[robjtede]

The Twitter API is just a well-known example.

I think I've said all I can say for now. I need you to show me how these potential solutions will break the web. Concretely. You all seem to think this isn't fully backwards compatible, please show me why; I get that anything less is not acceptable. I want to understand.

[jakobkummerow]

Let me reiterate the core point:

From looking at a sequence of digits, you cannot tell whether it is a Number or a BigInt.

Both small (2) and large (9007199254740994) digit sequences are examples of this. A magic .parse implementation can't tell, JSONNumber.isBig() can't tell, "bigInt": "whenNeeded" can't tell. There is no "explicit conditional conversion" solution for this. (And as you point out, the other proposal to serialize as string with quotes has the same problem.)

The only way to restore BigInts is to know, somehow, which serialized field should restore back to BigInt. But if deserializing code has to know that anyway, then no further spec changes are needed:

BigInt.prototype.toJSON = function() { return this.toString(); }
function MyObj() { 
  this.bigint = 2n;
  this.number = 2;
}
function Deserialize(str) {
  var result = JSON.parse(str);
  result.bigint = BigInt(result.bigint);
  return result;
}
var roundtripped = Deserialize(JSON.stringify(new MyObj()));
roundtripped.bigint += 1n;  // works
roundtripped.number += 1;  // works

The "options object" idea does essentially this; except it is coarse-grained and only allows you to express "all digit sequences are numbers" or "all digit sequences are bigints", so it can't handle objects with some properties of each kind. (The "whenNeeded" option is not an option because it can't possibly work, e.g. in the example above it doesn't stand a chance to do the correct thing.) The "JSONNumber" idea gives programmers a case-by-case choice, but it is no better than the above, and it would change the behavior of existing code: even with all the prototype mucking you can throw at it, typeof would behave differently. The general assumption is that changed behavior means broken code.

If you want to discuss any further points in more detail, please make a specific proposal; don't just refer to "any of the proposals above".

[robjtede]

Thank you for setting out a well-rounded argument.

The typeof problem definitely throws out the JSONNumber idea.

I agree there’s no perfect or magic solution for this. It’s a frustrating position to be in not using JSON to its fullest advantage. Strings feel like such an unsatisfactory and half-baked solution. But a solution nonetheless if you’re willing to make those per-property decisions.

[cyberphone]

@robjtede

The typeof problem definitely throws out the JSONNumber idea.

Well, using an option for parsing, typeof x === 'jsonnumber' would be a valid construct and let developers deal with arbitrary JSON Numbers which is what I thought was the request. https://github.com/tc39/proposal-bigint/issues/162#issuecomment-408679884 shows an example of that.

But a solution nonetheless if you’re willing to make those per-property decisions.

I'm not aware of any other solution for parsing since the root of the problem is an overloaded data type which (pardon for repeating myself...) no other information exchange format uses. Well, if we switched to BigNumber as the JS Number type everything would run smooth but that's hardly an option.