pubnub / dart

PubNub Dart SDK
Other
27 stars 15 forks source link

Flutter Web timetoken big int in javascript - Precision loss in JavaScript when representing large numbers #110

Closed RicharC293 closed 1 year ago

RicharC293 commented 1 year ago

Pubnub uses timestamp in nanoseconds (1 billionth of a second) For example timetoken = 16819214431624417

I encountered an issue while working with large numbers in JavaScript. Specifically, when attempting to represent the number '16819214431624417', JavaScript is representing it as '16819214431624416', which results in a loss of precision.

After researching this issue, I discovered that this is due to the fact that '16819214431624417' is larger than the safe maximum value that can be accurately represented in JavaScript using the "Number" data type, which is (2^53 - 1) or 9007199254740991.

While there are libraries available, such as BigNumber.js, that can provide an arbitrary-precision number data type for performing calculations with extremely large or small numbers, it would be helpful if there was a built-in solution in JavaScript to handle these types of scenarios.

I believe that this issue is important to address, as it can cause unexpected behavior in JavaScript applications that rely on accurate representations of large numbers.

This issue cause a infinity loop here (Only in Javascript - Flutter Web)

  Future<void> fetch() async {
    var _cursor = from;
    _messages = [];

    do {
      var result = await defaultFlow(
          keyset: _keyset,
          core: _core,
          params: FetchHistoryParams(_keyset, _channel.name,
              reverse: true,
              count: 100,
              start: _cursor,
              end: to,
              includeToken: true),
          serialize: (object, [_]) => FetchHistoryResult.fromJson(object));

      _cursor = result.endTimetoken;
      print("CURSOR: $_cursor");
      _messages.addAll(await Future.wait(result.messages.map((message) async {
        if (_keyset.cipherKey != null) {
          message['message'] = await _core.parser.decode(_core.crypto
              .decrypt(_keyset.cipherKey!, message['message'] as String));
        }
        print(message);
        return BaseMessage(
          publishedAt: Timetoken(BigInt.from(message['timetoken'])),
          content: message['message'],
          originalMessage: message,
        );
      })));
    } while (_cursor.value != BigInt.from(0));
  }
}

Version package pubnub: ^4.2.1

are commented 1 year ago

I believe the issue stems from incorrect conversion to an int here.

RicharC293 commented 1 year ago

Notice that if you print this number 16819214431624417 in chrome console this print as 16819214431624416.

are commented 1 year ago

Precisely! Using BigInts for timetoken was supposed to mitigate that, but the endpoint here needs to return a string instead of a number, so it can be properly serialized without losing precision.

jjsebastianfuertes commented 1 year ago

@are This bug is affecting our production environment. Do you have an ETA for this fix? or do you suggest an alternative to use meanwhile