Support for BigInt

[codingnuclei]

Describe the bug

I am building a geolocation service with Dynamodb which uses S2 to create geohashes. These are very large numbers. When trying to save an Entity with a large number it errors: Error thrown by DynamoDB client: \"Number 5226365420151572000 is greater than Number.MAX_SAFE_INTEGER. Use BigInt.

However electrodb does not seem to support bigInt as a primitive. So i use the escape hatch type of any however now i get:

Invalid composite attribute definition: Composite attributes must be one of the following: string, number, boolean, enum

As the error suggests if I use a bigint then i cant use it to create a composite key :(

Any suggestions?

ElectroDB Version Specify the version of ElectroDB you are using 2.14.1

Expected behavior Be able to use a BigInt to save a large number to the Db as a number

Errors

Error thrown by DynamoDB client: \"Number 5226365420151572000 is greater than Number.MAX_SAFE_INTEGER. Use BigInt.

Invalid composite attribute definition: Composite attributes must be one of the following: string, number, boolean, enum

Additional context Add any other context about the problem here.

[zirkelc]

I'm not sure if the DynamoDB SDK for JavaScript supports BigInt, but as an easy workaround you could store the BigInt as String and use get and set to convert between the types. Here's an example:

Playground

[codingnuclei]

Hi thanks for the quick response :)

So I had a native solution (i.e one without electrodb) and it seemed to support BigInt.

Error thrown by DynamoDB client: \"Number 5226365420151572000 is greater than Number.MAX_SAFE_INTEGER. Use BigInt.

This error which is thrown by the client suggests the SDK does support it.

I had the exact same idea as yourself regards the work around however not 100% confident in it as as part of a geospatial search you do a min max query (see img below). Looks like when Dynamodb does this for strings it does not behave as a number would :(

Thanks

(fyi the playground link doesn't seem to work)

[zirkelc]

Oops, I updated the playground link: Playground

Do you mean you get different results for between-query when the values in sk are stored as number vs. string?

[codingnuclei]

Oops, I updated the playground link: Playground

Do you mean you get different results for between-query when the values in sk are stored as number vs. string?

Yea exactly - seems like there is different behaviour:

Working with number:

And then the same query as was with string example on the previous comments:

As you can see it fails - as with string it gave a result

[zirkelc]

I think you have a typo in your string example. The first screenshot contains the values:

5226365420151572000
5226365420151572001

The second screenshot has a shorter max value. It misses a zero in the last three digits:

5226365420151572000
522636542015157201

It should be 2001 and not 201.

[codingnuclei]

ah yes - sorry :(

Ok looks like storing as a string will work (will build it out fully and run some tests).

I think in the long term it would be really neat if this lib could support large numbers...

I found some usual docs for how currently lib-dynamodb handle different primitives

https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-dynamodb/README.md#marshalling-input-and-unmarshalling-response-data https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-dynamodb/README.md#large-numbers-and-numbervalue

Thanks for the support and fantastic project! 💯

[codingnuclei]

As an aside this answer on SO shows how strings are compared in Dynamodb using lexicographic order:

https://stackoverflow.com/questions/58942403/how-does-the-between-operator-work-in-dynamodb-with-strings

So my usecase i might get away with string comparison when dealing with large numbers, however other use cases might not :(

[zirkelc]

To be honest, I would prefer to work with string than bigint if possible. If you have to do arithmetic, then of course you have to use them.

Bigint is not JSON serialisable, so you may have to convert them to string anyway. For example, returning a bigint from a Lambda function caused an error (https://github.com/aws-amplify/amplify-category-api/issues/733) the last time I checked (might be possible now).

[codingnuclei]

Closing as no real use for electrodb to support bigints... yet 😄