Luidog / node-red-contrib-filemaker

A Node Red FileMaker module. This module uses the FileMaker Data API to connect with a FileMaker server.
32 stars 7 forks source link

UUIDNumbers are converted to scientific format #61

Open stathisaska opened 2 years ago

stathisaska commented 2 years ago

Created the following test to get data using the 'Get one record' node.

Screenshot 2022-01-31 at 18 59 00

The result I got, converted UUIDNumber into scientific format (not this thing again 😓)

            "fieldData": {
                "_ID": 3.070884973408834e+57,
                "_IDf_Environment": 4.770391498406892e+56,
                "g__Result": "",
                "g__ResultContainer": "",
                "z__CreatedBy": "Stathis Askaridis",
                "z__CreationTimestamp": "01/26/2022 20:07:09",
                "z__ModificationTimestamp": "01/26/2022 20:38:11",
                "z__ModifiedBy": "Stathis Askaridis",
            }

Expected result would be to get the UUIDNumber in non-scientific format, as it already occurs using the Data API get one record


"fieldData": {
                    "_ID": 3070884973408833919385016168376332397025719676107210536517,
                    "_IDf_Environment": 477039149840689202950165380468774030219968005600531762389,
                    "g__Result": "",
                    "g__ResultContainer": "",
                    "z__CreatedBy": "Stathis Askaridis",
                    "z__CreationTimestamp": "01/26/2022 20:07:09",
                    "z__ModificationTimestamp": "01/26/2022 20:38:11",
                    "z__ModifiedBy": "Stathis Askaridis",
                }

Both calls were built on Postman, the first one was to {nodejsserver.com}/test the second one using the FileMaker Data API 'Get one record' call, after of course authenticating

jwillinghalpern commented 2 years ago

Hey, the main issue here is that JS naturally truncates long numbers when parsing JSON. Here's the result of parsing your exact data in the Chrome console:

Screen Shot 2022-02-01 at 1 22 07 PM

Can you add a text version of your field to your database (a good option to avoid index bloat is an unstored calc field like _IDasText)? That avoids the problem and provides a response that's useable within JavaScript.

It is possible to convert long numbers to strings automatically within fms-api-client, but I think the problem is better addressed on your end since converting numbers to strings without consent in the library sounds kinda icky.

stathisaska commented 2 years ago

Hi @jwillinghalpern

Thank you for contributing in this issue.

I ‘d really rather find a solution within the fms-api-client, as creating text versions for every primary key and foreign keys would be an unnecessary clutter in the table definitions

jwillinghalpern commented 2 years ago

I hear you, I really wish FileMaker hadn't introduced UUIDNumber in the first place. Another possible solution on your end would be to change all the id fields to text fields, which is considerably less cluttered. For what it's worth, I've done various tests trying to produce some of the supposed performance benefits of UUIDNumber and have found exactly zero instances where performance was improved. The only difference I've seen is a small reduction in index size. So if text is acceptable, you might consider switching.

We'll think about solutions in fms-api-client. One idea is to have a flag somewhere to convertLongNumbersToStrings to maintain backward compatibility. It might also be acceptable to automatically convert long nums to strings since they're not particularly useful in truncated e notation, but I'm not thrilled about deviating from the DAPI's default behavior of returning long numbers without quotes.

stathisaska commented 2 years ago

Thank you @jwillinghalpern for your fast reply !

I ‘d rather use unstored calcs and clutter the table definitions or ditch record cross-table uniqueness and going with a serial number, than sacrificing the 10-50% performance number indexes deliver !

I understand your dilemma, the flag seems like a great option.

jwillinghalpern commented 2 years ago

10-50% performance boost?? I've not seen any improvement in my (perhaps flawed) tests. Can you shoot me a line about how you found that, maybe via DM on the Claris Community site (my nickname there is joshwilling)?

stathisaska commented 2 years ago

👍🏻