brianc / node-pg-types

Type parsing for node-postgres
267 stars 54 forks source link

Parser not working for bigint values #118

Closed budarin closed 4 years ago

budarin commented 4 years ago

What I do

import pg from 'pg';

pg.types.setTypeParser(pg.types.builtins.INT8, (x) => {
    console.log('converting', x);
    return BigInt(x);
});
...
const pool = new pg.Pool({...});
const result = pool.query(...);
console.log(result);

The query:

select json_build_object(
        'tz', now()::timestamptz(0),
        'bigserial', '9223372036854775807'::BIGINT,
        'integer', 2147483647,
        'numeric', 23.5141,
        'decimal', 23.5141::DECIMAL(10,2),
        'real', '1e+23'::REAL,
        'boolean', true,
        'uuid', public.gen_random_uuid()
  )

then run code and have a such output

{
    "command": "SELECT",
    "rowCount": 1,
    "oid": null,
    "rows": [

          {
            "tz": "2020-06-19T18:40:25+00:00",
            "real": 1e+23,
            "uuid": "5cc0cec3-bdfd-4a4a-8653-6d6864efe06c",
            "boolean": true,
            "decimal": 23.51,
            "integer": 2147483647,
            "numeric": 23.5141,
            "bigserial": 9223372036854776000
          }

    ],
    "fields": [
      {
        "name": "select",
        "tableID": 0,
        "columnID": 0,
        "dataTypeID": 3802,
        "dataTypeSize": -1,
        "dataTypeModifier": -1,
        "format": "text"
      }
    ],
    "_parsers": [
      null
    ],
    "_types": {
      "_types": {
        "arrayParser": {},
        "builtins": {
          "BOOL": 16,
           ....
          "REGROLE": 4096
        }
      },
      "text": {},
      "binary": {}
    },
    "RowCtor": null,
    "rowAsArray": false
  }

what I can see

    "_parsers": [
      null
    ],

My parser is not set and bigint value converted to int Something went wrong :(

bendrucker commented 4 years ago

Took me a second to notice that you're calling json_build_object. Note that the field data type ID is jsonb:

https://github.com/brianc/node-pg-types/blob/b96a92e0871c0f0611bc3fd97c4d12d239f6fffa/lib/textParsers.js#L157

JSON does not specify size limits on numbers so you risk losing data during JSON.parse. If you change your query to select '9223372036854775807'::BIGINT you'll get a BigInt result.

bendrucker commented 4 years ago

Some info on this:

https://golb.hplar.ch/2019/01/js-bigint-json.html https://itnext.io/lets-talk-about-big-numbers-in-javascript-b0a2c8edb170

There's obviously a lot of backwards compatibility concerns in adding BigInt support to the JSON methods so if you really do need to parse JSON with bigints included in JS you may have trouble finding good libraries.

budarin commented 4 years ago

@bendrucker thanks!