brianc / node-pg-types

Type parsing for node-postgres
271 stars 56 forks source link

Reuse postgres-bytea #130

Closed markddrake closed 3 years ago

markddrake commented 3 years ago

Trying to use pg with Vertica

Vertica supports 3 Binary Types

Binary ID 116 VarBinary ID 17 LongVarBinary ID 117

So after digging around the code it appears that PG uses the function

    parseByteAArray = function (value) {
    console.log(value)
      return array.parse(value, parseByteA)
    }   

To convert BYTEA columns to BUFFER

I tried adding the following code to my class

const array = require('postgres-array')
const parseByteA = require('postgres-bytea')

    parseByteAArray = function (value) {
    console.log(value)
      return array.parse(value, parseByteA)
    }   

          types.setTypeParser(17,this.parseByteAArray)
      types.setTypeParser(116,this.parseByteAArray)
      types.setTypeParser(117,this.parseByteAArray)

When I run I see the following value logged as for any column of type BINARY, VARBINARY or LONG VARBINARY

\000\001\002\003\004

But I get an empty array as the result of array.parse

Can anyone give me some pointers.

If I comment out the Parser for 17 (Varbinary or ByteA in Postgres) then this column works and is returned as BUFFER

charmander commented 3 years ago

bytea (OID 17) shouldn’t be using the parser for a bytea[] (parseByteAArray), so the line you’re commenting out should be removed entirely. What gets logged for the arrays?

markddrake commented 3 years ago

Looks like I am misunderstanding the default behavior for OID 17. I thought the call to register(17, parseByteA) in textParsers.js was configuring the handler for OID 17, and was trying to add similar mappings for 116 and 117

In Vertica there are 3 'Binary' datatypes

VARBINARY OID 17 LONG VARBINARY OID 117 BINARY OID 116

When I try and use PG with Vertica my VARBINARY columns are returned as BUFFER (as expected), but my LONG VARBINARY and BINARY columns are returned as strings. What I was trying to do was to force OID 116 and OID 117 to be treated the same as OID 17.

What I am looking for is what would be the correct 'setTypeParser' calls to get 116 and 117 to go through the same code path as 17. I dont intend configuring 17 at all since that already works.

markddrake commented 3 years ago

To answer your question the value is a string in the form \000\001\002\003\004 and the output is an empty array [].

charmander commented 3 years ago

\000\001\002\003\004 is a valid text representation of a bytea, but not an array. Are you sure you’re associating the right log message with the right field?

markddrake commented 3 years ago

I see where I went wrong.. I was reading the code for the "array of bytea (byte array)" as the code for bytea. I don't need the array parser...

Looking at it again I think all I need is

types.setTypeParser(116,parseByteA)
types.setTypeParser(117,parseByteA)

Will test later today

markddrake commented 3 years ago

That fixed it.. Thx