search

jawj / zapatos

Zero-abstraction Postgres for TypeScript: a non-ORM database library
https://jawj.github.io/zapatos/
Other
1.3k stars 46 forks source link

Buffer/Blob as string? #88

Closed delaneyj closed 3 years ago

delaneyj commented 3 years ago

I have a blob column and I'd think it would be a Buffer type, not a string.

If I encode with base64 it will get inserted but its he base64, not the contents. Try with the hex and \\\\x${buf.toString('hex')} and get errors with encoding. The docs don't seem to cover this situation or I'm missing something.

jawj commented 3 years ago

OK, I'll look into this. Basic first question: is that the right number of backslashes before the x?

delaneyj commented 3 years ago

I honestly don't know, coming from Go where its simple a []byte interface. Was going off of https://stackoverflow.com/a/14408194 but I'd hope you could just hand it a Buffer as it makes the most sense personally. I tried multiple versions and nothing worked so wasn't clear what the intent was.

jawj commented 3 years ago

Can you post the code that's giving you trouble, and the error message that describes that trouble? For me, this all seems to work as expected: I can pass either a hex string or Buffer to db.insert and both seem to work fine:

await db.insert('mytable', [
  { byteaColumn: Buffer.from('abc')}
  { byteaColumn: `\\x${Buffer.from('abc').toString('hex')}` }, 
]).run(pool);
delaneyj commented 3 years ago

I'm wondering if its because I'm actually using CockroachDB and zapatos is not getting all the info it needs

jawj commented 3 years ago

Oh, interesting. I've not tried Zapatos with CockroachDB.

That said, I don't think anything Zapatos does changes this situation compared to using the pg package directly.

jawj commented 3 years ago

Looks like CockroachDB has their own syntax for byte array literals: https://www.cockroachlabs.com/docs/v21.1/sql-constants#hexadecimal-encoded-byte-array-literals

So I think this is something that probably needs fixing by using cockroachdb instead of pg as the layer beneath Zapatos. I may have a look at some point at how feasible that is.

jawj commented 3 years ago

I did a bit of testing with CockroachDB and found a more serious problem. Did this not cause issues for you too?

jawj commented 3 years ago

I've now rewritten Zapatos slightly to work around that problem, and actually I wasn't able to reproduce your issue with bytea columns on CockroachDB. Do let me know if you're still having trouble.