search

hivejs / hive-core

Other
0 stars 0 forks source link

Server crashes saving newlines to postgres #1

Open MrTin opened 8 years ago

MrTin commented 8 years ago

What happens?

The server crashes when newlines are being sent. It looks like it's related to the ORM adapter. invalid input syntax for type bytea. I tried using the waterline-postgresql adapter too but I could not even get that one to properly install. I also forcefully updated the waterline and sails-postgresql packages to latest versions but it still crashed when sending newlines.

I'm wondering if there's a module that needs to be activated within Postgres and what version you are running where it works?

Environment

This is the stacktrace:

hive_1      | [2016-08-30 09:23:15.249] [DEBUG] http - GET /stream/?_primuscb=LRQmdrd&EIO=3&transport=polling&t=LRQmdsW&b64=1&sid=QyTSqoaOaiYhX2l7AAAA
hive_1      | [2016-08-30 09:23:15.257] [DEBUG] http - POST /stream/?_primuscb=LRQmdrd&EIO=3&transport=polling&t=LRQmdsa&b64=1&sid=QyTSqoaOaiYhX2l7AAAA
hive_1      | [2016-08-30 09:23:15.321] [DEBUG] http - POST /stream/?_primuscb=LRQmdrd&EIO=3&transport=polling&t=LRQmdtm&b64=1&sid=QyTSqoaOaiYhX2l7AAAA
hive_1      | [2016-08-30 09:23:15.370] [DEBUG] http - GET /static/build/codemirror.js
hive_1      | [2016-08-30 09:23:15.416] [DEBUG] http - GET /static/codemirror/mode/meta.js
hive_1      | <- _write: ["requestInit"]
hive_1      | <- _write: ["authenticate","eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjoyLCJpYXQiOjE0NzI1NDg5OTR9.3iaZsk-bWdygOn-_cRCGDoePKH3H7C8PzmnhmKQtx9Y"]
hive_1      | <- _write: ["requestInit"]
hive_1      | Authorization for document:read { id: '1' }
hive_1      | Authorization for document:read { id: '1' }
hive_1      | -> ["init",{"contents":"testar lite","edit":"{\"parent\":\"38z6kg4celq82lw5y852q1fk\",\"id\":\"357ag01fnvk0l8t6o4lhjq8\",\"cs\":\"[8,\\\"ite\\\"]\"}"}]
hive_1      | <- _write: ["edit","{\"parent\":\"357ag01fnvk0l8t6o4lhjq8\",\"id\":\"2aj8zkveqo02mweyo53zlsw\",\"cs\":\"[11,\\\"\\\\n\\\"]\"}"]
hive_1      | Authorization for document:change { id: '1' }
hive_1      | receiveEdit {"parent":"357ag01fnvk0l8t6o4lhjq8","id":"2aj8zkveqo02mweyo53zlsw","cs":"[11,\"\\n\"]"}
hive_1      | Document: apply edit Edit {
hive_1      |   ottype:
hive_1      |    { name: 'text',
hive_1      |      uri: 'http://sharejs.org/types/textv1',
hive_1      |      create: [Function],
hive_1      |      normalize: [Function],
hive_1      |      apply: [Function],
hive_1      |      transform: [Function],
hive_1      |      compose: [Function],
hive_1      |      transformSelection: [Function],
hive_1      |      selectionEq: [Function],
hive_1      |      api: { [Function: api] provides: [Object] } },
hive_1      |   id: '2aj8zkveqo02mweyo53zlsw',
hive_1      |   changeset: [ 11, '\n' ],
hive_1      |   parent: '357ag01fnvk0l8t6o4lhjq8' }
hive_1      |
hive_1      | events.js:160
hive_1      |       throw er; // Unhandled 'error' event
hive_1      |       ^
hive_1      | Error (E_UNKNOWN) :: Encountered an unexpected error
hive_1      | error: invalid input syntax for type bytea
hive_1      |     at Connection.parseE (/app/node_modules/sails-postgresql/node_modules/pg/lib/connection.js:539:11)
hive_1      |     at Connection.parseMessage (/app/node_modules/sails-postgresql/node_modules/pg/lib/connection.js:366:17)
hive_1      |     at Socket.<anonymous> (/app/node_modules/sails-postgresql/node_modules/pg/lib/connection.js:105:22)
hive_1      |     at emitOne (events.js:96:13)
hive_1      |     at Socket.emit (events.js:188:7)
hive_1      |     at readableAddChunk (_stream_readable.js:177:18)
hive_1      |     at Socket.Readable.push (_stream_readable.js:135:10)
hive_1      |     at TCP.onread (net.js:542:20)
marcelklehr commented 8 years ago

I haven't tested thoroughly on postgres, yet, so this is a good catch! The bug you referenced seems like it describes the problem. In order to get up and running I recommend using MySQL.

No doubt you will ask why I'm using binary data types here, though. And the answer is that I'm hoping to be able to cram more data into the db column than with simple text data types: Sadly waterline doesn't (to my knowledge) allow specifying the expected size of a column, so we get overflows with larger documents :/ This would also be a good thing to investigate, because I'm not convinced binary data types solve this problem, long-term-wise.

marcelklehr commented 8 years ago

Apparently they allow you to specify the size, now. \o/

See https://github.com/hivejs/hive/issues/108

Also, I prefer keeping all issues in one place, since that makes organizing (assigning them to a milestone etc.) them easier. Perhaps one day we'll use our own bug tracker, but until then https://github.com/hivejs/hive/issues should do.

MrTin commented 8 years ago

varchar and text are just aliases in Postgres. In Mysql, varchar and text are handled the same way by the Innodb engine, only difference is really that varchar supports indexation and text does not.

Unless we need an index on this (which I suppose not), I think the best solution would be to use atextfield for better support and, as time goes and if anyone experiences performance issues, I would optimize to each database accordingly.

MrTin commented 8 years ago

Will post in the main repo in the future! Sorry about that :)