sehrope
commented
6 years ago I'm able to reproduce this and I think it's related to the internal client query queue. I haven't totally figured out the cause but I suspect it's from the FEBE messages for the stream.close() being sent after the messages for the next query. If you change the repro code to wait for the completion of the stream.close(...) by giving it a callback and waiting for it's execution then the error does not occur:
// npm install pg pg-query-stream async-main
const pg = require('pg');
const { Client } = pg;
const QueryStream = require('pg-query-stream');
const main = require('async-main').default;
function log(message, ...args) {
console.log(`%s ${message}`, new Date().toISOString(), ...args);
}
function streamToPromise(stream) {
return new Promise((resolve, reject) => {
stream.on('end', () => resolve());
stream.on('error', reject);
});
}
function streamClose(stream) {
return new Promise((resolve) => {
stream.close(resolve);
});
}
main(async () => {
log('Started');
const client = new Client(process.env.DATABASE_URL);
try {
log('Connecting');
await client.connect();
log('Connected');
let stream;
try {
const query = new QueryStream('SELECT $1::uuid AS x', ['bad data']);
stream = client.query(query);
stream.on('data', (data) => {
log('Received data: %j', data);
});
await streamToPromise(stream);
} catch (e) {
log('Error streaming query: %s', e);
} finally {
if (stream) {
log('Closing stream');
// Replace with `stream.close()` to get the error
await streamClose(stream);
log('After close stream');
}
}
log('Before execute new query');
const result = await client.query('SELECT 1 AS x');
log('Result: %j', result.rows);
} finally {
await client.end();
}
});Output when waiting for stream.close():
2018-06-13T11:48:29.269Z Started
2018-06-13T11:48:29.272Z Connecting
2018-06-13T11:48:29.279Z Connected
2018-06-13T11:48:29.281Z Error streaming query: error: invalid input syntax for uuid: "bad data"
2018-06-13T11:48:29.281Z Closing stream
2018-06-13T11:48:29.324Z After close stream
2018-06-13T11:48:29.324Z Before execute new query
2018-06-13T11:48:29.326Z Result: [{"x":1}]Replacing the await streamClose(...) with a regular stream.close() gets the same error as you.
There's probably a fix that could be added to the driver to better handle this situation but knex should also be waiting for the close() event prior to continuing.
BasixKOR
alxndrsn
nathanjcochran
lognaturel
brianc
Hi, I have encountered a really bizarre issue when using
knexwithpgand query streams. A parameterized QueryStream which fails in the parsing stage (e.g. because of a column type / value type mismatch) breaks subsequent queries on the same connection - either by:result.rows(this is much harder to reproduce and will not be covered here, but it occurs in one of our apps usingknex)Below are two ways to reproduce this issue, one using
knexand another simplified to barepg(which simulates howknexinteracts withpg).Test conditions:
CREATE TABLE events (id UUID);pg@7.4.3pg-query-stream@1.1.1knex@0.14.6(optional, for the knex-enabled test)PostgreSQL 9.6.7(this probably does not matter)What should happen:
What actually happens
TypeError: Cannot read property 'handleRowDescription' of null, as described at the beginning of this bug reportIt does not crash if:
idparam is a valid UUID string like8b0fb8cb-b145-4e41-a847-dfe2f1a47aa1(the second query then also runs OK)rawStream.close();from our stream error handler (but knex.js does it automatically)Overall, I think some state machine inside
pg.Clientbreaks, andactiveQueryis set to null because of it, but I have not been able to pin-point exactly where or why it happens. It seems to be connected to closing the ReadableStream returned fromclient.query().This is especially hard to debug when using
pgwith a pool (such as knex.js's), because the error may go unnoticed until the same client object is re-acquired from the pool and used; this is why I opted for single-connection knex / pg examples above.