code: EPIPE

[montenegrojd]

Hi there, this is a great component. However after trying few queries I get this error: {"code":"EPIPE"}. It looks like it got disconnected from the server. Have you seen this error before? Is there a way to force a reconnect from you code?

Here is a sample from my code

exports.data = function(req, res) {
    var redshiftClient = require('../redshift.js');
    var util = require('util');
    var from = req.query.from;
    var to = req.query.to;
    console.log('req:' + from + ' ' + to);

    var sql = "select ... from metrics where date between '%s' and '%s' group by 1,2 order by 1,3 desc,2";
    var query = util.format(sql, from, to);

    // options is an optional object with one property so far {raw: true} returns  
    // just the data from redshift. {raw: false} returns the data with the pg object 
    redshiftClient.query(query, {
        raw: true
    }, callback);

    function callback(error, data) {
        res.json({
            error: error,
            report: data
        });
    }
};

Thanks

[dmanjunath]

Hey @montenegrojd, would you happen to have more information about a stacktrace? I've seen redshift close all server connections a few times when it's undergoing maintenance, usually at night time, but that's usually rare, like one a month max. How often are you getting disconnected?

In terms of a way to reconnect, there is potentially a different approach you could take. Inside connection.js, I use connection pooling to connect to redshift. Instead what you could do is not use connection pooling, but instead create a client instance in query.js every time you query, so it creates the redshift connection, queries for data, and closes the connection. That means you'll get a fresh connection you can use once and then kill it, but setting that connection each time can take up a few milliseconds. If we can't figure out the root cause of the disconnects and you want to try this approach i can try to help you with that, but I'd keep this as a last resort.

[dmanjunath]

@montenegrojd I also just noticed in your callback you have callback(result, data). In the query function, the callback format is callback(error, data). Could that be causing any problem?

[montenegrojd]

Thanks for you prompt response. It may be related to the connection pool. I have tried another library https://github.com/brianc/node-postgres. Not using the connection pool the issue is gone.

I have updated the code on my answer to match the callback format.

[dmanjunath]

@montenegrojd Glad to hear that solved your problem. Under the hood I use node-postgres too and I'm going to do some long term connection testing to see if I can reproduce this problem with connection pooling. Thanks for the heads up. And thanks for using this module too.

[bdipak]

Hi,

I am also facing the same issue. Looks like the connection gets disconnected after few seconds. I am facing this consistently. After few seconds (~30 sec), I start getting this error.

Is there any solution for this ?

2016-05-12 17:22:18.202 ERROR app - { [Error: This socket has been ended by the other party] code: 'EPIPE' } Error: This socket has been ended by the other party at Socket.writeAfterFIN as write at Connection.query (/node_modules/node-redshift/node_modules/pg/lib/connection.js:189:15) at Query.submit (/node_modules/node-redshift/node_modules/pg/lib/query.js:115:16) at Client._pulseQueryQueue (/node_modules/node-redshift/node_modules/pg/lib/client.js:298:24) at Client.query (/node_modules/node-redshift/node_modules/pg/lib/client.js:326:8) at runQuery (/node_modules/node-redshift/lib/query.js:37:15) at Redshift.query (/node_modules/node-redshift/lib/query.js:33:8)