use with node.js websocket?

[rpedela]

I am trying to simply connect right now, but it is not working due to ES returning status code 200 instead of 101. It very easily could be that I am using the wrong URL, port, etc. I am not even sure how to debug this, so any pointers would be helpful.

Install

npm install websocket

Code

var WebSocketClient = require('websocket').client;

var ws = new WebSocketClient();
ws.connect('ws://localhost:9400/_stats');
ws.on('connect', function(connection) {
    console.log('connected!');
});
ws.on('connectFailed', function(err) {
    console.log('ERROR: ' + err);
});

Output

ERROR: Server responded with a non-101 status: 200
Response Headers Follow:
content-type: application/json; charset=UTF-8
content-length: 97

[rpedela]

Based on https://tools.ietf.org/html/rfc6455 it looks like maybe the connect handshake is not handled properly. So is that because of a bug or do I have the wrong URL?

[jprante]

It should be ws://localhost:9400/websocket, WS negotiation is only invoked when path is websocket.

A _stats endpoint for WebSocket is not implemented BTW.

[rpedela]

Yeah I just found https://github.com/jprante/elasticsearch-client-websocket. Thanks for the fast response, and I will let you know if I have any questions. It looks like you have implemented all the necessary bulk commands which is why I want to use this. So that is great!

[jprante]

In the tests, there are some small examples of connecting to the websocket transport (HelloWorld, indexing,...)

[rpedela]

Okay, I got it to mostly work. It seems to index documents correctly, but I do not receive an acknowledgement from the server. The Get API retrieves the indexed documents though so I know the server is happy. There is nothing suspicious in the server log either. Do you have any pointers on how to debug this? Where to look in the plugin code? Is there a way to enable verbose debug output?

Below is the Node.js code. BTW this all works fine with the Java tests.

var WebSocketClient = require('websocket').client;

function getIndexMessage(docId) {

    var message = {
        type: 'index',
        data: {
            index: 'twitter',
            type: 'tweet',
            id: docId.toString(),
            data: {
                user: 'rpedela',
                post_date: new Date().toISOString(),
                message: 'test websocket'
            }
        }
    };

    return JSON.stringify(message);
}

function getFlushMessage() {

    var message = {
        type: 'flush'
    };

    return JSON.stringify(message);
}

var ws = new WebSocketClient();

ws.connect('ws://localhost:9400/websocket');

ws.on('connect', function(connection) {
    console.log('connected!');

    connection.on('close', function(reasonCode, description) {
        console.log('connection closed: ' + reasonCode + ' - ' + description);
    });

    connection.on('error', function(err) {
        console.log('connection error: ' + err.stack);
    });

    connection.on('frame', function(frame) {
        console.log('frame: ' + JSON.stringify(frame));
    }); 

    connection.on('message', function(message) {
        console.log('message: ' + JSON.stringify(message));
    });

    console.log('send index messages');
    connection.send(getIndexMessage(1));
    connection.send(getIndexMessage(2));

    console.log('send flush message');
    connection.send(getFlushMessage());
});

ws.on('connectFailed', function(err) {
    console.log('connection failed: ' + err.stack);
});

[rpedela]

I should add that errors are received correctly. So it is like the client is waiting for something in the index/flush case.

[rpedela]

Alright more investigation. In order for a response, BulkHandler.execute(channel) must be called. This seems to happen only when some bulk indexing threshold is reached. It does not happen when flush is called. So what happens is that the last set of bulk operations are not acknowledged even after a flush. I will fork and make a pull request.

[rpedela]

This also shows up in BulkTest. 2 acknowledge messages are returned, but it should be 3.

[rpedela]

node.js client now works fine.

[gaving]

@rpedela Hey, I don't suppose you are aware of any changes which would have broken your node example?

Trying myself and while I can connect and send messages, I can't seem to receive anything back.

Elasticsearch 1.4.2 / latest build of transport-websocket

[rpedela]

No, I do not. I have switched to using the official ES Node client. Sorry.

[gaving]

Ah, no worries. Are you now just polling with the official client?

Basically trying to get some subscription/push going on but until https://github.com/elasticsearch/elasticsearch/issues/1242 is sorted it looks like I'm running out of options!