Websocket Channels

[myrulezzz]

Hi does the plugin support websocket channels? The usecase is to create a websocket server which receives data from multiple kafka consumers. I wanted to use websocket channels to streaming consumer data one topic per channel. However i was not able to define the websocket channel throw the plugin? Im i missing something or the plugin is not compatible with websocket channels? Any idea how to implement the scenario i briefly explained?

[myrulezzz]

@miguel91it any idea? Does the plugin support websocket channels?

[beriberikix]

Hi @myrulezzz,

Can explain what a websocket channel is? I'm not familiar with that term. Is that a language or framework-specific use of WebSockets?

[myrulezzz]

@beriberikix Here is an example of websocket server with 3 websocket channels and a kafka consumer that reads dara from 3 topics and send them to the appropriate channel. const WebSocket = require('ws'); const Kafka = require('node-rdkafka');

const consumer = new Kafka.KafkaConsumer({ 'group.id': 'my_group', 'metadata.broker.list': 'kafka1:9092,kafka2:9092', 'enable.auto.commit': true }, {});

consumer.connect();

consumer.on('ready', function() { consumer.subscribe(['topic1', 'topic2', 'topic3']);

consumer.consume(); });

consumer.on('data', function(data) { console.log(data.value.toString()); if (data.topic === 'topic1') { channel1.clients.forEach(function each(client) { if (client.readyState === WebSocket.OPEN) { client.send(data.value.toString()); } }); } else if (data.topic === 'topic2') { channel2.clients.forEach(function each(client) { if (client.readyState === WebSocket.OPEN) { client.send(data.value.toString()); } }); } else if (data.topic === 'topic3') { channel3.clients.forEach(function each(client) { if (client.readyState === WebSocket.OPEN) { client.send(data.value.toString()); } }); } });

const channel1 = new WebSocket.Server({ port: 8888 }); const channel2 = new WebSocket.Server({ port: 8889 }); const channel3 = new WebSocket.Server({ port: 8890 });

channel1.on('connection', function connection(ws) { console.log('WebSocket connected to channel 1');

ws.on('close', function close() { console.log('WebSocket disconnected from channel 1'); }); });

channel2.on('connection', function connection(ws) { console.log('WebSocket connected to channel 2');

ws.on('close', function close() { console.log('WebSocket disconnected from channel 2'); }); });

channel3.on('connection', function connection(ws) { console.log('WebSocket connected to channel 3');

ws.on('close', function close() { console.log('WebSocket disconnected from channel 3'); }); }); If i create a node.js websocket client would be sometging like this to subscribe to the channel:

const WebSocket = require('ws');

const ws = new WebSocket('ws://your-websocket-server');

ws.on('open', function open() { // Send a message to the server to subscribe to channel1 ws.send(JSON.stringify({type: 'subscribe', channel: 'channel1'})); });

ws.on('message', function incoming(data) { // Print out the data that was received on channel1 console.log(data); }); Is there a way to pass the channel in grafana websocket plugin?

[beriberikix]

Thanks for the information! If I understand the code correctly, what you were referring to was two things:

• Kafka Channels
• Interfacing with Kafka Channels over WebSockets

The plugin doesn't explicitly have support for Kafka. It may be possible to extend the plugin, though we don't have any plans to do so at this time. But it is open source and contributions are welcome!

[myrulezzz]

@beriberikix Channels are part of websocket.Are not related to kafka. So now in my grafana i created a websocket server which retrieves $.colum1 , $.colum2 and $.colum3 from the server.This columns are coming in to the websocket server from a kafka consumer. Now if i send to websocket server $.colum4 and $.colum5 which is a different json message than the previous one, i cannot use the same websocket server in grafana as on the table fhat i am filtering the first 3 columns i will get received error data because the message had $.colum5 and $.colum6.Is the plugin compatible to receive multiple different messages and use them in different panels or is one plugin per one type of message?

[chireia]

Hey @myrulezzz, we didn't try to use different messages into one single Grafana data source. We wouldn't doing that to avoid "empty" data points for panels that aren't receiving any messages. What I'd do is to create multiple datasources with a specific path to listen only for the type of response that would be predictable, with that you could panels selecting specific datasources.

Does that make any sense?

[miguel91it]

@myrulezzz, i read your sample code and i understood what you are calling by websocket channels are, in fact, websocket servers, for that you Kafka can send stream from the topics direct to the websocket clients connected to your webbsocket channels.

If i'm right, i believe you can create a single instance of the Golioth WebSocket Plugin for each websocket channel you have, because your websocket channels are different servers. Our plugin cannot connect to multiple servers by instance.

When you create this channel:

const channel1 = new WebSocket.Server({ port: 8888 });

i think it means your machine is running a server for websockets and listening at port 8888, and the full address to reach it (in the case you are running both the Grafana and your code at the same machine) would be ws://localhost:8888. You need to use it as the Host when configuring the plugin.

We wait for your feedback.