New(ish) plugin: logstash-input-mqtt

[widarlein]

Hi!

I created a basic input plugin https://github.com/widarlein/logstash-input-mqtt, for mqtt which actually has gotten some traction, even though I just created it for my own purposes. It is poorly documented, but it is simple enough to understand. My original intention was to publish it as a proper plugin, but I have neither had the time or motivation to properly finish the docs or to maintain it. To top it all off, I'm not even a ruby developer.

I got an offer from @ph to move it to the logstash-plugins repo, which I would love. It would be nice if it could be helpful to other people :)

Is this still a possibility? Can I handover the plugin in its entirety to you guys? This has been nagging at my conscience, but better late than never, I guess.

I see that there has been some forks made that has among other things updated versions and dependencies and someone pushed it to rubygems. I'm not sure how to handle this practically?

[unixbigot]

@widarlein I'm willing to take over maintenance (despite being a ruby monkey-see-monkey-do level noob). I'm having trouble getting it working --- I can talk via mosquitto_sub to my mqtt server, but my logstash system never does anything.

Can you shoot me a working example? I can at least write some doco.

[lbrutti]

Hello @unixbigot , did you manage to make the plugin work? I'm trying with these conf but with no success:

input {
        mqtt{
                topic=> "test"
        }
}

filter {

}

output {
        elasticsearch {
                hosts => "http://localhost:9200"
                index => "mqtt-test-index"
        }
        stdout {}
}

Thanks, Lorenzo

[unixbigot]

I did get it working. I had to update the plugin you can find my changes here https://github.com/unixbigot/logstash-input-mqtt/tree/dockerise https://github.com/unixbigot/logstash-input-mqtt/tree/dockerise

— Christopher Biggs <christopher@biggs.id.au mailto:christopher@biggs.id.au> Brisbane, Australia. +61-407-722-799 @unixbigot

On 10 Oct 2017, at 8:24 am, lbrutti notifications@github.com wrote:

Hello @unixbigot https://github.com/unixbigot , did you manage to make the plugin work? I'm trying with these conf but with no success:

input { mqtt{ topic=> "test" } }

filter {

}

output { elasticsearch { hosts => "http://localhost:9200" index => "mqtt-test-index" } stdout {} } Thanks, Lorenzo

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/elastic/logstash/issues/6804#issuecomment-335306536, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFaX6K3_kYgslMSdjFRTc0WtIqPbRPIks5sqp0TgaJpZM4MVirc.

[lbrutti]

Thank you @unixbigot, I'll try again tonight ;D

[lbrutti]

Hello @unixbigot , i managed to build and install your plugin fork (i had to modify the .gemspec file dependecies as follows: # Gem dependencies s.add_runtime_dependency "logstash-core", '>= 1.4.0', '< *5.99*'

But, testing with mosquitto_pub on localhost, i stille get this error: [ERROR] 2017-10-11 01:00:03.506 [[main]<mqtt] pipeline - A plugin had an unrecoverable error. Will restart this plugin. Plugin: <LogStash::Inputs::Mqtt topic=>"test", id=>"510cfdad5b68aded39d0a875432c993078fddb75-1", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_d3936e9f-4f44-42fb-bfc2-8a929e884412", enable_metric=>true, charset=>"UTF-8">, mqttHost=>"localhost", port=>1883, ssl=>false, client_id=>"logstash-mqtt-inputopx1", clean_session=>true, qos=>0> Error: Direct event field references (i.e. event['field']) have been disabled in favor of using event get and set methods (e.g. event.get('field')). Please consult the Logstash 5.0 breaking changes documentation for more details.

[lbrutti]

I solved the problem by fixing the direct reference with getter and setter. in the next days i will send you PR :)

[vesa666]

I'm about to decide what to with my mqtt data.. so I'd like to try the setter - getter enhanced version. Has it been published somewhere already ?

[riker09]

I'm looking for a solution to store my sensor data long term and have a nice dashboard for statistics pr0n (hello, kibana). This looks promising, but documentation is scarce.