Implement MQTT-Binding

[GoogleCodeExporter]

Client implementation is available at eclipse http://www.eclipse.org/paho/ 
project.

Specification is available at 
http://www.redbooks.ibm.com/redbooks/pdfs/sg248054.pdf

Additional information:

* 
http://www.computerworld.com/s/article/9238726/The_Internet_of_Things_gets_a_pro
tocol_it_s_called_MQTT 
* 
http://www.heise.de/ix/meldung/M2M-Protokoll-MQTT-soll-Internet-Standard-werden-
1852990.html / 
* 
http://news.techworld.com/networking/3444491/oasis-mqtt-to-be-the-protocol-for-t
he-internet-of-things/

Original issue reported on code.google.com by teichsta on 7 May 2013 at 3:44

[GoogleCodeExporter]

see 
http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.java.git/tree/org.eclipse.pa
ho.client.mqttv3/src/org/eclipse/paho/client/mqttv3/MqttClient.java for the 
relevant Client implementation

Original comment by teichsta on 7 May 2013 at 8:23

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Samplecode available at: 
https://github.com/eclipse/paho.mqtt.java/blob/master/org.eclipse.paho.sample.mq
ttv3app/src/org/eclipse/paho/sample/mqttv3app/SampleAsyncCallBack.java

Original comment by teichsta on 31 May 2013 at 2:17

[GoogleCodeExporter]

Original comment by teichsta on 8 Jul 2013 at 6:26

• Changed state: Started

[GoogleCodeExporter]

Hi Thomas, I was thinking about building an MQTT binding myself, but just 
noticed you already started.  How are you getting along with this?  Do you need 
any help implementing this binding or someone to bounce ideas off? I'm quite 
keen on starting to use an MQTT binding myself..

Original comment by davy.van...@gmail.com on 16 Jul 2013 at 6:55

[GoogleCodeExporter]

Hi Davy,

great to know, that you're still aboard! How is your SmartHome installation 
going?

Yep, i started ... and i really just started ;-) I've attached my current 
dev-state which is not more than a collection of possibilities how to get 
things done with the existing MQTT-client. Since I've started it as a copy of 
the HttpBinding there are still lot's of references to the other code, 
apologies for that ;-)

Furthermore MQTT shouldn't be a binding rather than a transport bundle (under 
the namespace org.openhab.io.transport.mqtt). As you know MQTT defines no 
semantics so the concrete bindings have to provide the appropriate set of 
topics to be applied to the specific items. But should not care about any 
connection/handling issues. The transport should support sync and async 
messaging.

Does that makes sense?

Regards,

Thomas E.-E.

Original comment by teichsta on 16 Jul 2013 at 7:09

Attachments:

• org.openhab.binding.mqtt.zip

[GoogleCodeExporter]

Hi Thomas,

Well, my SmartHome is progressing much slower than I would like, but that
is probably a common problem.  I've been working on a binding for the
Nikobus automation system and it's almost ready, but I'm currently a bit
stuck whilst waiting for a new TTL to Serial converter to arrive.  My
current USB-Serial converter keeps crashing my raspberry pi as soon as I
plug it in.  I'm still hoping to get the binding ready before the openHab
1.3 release, although I'm not sure when that release will be...

In the mean time, I started to investigate how I can convert my doorbell to
an MQTT doorbell and have my raspberry send MQTT events whenever water or
gas is consumed.  The next step is to get openhab to accept these events,
which brings me to the MQTT binding.

Shouldn't there be both an MQTT transport bundle and a MQTT (generic)
binding bundle?

The MQTT transport bundle, could be similar to the serial bundle and
provide the core libraries (i.e. paho) and some standard connection
handling methods.  When really complex MQTT messaging is required, bindings
could use this bundle for the transport in a similar way as the serial
bundle.

The (generic) MQTT binding could be similar to the HTTP binding and allow
for MQTT integration without having to create a custom binding for every
device you want to integrate via MQTT.
For example, if I would like to receive a state change/command on my
doorbell when it is pressed and I would like to configure this as:

Item itemName  {mqtt="<direction>[<broker-alias>:<topic>:<command or
transformation rule>]"}

Where :
The broker alias would refer to a broker host/port combination defined in
the openhab.cfg file.

where the <command or transformation rule> could be:
- not provided : in this case, it's assumed the content of the message
contains the value/command to use
- a command : always use this 'hard coded' command whenever 'any' event
arrives on this topic
- a transformation rule : a rule which transforms the message content to an
openhab command/state
e.g.
Item doorbell {mqtt="<[mybroker:/myHome/doorbell:XSLT(doorbell.xslt)]"}
where the content would be a valid openHab command or the transformation
rule would change it into a valid openhab command or state (e.g. ON).

Another example could be for receiving data events:
e.g.
Number temperature {mqtt="<[publicweatherservice:/mycity/temperature]}

Outbound messages could work in a similar way:
Item itemName {mqtt="<direction>[<command>:<broker alias>:<topic>:<content
or transformation rule>]" }
e.g.
Switch mySwitch {mqtt=">[ON:mybroker:/mytopic:1] >[OFF:mybroker:/mytopic:0]
"}

Original comment by davy.van...@gmail.com on 16 Jul 2013 at 8:31

[GoogleCodeExporter]

> 1.3 release, although I'm not sure when that release will be...

Somewhere around beginning of September

> The next step is to get openhab to accept these events, which brings 
> me to the MQTT binding.

great idea :-)

> Shouldn't there be both an MQTT transport bundle and a MQTT (generic)
> binding bundle?

agreed!

> The MQTT transport bundle, could be similar to the serial bundle

yep, that was the intention.

> The (generic) MQTT binding could be similar to the HTTP binding and allow
> for MQTT integration without having to create a custom binding for every
> device you want to integrate via MQTT.

That sounds reasonable to me, too.

> Where :
> The broker alias would refer to a broker host/port combination defined in
> the openhab.cfg file.

and username password i would assume?!

> - not provided : in this case, it's assumed the content of the message
> contains the value/command to use

or in outbound case the item state

> - a command : always use this 'hard coded' command whenever 'any' event
> arrives on this topic

Which commands are you thinking about? Could you give an example?

Original comment by teichsta on 16 Jul 2013 at 8:58

[GoogleCodeExporter]

Yep, for SSL MQTT connections, that would need to be added.

Standard openHab commands, for example the following config:

Item doorbell {mqtt="<[mybroker:/myHome/doorbell:ON]"}

would always send the openHab ON command to the doorbell Item, whenever a
message is received on the topic, regardless of the content of the message.

Did you see the last part of my message with regards to the proposed
outbound message config?  I noticed that a few lines of my reply got cut
off in the issue
<https://code.google.com/p/openhab/issues/detail?id=281> comments
(luckily
it was only the salutation), so I was wondering if something may also have
been cut off from my email.

Original comment by davy.van...@gmail.com on 16 Jul 2013 at 9:24

[GoogleCodeExporter]

agreed! So we would treat ON as kind of "static transformation"

yep, saw it

Original comment by teichsta on 16 Jul 2013 at 9:32

[GoogleCodeExporter]

I started a wiki page for the MQTT binding: 
https://code.google.com/p/openhab/wiki/MQTTBinding. We could use this as 'the 
specification' for the development...

All feedback is welcome.

Original comment by davy.van...@gmail.com on 17 Jul 2013 at 7:02

[GoogleCodeExporter]

cool, will read it later ...

Original comment by teichsta on 18 Jul 2013 at 4:46

[GoogleCodeExporter]

I started to make changes for the mqtt binding. Work is progressing and there 
is already not much left over from the original dev-state structure after 
refactoring to split the binding into a separate transport and binding bundle. 
As I progress and run into issues (or new ideas), I'll keep the wiki page up to 
date, so that it's clear in which direction I'm heading with my changes..
With a bit of luck, I should be able to complete the binding before the end of 
this month.

Original comment by davy.van...@gmail.com on 21 Jul 2013 at 4:50

[GoogleCodeExporter]

Davy, thank you very much for driving this issue! Saw your changes to the wiki 
page as well!

Fingers crossed ;-)

Original comment by teichsta on 24 Jul 2013 at 11:56

[GoogleCodeExporter]

Great! <thumbs up>

Looking forward for this 
i have an electricity meter (flukso) which i want to read it true mqtt.

Regards,
Joey

Original comment by SJo...@gmail.com on 27 Jul 2013 at 6:15

[GoogleCodeExporter]

Davy, saw your "initial" checkin of the binding, transport, test … WOW. I 
assume the outcome of the review protocol will be _very_ short :-)

Original comment by teichsta on 1 Aug 2013 at 11:31

[GoogleCodeExporter]

Thanks! Yep, I made good progress on the binding :-)  There are still some 
parts which are completely untested (like SSL connections), but for now I'm 
struggling with Tycho.  The tests work fine when run from inside eclipse, but 
when running a maven build the tests fail.  The test bundle is a fragment to 
the binding bundle which has a dependency on the transport bundle. In the test 
bundle I'm using a class from the transport bundle, and when running in a maven 
build, Tycho just doesn't want to resolve that dependency, so the test fails on 
a ClassNotFoundException.  If you have seen this issue before (and know a 
solution) that would be helpful..

Original comment by davy.van...@gmail.com on 1 Aug 2013 at 11:46

[GoogleCodeExporter]

Does it help if you also add the dependency to the manifest of the fragment?

Original comment by kai.openhab on 1 Aug 2013 at 4:13

[GoogleCodeExporter]

> Does it help if you also add the dependency to the manifest of the fragment?

no, unfortunately not :-/

Original comment by teichsta on 1 Aug 2013 at 4:14

[GoogleCodeExporter]

I just pushed a fix into my clone.  There was an error in the build.properties 
for the transport bundle.  

Original comment by davy.van...@gmail.com on 1 Aug 2013 at 5:54

[GoogleCodeExporter]

argh - overlocked that again and again, sorry! Thanks for fixing this yourself 
…

Original comment by teichsta on 1 Aug 2013 at 7:01

[GoogleCodeExporter]

I've pushed some more build fixes. Now the mqtt binding should build ok and 
also work when deployed in an openHab instance, rather than only working 
through the launch configuration in eclipse :-)

Original comment by davy.van...@gmail.com on 1 Aug 2013 at 10:58

[GoogleCodeExporter]

Hi Thomas, the MQTT stuff is ready for review. Please also have a look at the 
change I made in the StringType.  This change allows a behaviour closer to 
regular java strings where any object can be converted to a string. This 
behaviour is required when an event bus binding is set up and a string item may 
receive both number values and string values in the mqtt message.

Original comment by davy.van...@gmail.com on 10 Aug 2013 at 10:30

• Changed state: ReadyForReview

[GoogleCodeExporter]

Would it be a good idea to add an org.openhab.persistence.mqtt bundle?
In addition to what is available now, it would allow storing of timeseries 
through MQTT. 

The topic and message format could be made configurable in the cfg file, e.g.
mqtt-persistence.broker=<brokername>
mqtt-persistence.topic=/v2/feeds/${item}.csv
mqtt-persistence.payload=${state},${timestamp}

Original comment by davy.van...@gmail.com on 13 Aug 2013 at 6:13

[GoogleCodeExporter]

> Would it be a good idea to add an org.openhab.persistence.mqtt bundle?

yes, definitely! You could look into the existing implementations or the newly 
created service for AirVantage (see Benjamins Blogpost 
http://airvantage.github.io/blog/2013/08/08/openhab-with-airvantage/).

Thanks,

Thomas E.-E.

Original comment by teichsta on 13 Aug 2013 at 8:20

[GoogleCodeExporter]

But please open a new issue for it! This one is already done :-)

Original comment by kai.openhab on 13 Aug 2013 at 8:26

[GoogleCodeExporter]

Original comment by teichsta on 13 Aug 2013 at 4:04

• Added labels: Version-1.3.0
• Removed labels: Version-later

[GoogleCodeExporter]

merged to default branch (see 
http://code.google.com/p/openhab/source/detail?r=94b1950a2ff473a3cb33e1279c58c4f
565fa13c1) - BIG thanks to Davy for this great contribution!

Original comment by teichsta on 25 Aug 2013 at 8:17

• Changed state: Implemented

[GoogleCodeExporter]

Davy,

fyi, i made a little change to AbstractMqttMessagePubSub. I removed the call to 
FrameworkUtil to fetch the BundleContext and rather added an Activator which 
gives access to the BundleContext directly since this is more consistent with 
all other bindings. I hope you don't mind!

Best, Thomas E.-E.

Original comment by teichsta on 25 Aug 2013 at 8:36

[GoogleCodeExporter]

Activators are so old school...
Nope, I don't mind :-)

Original comment by davy.van...@gmail.com on 25 Aug 2013 at 8:58

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Are they? What is the "new school" style?

Original comment by teichsta on 25 Aug 2013 at 8:59

[GoogleCodeExporter]

OK, maybe not completely old school yet :-).  Anyway, I haven't used it yet and 
it's probably not relevant for openHAB, but my new school vote would go to 
http://felix.apache.org/site/ipojo-faq.html#iPOJOFAQ-Injectingthebundlecontextin
aPOJO

Original comment by davy.van...@gmail.com on 25 Aug 2013 at 9:11

[GoogleCodeExporter]

Davy,

out of curiosity does the binding (and the related bundles) really require 1.7 
execution environment?

Cheers,

Thomas E.-E.

Original comment by teichsta on 28 Aug 2013 at 8:17

[GoogleCodeExporter]

The previous paho jar that I was using was compiled with 1.7, so that did
require a 1.7 execution environment.
But since we replaced that jar with the new 0.4.0 lib, which seems to be
compiled using a 1.6 jdk, a 1.6 execution environment should work too.

Original comment by davy.van...@gmail.com on 28 Aug 2013 at 9:02

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

ok, i'll change the bundles to use the 1.6 execution environment

Original comment by teichsta on 6 Nov 2013 at 10:09