ESP8266 as mqtt broker.

[rohit-rcrohit7]

Is there any instruction on how to make esp8266 itself act as a MQTT broker?

I am not talking about creating MQTT broker in the laptop.(example -mosquitto). I want to create the broker in esp8266 itself, and read in a blog someone did it, but found no instruction on that ever.

does someone have any idea?

[gicho]

It is not impossible but it does not make much sense. The broker needs some resources like CPU, memory and number of TCP connections (one for each client), maybe non-volatile storage (for persistent messages), configuration for user access rights and many other details. At the moment there is no implementation of a broker that could run (or be easily ported) to the ESP. But as I said it does not make much sense - because the broker is the interconnection between clients (publisher, subscribers and both). Still publishing something to the broker is not enough to make full application - you need another client to consume the data (subscriber). In practice you will need to store (log) the data for processing later, to visualize, to control from browser, smart device and so on. It is hard to imagine an use case where all nodes are only ESP based. Maybe isolated home automation system where one node is an HMI (lcd with touch) and others are sensors, door locks, etc. could run like that. But in most cases you (or you customers) would need internet access to this system - for monitoring, remote control and so on. Exactly here ESP will fail - exposing ESP to the wild wild web is not recommended for security reasons. Also the ESP is not efficient as a router - while some clever people implemented NAT on it (with station + AP mode) it is limited in terms of performance. So my recommendation is to define a detailed use case - with all desired future improvements the you can forsee. This will help you to find out which precise pattern to use. If you wish to go on with a broker on the ESP check the eclipse paho mqtt packet libraries - those might be a starting point.

2017-01-07 11:07 GMT+02:00 thorburn1 notifications@github.com:

Is there any instruction on how to make esp8266 itself act as a MQTT server?

I am not talking about creating MQTT broker in the laptop.(example -mosquitto). I want to create the broker in esp8266 itself, and read in a blog someone did it, but found no instruction on that ever.

does someone have any idea?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/esp8266/Arduino/issues/2839, or mute the thread https://github.com/notifications/unsubscribe-auth/ALTjCRD2fN9vZKEXy7R9lyVZJSw_G_s1ks5rP1XcgaJpZM4LdX9N .

[rohit-rcrohit7]

Thanks alot , even I understand your concern you mentioned, but i think to som extent it might work . GO through the below links , someone does claim to work his way out and squeeze most out of the chip , Posting the links below, https://myesp8266.blogspot.in/2016/02/mqtt-broker-on-esp8266.html https://myesp8266.blogspot.in/2016/03/mqtt-broker-on-esp8266-2.html https://myesp8266.blogspot.in/2016/04/mqtt-broker-on-esp8266-3.html

[gicho]

Yes, I saw that - impressive but I don't think it is practical. I don't see the source code there. Still you can try to contact him and discuss. The sweet part about MQTT (and standards in general) is that you can develop your application with mosquitto as a broker (on remote machine) and use it later with broker on ESP.

2017-01-07 11:47 GMT+02:00 thorburn1 notifications@github.com:

Thanks alot , even I understand your concern you mentioned, but i think to som extent it might work . GO through the below links , someone does claim to work his way out and squeeze most out of the chip , Posting the links below, https://myesp8266.blogspot.in/2016/02/mqtt-broker-on-esp8266.html https://myesp8266.blogspot.in/2016/03/mqtt-broker-on-esp8266-2.html https://myesp8266.blogspot.in/2016/04/mqtt-broker-on-esp8266-3.html

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/esp8266/Arduino/issues/2839#issuecomment-271073814, or mute the thread https://github.com/notifications/unsubscribe-auth/ALTjCVgMUqdhXUfszMb4fZIDAYTrWfz9ks5rP18bgaJpZM4LdX9N .

[rohit-rcrohit7]

Yes, I have done that already :), but the blog is really promising if it does work as he says so, too bad he didn't want to share the code.

[hunnimonstr]

I too am interested in this, my particular application is a distributed brewery control system using a number of esp8266s only. each one will be connected to a single sensor or relay/actuator for the most part with a couple of dedicated s/w pids with temp sensors and ssrs attached to gpio pins , some data will be logged to an external iot host mainly temperature measurements and power outputs from s/w pids.

originally i conceived the plan using targeted udp packets within a statically addressed subnet, however the more i read around the subject of the esp8266 and the toolkits available such as Sonoff-Tasmota/esppeasy. it seems sensible to not reinvent the wheel and use the tools already tried and tested.

However including a wan server to execute messaging is a bit ott and could introduce unnecessary delays, especially when executing a time critical function such as turning off the mains water supply when a specified mass of water has filled a vessel, as when the scales hit the target mass the water supply MUST be shut off Then, any delay is a catastrophic system failure.

Then it seems wasteful to employ a puter solely for a message broker, I could run something on my lappy during a brew to facilitate the brewday. But at the moment a purely esp solution is appealing.

within the brewery system at any time there will be no more thane 3 x inputs publishing messages, and no more than 2 subscribers awaiting trigger messages. its a fairly straightforward proceedure involving 1) filling a kettle, Flow sensor publishing solainoid valve subscribing 2) heating the strike liquor, Pid s/w on an esp will read temps and apply pid algorithm to firring ssrs for element control, iot logging ... 3) Adding strike volume to grains Scales/esp publishing mass pump subscribing 4) mash control either rims/herms pid controlled as per strike heat up logging to iot host only publishing a Done message.. (ignoring steps for now) 5) sparge control - reiteration x2 of step 3 6) boil and hop alarms, publising messages to indicate time to make hop additions, Me in real life subscribing and responding.. (hop times being configured in advance)

So as you can see not huge traffic volumes But some time critical responses needed to avoid overfilling pots and ruining the brew. is mqtt suitable or should i go back to a plan of static ip's and targeted udp messages??

Tia.

[cbrum11]

Tia,

You should look into a raspberry pi as your broker running Node-Red to tie in all the gadgets and a nice user interface. I've just set up a system with about 5 Esp's communicating via the open-source Mosquitto Broker on the pi and using Node-Red. It works like a charm and isn't all that complicated to setup once you get the basics.

Node-Red specifically made the barrier to entry super appealing for someone like me. Drag and drop a few boxes and you've got MQTT to HTTP to MySQL or even MongoDB... and even the ability to interact with each ESP (think toggling pins or changing settings on the fly).

Also, as a side note, I'm not sure exactly what your definition of "time critical" is, but MQTT is meant to be an extremely fast messaging protocol. I believe it was originally created to transfer oilfield pipeline data over long distances. I'm sure, in an application like that, near real-time data is pretty important, and I can tell you, in my own testing, the lag time between a client publishing a message and a subscriber reading/reporting the message is literally not perceivable.

Anyway, sounds like you're working on a really cool project. Let me know if you've got any questions about how I set things up and I'd be happy to help. I'm certainly still learning and only code/tinker as a hobby, but it's always nice to swap ideas.

Peace!

[hunnimonstr]

Hi Chase, Thanks for the encouragement..  FYI i have spent the last 24 hours chasing a connection to a local mosquitto broker. After reading your advice ( No pressure.. well  not much) And such folk as Pete Scargill and 'the chap with the swedish accent'  i ordered a Pi-Zero W and downloaded a dietpi image apparently with 'everything' installed and ready to go... Well after a LONG day, ive just pointed all my "hello world" grade 1st nodes at test.mosquito.org and its all bloody green lights!! Since you have it all up n running could i ask what config of *nix you have installed and is there a fools step by step guide you could suggest for me to follow?  I can only assume i missed a crucial step in the utube guide i tried following,  Btw TIA is old Newsgroup speak for Thanks In Advance.. Cheers Phil K

On Tuesday, 11 April 2017, 6:17, Chase <notifications@github.com> wrote:

Tia,You should look into a raspberry pi as your broker running Node-Red to tie in all the gadgets and a nice user interface. I've just set up a system with about 5 Esp's communicating via the open-source Mosquitto Broker on the pi and using Node-Red. It works like a charm and isn't all that complicated to setup once you get the basics.Node-Red specifically made the barrier to entry super appealing for someone like me. Drag and drop a few boxes and you've got MQTT to HTTP to MySQL or even MongoDB... and even the ability to interact with each ESP (think toggling pins or changing settings on the fly).Also, as a side note, I'm not sure exactly what your definition of "time critical" is, but MQTT is meant to be an extremely fast messaging protocol. I believe it was originally created to transfer oilfield pipeline data over long distances. I'm sure, in an application like that, near real-time data is pretty important, and I can tell you, in my own testing, the lag time between a client publishing a message and a subscriber reading/reporting the message is literally not perceivable.Anyway, sounds like you're working on a really cool project. Let me know if you've got any questions about how I set things up and I'd be happy to help. I'm certainly still learning and only code/tinker as a hobby, but it's always nice to swap ideas.Peace!— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[duncan-a]

It's a 'Swiss' accent... He does do good videos...!

On 13 Apr 2017 22:58, "hunnimonstr" notifications@github.com wrote:

Hi Chase, Thanks for the encouragement.. FYI i have spent the last 24 hours chasing a connection to a local mosquitto broker. After reading your advice ( No pressure.. well not much) And such folk as Pete Scargill and 'the chap with the swedish accent' i ordered a Pi-Zero W and downloaded a dietpi image apparently with 'everything' installed and ready to go... Well after a LONG day, ive just pointed all my "hello world" grade 1st nodes at test.mosquito.org and its all bloody green lights!! Since you have it all up n running could i ask what config of *nix you have installed and is there a fools step by step guide you could suggest for me to follow? I can only assume i missed a crucial step in the utube guide i tried following, Btw TIA is old Newsgroup speak for Thanks In Advance.. Cheers Phil K

On Tuesday, 11 April 2017, 6:17, Chase notifications@github.com wrote:

Tia,You should look into a raspberry pi as your broker running Node-Red to tie in all the gadgets and a nice user interface. I've just set up a system with about 5 Esp's communicating via the open-source Mosquitto Broker on the pi and using Node-Red. It works like a charm and isn't all that complicated to setup once you get the basics.Node-Red specifically made the barrier to entry super appealing for someone like me. Drag and drop a few boxes and you've got MQTT to HTTP to MySQL or even MongoDB... and even the ability to interact with each ESP (think toggling pins or changing settings on the fly).Also, as a side note, I'm not sure exactly what your definition of "time critical" is, but MQTT is meant to be an extremely fast messaging protocol. I believe it was originally created to transfer oilfield pipeline data over long distances. I'm sure, in an application like that, near real-time data is pretty important, and I can tell you, in my own testing, the lag time between a client publishing a message and a subscriber reading/reporting the message is literally not perceivable.Anyway, sounds like you're working on a really cool project. Let me know if you've got any questions about how I set things up and I'd be happy to help. I'm certainly still learning and only code/tinker as a hobby, but it's always nice to swap ideas.Peace!— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/esp8266/Arduino/issues/2839#issuecomment-294019753, or mute the thread https://github.com/notifications/unsubscribe-auth/ALP8xUW6Yd-E7scCCpqq3o3CNv7lD63Aks5rvoxpgaJpZM4LdX9N .

[cbrum11]

@hunnimonstr

The "TIA" abbreviation got me for sure. Learn something new every day haha.

So let me clear a few things up so we can see if I can actually be of help to you or not. My system is currently running on a Raspberry Pi 3 with Raspian Jessie installed through NOOBS.

I have never used a Zero W, but would very much like to for my next project, as the Pi 3 was probably overkill for what I'm currently doing. At the end of the day, the Pi Zero W should be very capable of both running the Mosquitto Broker and acting as a Wireless Access point (which are really the only two things you need to get this running).

My general process for getting this to work was as follows:

1. Fresh install of Raspian Jessie

2. Change all necessary raspi-config files (ESPECIALLY TIMEZONE AND KEYBOARD)

3. Plug into Ethernet jack and Update/upgrade everything

4. Follow this tutorial which guides you through how to make the Pi 3 a Wireless Access Point https://www.raspberrypi.org/documentation/configuration/wireless/access-point.md

   The tutorial includes:

         - Editing your network/interfaces file
         - Editing your wpa_supplicant file
         - Downloading and setting up Hostapd
         - Downloading and setting up DHCPD

   Although it was tested on a PI 3, they state it should also work on a Pi Zero W. I can confirm that it works flawlessly as long as you don't mix up a 1 (one) with an L (ell) when editing the driver name.

   See here for a good laugh --> My Ridiculous Forum Post

   I have not yet "bridged" my system to the outside internet, so I cannot attest to this part of the tutorial.

5. Reboot and test that your access point was created using a laptop or something.

**A note of caution: If you leave your Ethernet plugged in on reboot, your Raspberry Pi will likely have a different IP address when it creates the Access Point than if you did not. This caused headaches for me when I tried to code the ESP8266's because I coded them all with the IP address that showed up when the Ethernet cable was plugged in. I had to go back and re-code them after I figured this out.

6. Download the correct version of the Mosquitto Broker (there are different versions for Wheezey and Jessie). I found this tutorial worked well. https://oshlab.com/install-mqtt-mosquitto-raspberry-pi/

7. I believe once you have installed the broker it automatically starts running. If you're not positive, you can reboot and I'm 99% sure it runs at startup.

8. Open a command window and type

sudo mosquitto_sub -h localhost -t test/topic -v

9. Open a separate command window and type

sudo mosquitto_pub -h localhost -t test/topic -m "TEST MESSAGE"

If the words "TEST MESSAGE pop up in the first command window. Congratulations, you crushed it, Mosquitto is running and you're communicating via MQTT. (Visit the mosquitto.conf file if you'd like to set up different Mosquitto Server settings).

10. Go code an ESP8266 to connect to the Access Point you created on the raspberry pi, then to connect the MQTT server. Publish to the same topic "test/topic" and see if it shows up on your Raspberry Pi console window. If you need help with this, I've got some sample code I can share that should get you started.

11. Code everything you want on your micro-controllers and go learn a bit about Node-Red to see how easy it is to display the data.

node-red-start

Is the command to start node red. Then open a browser and navigate to https://"your pi's IP":1880

For example it could be:

http://192.168.0.1:1880

This will bring you to the node-red editing page. If you make it this far, let me know, and I'd be happy to help with a few basic node-red tricks to get you started.

I hope this was helpful, and after I graduate school I plan on making a much more detailed and useful tutorial on my own GitHub. Until then, feel free to ask for any clarification here and I'll do my best to respond.

Finally, just some food for thought. As my system is just for a quick demonstration and project, I have not implemented any security besides general, wireless wpa username and password. If I were making a more sensitive project, such as one where catastrophic failure/money loss/or real danger could occur if it were compromised, I would look into MQTT security.

There are a few tutorials floating around Google. One problem, however, is that, if you want to have top notch security a "certificate exchange" is required. I ended up experimenting a bit and getting this to work from the Raspberry Pi WIFI to a WPA2 Enterprise secure server. I then, also got a "self-signed" certificate to work between the MQTT broker and the ESP8266. To be honest, I know very little about wireless security, so I'm not sure if this is considered safe/a best practice, or if there are other methods that should be employed instead. The point, however, is if a malicious actor could cause any sort of real harm by getting into your system, I would seriously consider looking into this.

No one wants to see good beer spilled after all ;)

Please keep me up to date on your progress Phil, and if you end up successful, I call first dibs on a cold one.

Peace -Chase

P.S.

Just for some encouragement, here's a picture of my system currently ticking along (couldn't have done it without the amazing Node-Red interface tool. Those guys/gals have really created something special if a non/code guy like me can use it).

[martin-ger]

You might want to have a look into my latest project: the esp_uMQTT-broker at https://github.com/martin-ger/esp_wifi_repeater/tree/uMQTT_broker

[hunnimonstr]

Hi Chase, Swiss it is  DOH! Thank you for the above, Big Time..   I have not had the chance to fully digest it yet but it sure looks like a great road map to follow,  I will have a delve into it all this evening (fingers crossed..)   i did get a system up but wifi is only established and maintained with a dirty repetitive call to wpa_cli reconfigure post startup and kept alive with a regular chron job to do the same.  So its far from what i would consider a viable and reliable resource atm. without the above the wifi connects - to a point in that the pi can establish a connection and monitor handshaking traffic but no ip is agreed and the pi fails to qualify as an attached device.. and my wifi credentials are plastered about in a dozen text files all over the place..   It did give me access to a working mqtt broker and node red so i have lost my hello world cherry  PH, EC, and C02,  there have to be over 420 different things you could be monitoring there. I owe you more than a pint..  thanks.. Phil K    

[cbrum11]

there have to be over 420 different things you could be monitoring there

This. Was. Hilarious.

It's actually just Bibb lettuce for a school project. But no reason it couldn't be used for any crop that tickled one's fancy.

Excited to see how you progress. Your success could make all the hours I spent failing a little less painful. Let me know how it goes!

[lalo-uy]

Hi hunnimonstr
Been a home brewer myself I wish to talk some more about your setup. Was working myself on something very similar for the fermenters. Planed to implement pid loop on the esp, so only log actual temp and send the setpoint over the wire.

[hunnimonstr]

Im afraid life has got in the way and the 'simple' interface board i invested in for a usb master socket to attatch to an esp for interogaring scales has proven a tad more problomatic to intaerface with So i may have to have an arduino in the mix still too.  Have you checked out the brewpiless project ? https://github.com/vitotai/BrewPiLess which seems to have most fermentation bases covered ;)    While im very happy with my old atc800+ brewfridge controller,  with an ispindle awaiting assembly and gathering dust on my cluttered work/play shelf,  im considering the upgrade myself..  

On Friday, 2 June 2017, 17:23, lalo-uy <notifications@github.com> wrote:

Hi hunnimonstr Been a home brewer myself I wish to talk some more about your setup. Was working myself on something very similar for the fermenters. Planed to implement pid loop on the esp, so only log actual temp and send the setpoint over the wire.— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

| | Virus-free. www.avast.com |

[lalo-uy]

Many thanks for the pointings

Prost

2017-06-02 16:06 GMT-03:00 hunnimonstr notifications@github.com:

Im afraid life has got in the way and the 'simple' interface board i invested in for a usb master socket to attatch to an esp for interogaring scales has proven a tad more problomatic to intaerface with So i may have to have an arduino in the mix still too. Have you checked out the brewpiless project ? https://github.com/vitotai/BrewPiLess which seems to have most fermentation bases covered ;) While im very happy with my old atc800+ brewfridge controller, with an ispindle awaiting assembly and gathering dust on my cluttered work/play shelf, im considering the upgrade myself..

On Friday, 2 June 2017, 17:23, lalo-uy notifications@github.com wrote:

Hi hunnimonstr Been a home brewer myself I wish to talk some more about your setup. Was working myself on something very similar for the fermenters. Planed to implement pid loop on the esp, so only log actual temp and send the setpoint over the wire.— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

| | Virus-free. www.avast.com |

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/esp8266/Arduino/issues/2839#issuecomment-305883086, or mute the thread https://github.com/notifications/unsubscribe-auth/AK4bcROJONIl952CiGcqJdmmq4Gns4_5ks5sAF0wgaJpZM4LdX9N .

[TridentTD]

Upload Mqtt Broker to ESP8266 (in Thai)

https://www.youtube.com/watch?v=K__dI5hrO-A

Test connecting to Mqtt Broker on ESP8266 (in Thai)

https://www.youtube.com/watch?v=RjYThSoi7g0

[bcatalin]

Hi, I am the author of the: https://myesp8266.blogspot.in/2016/02/mqtt-broker-on-esp8266.html https://myesp8266.blogspot.in/2016/03/mqtt-broker-on-esp8266-2.html https://myesp8266.blogspot.in/2016/04/mqtt-broker-on-esp8266-3.html

I've started an experimental site http://iotcentral.eu where you will be able to download the bin file for the MQTT broker Bondar to run it on the ESP8266. Please note that everything is still under development but the main functions offered are:

-Bondar ESP8266 MQTT broker ( up to 24 devices can be connected) -Bondar ESP8266 MQTT broker is bridging messages to the iotcentral.eu:1883 or to iotcentral.eu:9004 for websockets -Bondar ESP8266 MQTT broker is offering an websocket connection on the port 1883

More details are on http://iotcentral.eu where you can have a cloud and private MQTT broker.

Go to https://github.com/bcatalin/bondar for issues and discussions.

Thanks.

[martin-ger]

Will look into that, but I would really love to have a look into the sources. Especially I would like to see and test the 24 connections. I wonder whether you managed to get around the 8 WiFi-clients limit of the ESP and the 10 TCP-connections.

[bcatalin]

We can see here https://myesp8266.blogspot.ro/2016/11/mqtt-broker-on-esp8266-5.html a post about the connection. Customizing lwip in FreeRtos was the solution.

[scargill]

and if you need MQTT on a PI or other device... https://tech.scargill.net/a-christmas-script/

[devyte]

Interesting discussion, but it would be best to hold it at a community forum, as it is not an issue. Closing due to off-topic for this Issue Tracker, please see #3655 .

[hsaturn]

Hi all

I wrote a TinyMqtt library for Esp8266. Two days of coding and I have a working beta solution. (github hsaturn tinyMqtt)

For some reasons, I won't install a Pi or anything else just to have a mqtt broker. But for the sake of connectivity with existing solutions, I want to use Mqtt.

This is the reason for this TinyMqtt solution I wrote.

In fact I will go beyond that. I want my system to be very resilient, even if the Mqtt broker is down !!!

And beyond is :

• I want to have a Mqtt client that ... does not need a broker when this client subscribes to its own topics.
• I want to be able to have a broker AND a client on the same ESP !!!

Sounds crazy ? Imagine this situation :

My unique ESP connected to a temperature sensor AND a relay. It's really easy to write a standalone programm to make this ESP control the relay in order to have the temperature regulated with the relay.

But this autonomous system cannot send its temperature neither the state of the relay. So adding Mqtt in order to publish sensors/temp and actors/relay is a good way to achieve this.

Once done, it would be convenient to subscribe to this temp instead of taking it from the sensor. That way, the system becomes more flexible but wait ... what if the mqtt broker fails to respond ? => Nothing will work, the house becomes cold, (and dark because light switch may need a broker also...)

==> I *REALLY do not want that to happen. (Or my wife will kill me, for sure !)

Having a smart client that still continue to work in some kind of degraded mode (continue to receives its own topics) is the solution. I'm working on that.

Fine, but why do I need a broker on one ESP so ? If you have a Rasberry PI broker, it's ok to achieve all what you want to...

-> Right, but I do not want to install a PI just for that. More than that, when I'll have two ESP on my house, one can be used as a broker.

Yet, my broker works and has been tested with Mqtt-box, Mqtt-explorer and it seems to work very well. Also I've been able to connect up to 10 clients (not tried more).

I'm working now on creating the library for Esp. (should be quick). When this is done, I'll work on writing the tiny mqttclient that ... won't need a broker.

[bertmelis]

Why on earth would you want to subscribe to your own topics?

[hsaturn]

Why on earth would you want to subscribe to your own topics?

I want modules in the same ESP to subscribe to topics that eventually come from the same ESP. I want that to be easy to configure (no need to reflash anything) and I want that to still work if my broker is down.

Example: a relay can be controlled with a switch on the same ESP.... or a switch on another ESP .. or both... But if ever the broker fails, I want at least the 1st switch to continue to work.

(In fact I want a lot more than that). If you have a wife, you cannot explain her why this f..ing switch does not work. It HAS to work :-)

[hsaturn]

Why on earth would you want to subscribe to your own topics?

By the way Bert, congratz for your DHT lib. I wrote a 433Mhz interrupt driven lib for the same reason (but not yet publish as it can only handle my dio buttons).