search

homieiot / homie-esp8266

💡 ESP8266 framework for Homie, a lightweight MQTT convention for the IoT
http://homieiot.github.io/homie-esp8266
MIT License
1.36k stars 308 forks source link

Wishlist for v2.0.0 #87

Closed marvinroger closed 5 years ago

marvinroger commented 8 years ago

TODO

1st step

The 2.0.0 will be a breaking change. As such, we can safely change the API, the functions name, the order of the parameters... Let me know if you think there is any weirdness with the API, or the HTTP API of the config mode, etc. :smiley:

jpmens commented 8 years ago

+1 (and very much appreciated) for embedding the magic bytes for homie-ota; this will make our sketches look neater.

Regarding TLS (SSL): in spite of the huge amount of work you went to to get it to work, I have since reconsidered. Within a home, It's more important that devices are stable and "unbreakable" (eg killing them with PUBs) than having TLS. Furthermore I have the feeling that most people don't care about the security anyhow... In other words: drop TLS until ESP8266 supports it natively and asynchronously.

piotrwachowiak commented 8 years ago

Hi, once again thanks for great work. If it is the right place for some kind of wish list: For me, if not TLS and/or native PKI support would be great, if only esp can handle this. Deep sleep and battery control included in MQTT convention, like delayedHomieLoop(int sleepTime) or something like this, method which takes sleepTime as argument and handles everything. Just dreaming. I had some troubles creating universal OLED thing, which is capable of displaying given topic/message with given params. I'd love to see it done in professional way. It's probably just me, but event handling and subscribing is not clear for me - personally I failed with subscribe(), HomieNode relayNode("room", "relay", relayOnHandler, true) works, but I am not sure it's right way.

Best regards,

Piotr

unaiur commented 8 years ago

I wouldn't add custom settings. I think that we should allow to use WPS to connect to the WiFi network and use mDNS to discover a default mqtt broker service. Then, the remaining options can be configured with mqtt messages.

This way we didn't need any special BootConfig mode.

jpmens commented 8 years ago

+1 for configuration via MQTT.

marvinroger commented 8 years ago

@unaiur this would be ideal, but I am not sure it would fit everyone. WPS is probably not supported by all routers, and who actually advertise its MQTT broker via mDNS?

euphi commented 8 years ago

Regarding API: Make the callback function a virtual method of HomieNode.

This would allow for a more modular design.

You still can have the default implementation to call a NodeInputHandler.

marvinroger commented 8 years ago

@euphi Can you provide a sample code of how it would look like?

euphi commented 8 years ago

Proof of concept (on tag v1.5.0) patch_virtual_callback.txt

It was necessary to remove some const. Since I haven't coded for almost 6 years, I am a little bit out of practice, but I think the const was wrong there anyhow. With the call via the method pointer, you discard the const of the Homienode object - with the member function, this is no longer possible. And it is quite obvious that calling a callback method that expect to change a state of a object, should not be called on a const object :-)

I did not had the time to create a class derived from Homienode, but this patch at least do not break anything :-) Due to a business trip, I won't have time to try more for at least one week. :-(

euphi commented 8 years ago

Due to bad weather today I found some time to make use of the new HomieNode::Inputhandler(..)

I created a new project at GitHub.

Especially have a look at https://github.com/euphi/BalkonController/blob/master/src/RGBWNode.cpp and https://github.com/euphi/BalkonController/blob/master/src/RGBWNode.h.

It is work in progress, there are still some nodes with the old style of callback.

To use them with OpenHab, I use the following items configuration:

Number balkon_temp      "Temperatur [%.1f °C]" (Balkon, TempSens)  {mqtt="<[local:devices/balkctrl/Temperatur/degrees:state:default]"}  
Number balkon_press     "Luftdruck [%.1f hPa]"  (Balkon)            {mqtt="<[local:devices/balkctrl/Temperatur/hPa:state:default]"}  
Switch balkon_auto      "Automatik"             (Balkon)            {mqtt=">[local:devices/balkctrl/Auto/on/set:command:ON:true],>[local:devices/balkctrl/Auto/on/set:command:OFF:false]"}  
Switch balkon_pumpe     "Pumpe"                 (Balkon)            {mqtt="<[local:devices/balkctrl/Pumpe/on:state:ON:true],<[local:devices/balkctrl/Pumpe/on:state:OFF:false],>[local:devices/balkctrl/Pumpe/on/set:command:ON:true],>[local:devices/balkctrl/Pumpe/on/set:command:OFF:false]"} 
Switch balkon_valve1    "Kräuter bewässern (V1)"  (Balkon)        {mqtt="<[local:devices/balkctrl/Ventil/1:state:ON:true],<[local:devices/balkctrl/Ventil/1:state:OFF:false],>[local:devices/balkctrl/Ventil/1/set:command:ON:true],>[local:devices/balkctrl/Ventil/1/set:command:OFF:false]"}
Switch balkon_valve2    "Hecke bewässern (V2)" (Balkon)            {mqtt="<[local:devices/balkctrl/Ventil/2:state:ON:true],<[local:devices/balkctrl/Ventil/2:state:OFF:false],>[local:devices/balkctrl/Ventil/2/set:command:ON:true],>[local:devices/balkctrl/Ventil/2/set:command:OFF:false]"}
Switch balkon_valve3    "Himbeeren bewässern (V3)" (Balkon)        {mqtt="<[local:devices/balkctrl/Ventil/3:state:ON:true],<[local:devices/balkctrl/Ventil/3:state:OFF:false],>[local:devices/balkctrl/Ventil/3/set:command:ON:true],>[local:devices/balkctrl/Ventil/3/set:command:OFF:false]"}
Switch balkon_valve4    "Rosen bewässern (V4)" (Balkon)            {mqtt="<[local:devices/balkctrl/Ventil/4:state:ON:true],<[local:devices/balkctrl/Ventil/4:state:OFF:false],>[local:devices/balkctrl/Ventil/4/set:command:ON:true],>[local:devices/balkctrl/Ventil/4/set:command:OFF:false]"}
Number balkon_signal    "WiFi Signal [%.1f %%]" (Balkon)            {mqtt="<[local:devices/balkctrl/$signal:state:default]"}

Color  balkon_led       "LED-Streifen"  <slider>    (Balkon)
Dimmer balkon_led_R     "Rot [%d %%]"   <switch>    (Balkon)        {mqtt=">[local:devices/balkctrl/LEDs/r/set:state:*:default]"}
Dimmer balkon_led_G     "Grün [%d %%]" <switch>    (Balkon)        {mqtt=">[local:devices/balkctrl/LEDs/g/set:state:*:default]"}
Dimmer balkon_led_B     "Blau [%d %%]"  <switch>    (Balkon)        {mqtt=">[local:devices/balkctrl/LEDs/b/set:state:*:default]"}
Dimmer balkon_led_W     "Weiß [%d %%]" <slider>    (Balkon)        {mqtt=">[local:devices/balkctrl/LEDs/w/set:command:*:default]"}
unaiur commented 8 years ago

I've evolved the proposal from @euphi in a project I'm doing to control led strips with Homie. You can see the code in homie-nodes

TommySharpNZ commented 8 years ago

To be able to run Homie on a battery powered device would be a great addition especially for simple environmental sensors around the house.

powerwade commented 8 years ago

My wishlist item is to log/send messages to a local / remote syslog server.

jpmens commented 8 years ago

@powerwade I think that would be out of scope, in particular as it'd mean implementing syslog protocol: Homie is MQTT.

This can very easily be accomplished by creating a subscriber (server-side) which subscribes to your MQTT messages and pushes them out via syslog; a few lines of, say, Python would do the trick.

jayaras commented 8 years ago

I could see syslog being handy and have thought of a few use cases where it would/could be. At least for prototyping when I don't have serial output. I've been fighting with a faulty pn532 board and it would have been handy to get some debugging info over syslog.

flaviostutz commented 8 years ago

I like Homie API names and the code style very much. I made a feature request on https://github.com/marvinroger/homie-esp8266/issues/26 regarding to captive portal.

euphi commented 8 years ago

Regarding syslog: It would be nice, if Homie could use another logger. Therefore it should be possible to set a logger class, e.g. any class implemting the "Print" class.

So, it is possible to create a syslog logger from another library and set it as preferred logger for Homie.

marvinroger commented 8 years ago

Regarding the syslog: @jpmens stated deservedly that MQTT is a mean of communication. You could just create a system (or log or whatever you want) on which you publish your debugging messages to.

I do agree with @euphi however, being able to set a Print class to log to.

lighthousebulb commented 8 years ago

I love the proposed Idea to set a configuration via mqtt. But I have another suggestion: What about a several configuration-files with different WiFi-settings for example? The Node should scan for available Networks and choose a configuration depending on the available Networks. For example, when I set up a Node an move it to my friends house, with different WiFi-Settings, the Node could simply choose the right config-file for the different environment.

unaiur commented 8 years ago

I'm working in wifi wps setup. I'm planning to add a new configuration option EnableWPS. When defined it will use the integrated settings in the ESP8266WiFi library, or start WPS pairing if no network is configured. It's nearly done and it uses very little code.

unaiur commented 8 years ago

Another future ticket I would like to work at in the future is startup time.

I'm planning to implement a button on battery. When the button is pressed, the ESP will wake up from deep sleep, connect to the MQTT server and publish a topic.

The problem is that it needs about 4 seconds. I will try to: a. Save DHCP lease to persistent memory and bypass this phase while the lease isn't stale. b. Save mDNS service also.

These steps can halve the startup time. Do you have any other idea to try to reduce startup time even more?

unaiur commented 8 years ago

You can see my WPS implementation at https://github.com/unaiur/homie-esp8266/tree/wps It isn't fully tested yet, but you can taste it.

marvinroger commented 8 years ago

@unaiur it seems we can cut down the time needed to connect to the Wi-Fi by not calling WiFi.begin() on each boot, but let the SDK store the Wi-Fi config itself. (see https://github.com/z2amiller/sensorboard/blob/b33da405795761efce0dfa02f53b6932c87958ee/PowerSaving.md)

marvinroger commented 8 years ago

I am also considering removing the OTA mode and handle everything in the normal mode. MQTT would be used for the OTA, so OTA would not happen over HTTP anymore. This is now possible thanks to AsyncMqttClient and https://github.com/esp8266/Arduino/commit/2f933e20fa45f7c76347089f75974f557645fb03

@jpmens what do you think about it? homie-ota would need some change, but I think having only one protocol to handle everything (except for the HTTP API) is a step forward.

jpmens commented 8 years ago

@marvinroger am I understanding correctly that the new binary firmware would be sent to Homie via an MQTT publish? If so, I'm all for it, and we can easily change homie-ota to do it.

marvinroger commented 8 years ago

@jpmens Yes, this is what I mean. Let's go for it, then :)

marvinroger commented 8 years ago

@unaiur just achieved to connect to the Wi-Fi network in 1s in the latest Git. :smile: The code needs esp8266/Arduino 2.3.0-rc2 to run and needs either a change in Arduino.h or https://github.com/esp8266/Arduino/pull/2084 to be fixed, so you might not want to test.

marvinroger commented 8 years ago

OTA over MQTT works great! Thanks @me-no-dev for Update.runAsync(true). :innocent:

me-no-dev commented 8 years ago

you are welcome :)

gazally commented 8 years ago

I really like what you've done here, it's all very well thought out. But I have a wishlist that might take you to 3.0! 😄

I'd like to use Homie on an ESP8266 that is a packet radio gateway to battery-powered sensor devices that periodically wake and send data and are capable of downloading firmware updates as well. The packet radios have far greater range than wifi and use less power, at the expense of bitrate.

This means I need "child" devices which use the parent's wifi config and MQTT connection but have their own name, node list, uptime, firmware etc. My handler callbacks would then communicate with the sensors via packet radio.

But then thinking about the architecture, wouldn't it be great if the code in the sensors could look just like the code in the gateway? The user sketch code in your API now doesn't know anything about wifi or MQTT and wouldn't have to change if data was being sent by packet radio instead. To accomplish this there would need to be an API layer in your library where one configuration and communication protocol could be substituted for another. And then if someone wanted to communicate with child devices using bluetooth instead, they could write code for the API.

But once you have done that, the gateway's upstream communication is no longer limited to wifi. Communication protocols could be written for devices that get their IP address from a cellular modem, or simply an ethernet cable. Or someone might want to build gateways between different packet radio networks, where local gateways communicate with very low-power devices at short range and send the data at higher power to a gateway device which has an MQTT connection. Many, many possibilities!

gazally commented 8 years ago

I've now spent some time reading your code and I see that the upstream link over packet radio is best handled with a new library called homie-rfm69hw or some such. Which maybe I will have to write myself. 😄 But the concept of child devices that share the parent's upstream connection is still on my wishlist, and an API for downstream communication with the children that allows different methods of communication to be used transparently to the top-level code.

TommySharpNZ commented 8 years ago

@Gazally, have you looked into something like MySensors for what you are thinking?

marvinroger commented 8 years ago

@gazally this is indeed a good idea, but I am not sure this would be a good fit for this library. I'd like to keep Homie for ESP8266 (relatively) simple. It's meant to be used with MQTT. If ESP8266s were pricey, I would probably reconsider, but at ~3$ each, I think Wi-Fi is a great fit for IoT.

But, what you're asking is doable, even with the current codebase. You only have to mirror the MQTT communications, using the protocol you want. This approach is not as extensible as what you're asking for, but as @TommySharpNZ said, libraries like MySensors are great at handling that. :)

gazally commented 8 years ago

Thanks for the suggestions! I'll have a look at MySensors.

jamesmyatt commented 8 years ago

In case you weren't aware, esp8266/Arduino 2.3.0 was released last week: https://github.com/esp8266/Arduino/releases

marvinroger commented 8 years ago

@Nzbuu I was, thanks. :) But there is a lot of work left before v2 is ready.

hschroyen commented 8 years ago

Hi Marvin,

I am following your project very closely and like the idea of the framework. I also stumbled upon the following project which might be very interesting to incorporate some idea's in homie.

http://harizanov.com/wiki/wiki-home/three-channel-wifi-relaythermostat-board/

Keep up the good work

marvinroger commented 8 years ago

@hschroyen thanks, I'll take a look at it. :smiley:

All new features are implemented. The next step will be to make sure it's bulletproof (you can easily crash it by sending big payload for the moment), then it will be documentation time, as a lot of things changed (for the good).

marvinroger commented 8 years ago

Documentation is up: https://homie-esp8266.readme.io/ :smile: Let me know what you think. Please note it's not yet released, I want the v2 to be a minimum stable, without dumb negligences (though there will probably be some anyway)

euphi commented 8 years ago

I proposed some edit yesterday (description of the virtual handleInput(..) as fourth input handler variant.

Unfortunately there is no feedback on readme.io, so I'm unsure if you received it?

Furthermore, I think an upgrade guide from 1.5.0 would be useful. I cannot propose a new page on readme.io, so I will list some suggestions here:

Summary of important Changes

Caveeats

marvinroger commented 8 years ago

I'm sorry, my Internet connection is rather limited until I come back from vacations, so I won't work on the project for a week. Thanks for contributing to the docs, I'll take a look. It would be a good idea to create a new page about extending HomieNode, as it is a feature that deserves it's own space.

Le 21 août 2016 1:28 PM, "Ian Hubbertz" notifications@github.com a écrit :

I proposed some edit yesterday (description of the virtual handleInput(..) as fourth input handler variant.

Unfortunately there is no feedback on readme.io, so I'm unsure if you received it?

Furthermore, I think an upgrade guide from 1.5.0 would be useful. I cannot propose a new page on readme.io, so I will list some suggestions here: Summary of important Changes

  • Replace platformio libraries (PubSubClient replaced by AsyncMQTT)
  • Important API changes
    • No need to use registerNode(..)
    • There is no more subsribeToAll(..)
    • Use advertise(..) to create properties
    • Settable properties must be marked as such by calling settable(..) .
    • ....

Caveeats

  • Signature of setNodeProperty() changed, but still compiles fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/marvinroger/homie-esp8266/issues/87#issuecomment-241252284, or mute the thread https://github.com/notifications/unsubscribe-auth/AA8eNZHr3yHwfNfIYmzVTG6E9leyYjLlks5qiDZzgaJpZM4Iem2d .

jpmens commented 8 years ago

@unaiur has there been any progress with WPS for homie-esp8266? I am in a situation where this would probably be extremely useful for us.

shogsbro commented 8 years ago

Would it make sense to implement a captive portal while in Configure mode? I think it would only require capturing the iOS and Android test URL's, and returning a HTTP redirect.

marvinroger commented 8 years ago

@shogsbro the captive portal is already implemented in the latest git, but this is only useful if you've embedded the ui_bundle.gz

thorhs commented 8 years ago

I too would love a battery friendly deep sleep. Is that possible? I have a custom code for temperature sensing using ESP and MQTT. I would love to swap out the plumbing for Homie.

marvinroger commented 8 years ago

Deep sleep will be possible with #138, I don't know if there's something else you would like for deep sleep?

karlp commented 7 years ago

Where's the documentation on the "ui_bundle.gz" ? I can't find it in the 2.0 docs online.

marvinroger commented 7 years ago

@karlp just added a page: https://homie-esp8266.readme.io/v2.0.0/docs/ui-bundle

flaviostutz commented 7 years ago

@tiagostutz, see what an amazing v2.0 planning and execution for Homie 2.0...

tiagostutz commented 7 years ago

Awesome! Great expectations here...! Now let's rush with the Config JSON API Emulator project to see if it helps build the captive portal... When I finish the README.md I'll share here to get some insights from the community.

tiagostutz commented 7 years ago

The first version of the emulator is ready. It's not a big deal, but maybe it can be useful: