Interface for adding extra settings

[marvinroger]

As @furyfire asked in #8, it would be neat to have an API to add other settings than wifi, mqtt and ota in configuration mode.

[codmpm]

Hi Marvin, thanks for your obvious hard work.

I second the idea of having additional config parameters in configuration mode. Right now I'm building a NeoPixel Controller with mqtt connection and Homie-ESP just fits in to link this together. It would be very nice to configure the connected neopixel count or maybe the wanted protocoll for the neopixel stripe. Otherwise I have to put this in the sketch, which is ok but not as convenient as it could be if it was manageable from configuration mode.

WiFiManager for example has this possibility allthough you have to deal with the saving of the values by yourself. But it leaves me with the complete handling of events and mqtt-connection alone.

I've just wanted to give some insights why this option could be usefull. Thank you.

Regards, Patrik

[marvinroger]

Hi!

What do you mean by "the wanted protocoll"?

This is a feature I would like to implement at some point, but I need to work on the stability of the framework for the moment. There is a lot of new features to implement, and there are some heap issues showing up.

Anyway, thanks for telling your use case. ;)

[codmpm]

Sorry, I should have clarified this:

When using neopixels you have to choose the right protocoll for the controller chip you are using. For example WS2811, WS2812(B), SK6812rgb, etc. I try to build some kind of generic WiFi/MQTT/Neopixel thingy.

I understand that stability is a bigger issue than additional parameters. Right now I can work with the mentioned WiFiManager and do the MQTT stuff myself. Thought the usecase could give you an idea :-)

You're welcome and thanks again.

[marvinroger]

Alright, I got it. Let's discuss the implementation when the stability issues are resolved. :)

[rschaten]

Like @codmpm, I started a project very similar to this, based on WiFiManager.

My plan is to place some nodes around the house, with different capabilities. Some of the nodes should get a DHT22 to measure temperature and humidity, some of my boards come with on-board RGB LED and a LDR, some should get relays to switch other devices.

My own firmware shows a form with some configurable values, via WiFiManager. I can configure the capabilities of this special node, and it announces this via MQTT (much like the $nodes in Homie).

Apart from that, Homie does everything I did myself -- and much more in some places, so I'm currently abandoning my own project in favor of Homie. Thanks for all the work you put into this!

Edit: I just saw #47 -- seems to be the same thing?

[marvinroger]

Yes #47 asked the same thing, but I think this is out of the scope of Homie. However, what @jpmens asked might be resolved by this feature. You'll get the settings you want, and based on these settings, you'll be able to do whatever you want (subscribe to topics or not, register the nodes or not, react to events or not...).

[flaviostutz]

@marvinroger, I am about to fork your great work (and hopefully return in a pull request) in order to implement some features needed for a series of three products we have been developing. From the user perspective, we have modeled the following UX:

• The user unpacks the product (something like a "smart lamp")
• He plugs it to the wall
• He connects his smartphone to the Wifi network "Configure_Argibee_373773"
• A web browser opens up asking for his Wifi SSID/PASS
• He types his SSID/PASS
• The web page tells him it is trying to configure wifi with those credentials and tests for real Internet connectivity (maybe the wifi has no Internet connectivity and this will be told to the user)
• When Internet connectivity is OK, a second screen is shown asking him for his username/password of our "things" cloud service
• The user types his username/password and receives an "OK" from our cloud server, even without disconnecting from the "Configure_Argibee_373773" network (Homie will proxy requests between AP and the newly connected Station).
• At this moment the cloud server validates his username/pass, enrolls the new device instance to his account and returns the Homie configurations to the device
• The custom captive webpage creates the config.json file using data returned by our cloud server (MQTT server, base_topic, device name etc) and PUTs it to the device (through current api)
• The user receives a "Finished" message with a possible signal from the device (maybe a blink for the "lamp") and the Wifi AP vanishes

The solution:

• Create a new rest endpoint for immediate connection to Wifi (receives an SSID/pass) and another endpoint for polling wifi status (so that captive webpage can show immediate user feedback)
• Change AP only to AP+Station mode
• Create a "proxy endpoint" so that the custom captive webpage can call something like "POST http://homie.config/proxy/stutzthings.com/devices {username:flaviostutz, password:abc, device:argibee, pub_key:asdfaf}". When receiving this request, Homie will proxy this request to "http://stutzthings.com/devices" using his "station" Wifi connection and pass data between browser and server, with no evaluation.

I believe with those new features, product creators can invent any captive processes/configurations he judges necessary without adding specific features to Homie.

I am about to start this. What do you think, Marvin?

[marvinroger]

It seems to be the best thing to do for your use case. However, I see two problems, that make me think the ESP8266 is not (yet?) suitable for this:

• Originally, I did something similar for Homie, with AP + STA, but is was not stable at all. The configuration mode was working 1/5 times, whereas with AP mode it worked flawlessly.
• https-lack. In your use case, you will want to transmit sensible data over the Internet, you will quickly see that TLS support is very limited on the ESP8266. This might change with the ESP31B/ESP32, but I would not be confident if my ESP8266 would communicate with the outer world. That's why Homie for ESP8266 is built without any dependencies on the Internet.

I am currently very busy, I won't be able to help you very much on this, but I would obviously gladly accept PR. ;)

Good luck for your project!

[flaviostutz]

@marvinroger, I finished the implementation, tested it and gladly had not seen such instabilities. I had such a bad time with instabilities using Lua on NodeMCU before, so I am very happy :)

I made the transparent proxy based on "Host" header and it is working all fine. Honestly I hadn't seen no reboots nor strange behaviors while using the transparent proxy to bridge requests between pages of even 7kB (a surprise for me). Maybe we need some more tests, but for now I am confident.

Please check my PR at https://github.com/marvinroger/homie-esp8266/pull/95

Hope to be contributing positively to your great work.

[marvinroger]

@flaviostutz merged, good work on this one!

[marvinroger]

Here is the API I am thinking about for this:

#include <Homie.h>

CustomHomieSetting<bool> enableRgbSetting("enableRgb",  "Enable RGB output", true, validateEnableRgbSetting); // parameterName, parameterDescription, defaultValue (if not null then the setting is optional, otherwise mandatory), validationFunction

void validateEnableRgbSetting(bool entry) {
  return true; // a bool cannot be validated, here to illustrate the idea
}

void setup() {
  // Homie.setFirmware()... etc
  Homie.setup();
}

void loop() {
  if (Homie.isReadyToOperate()) {
    bool rgbEnabled = enableRgbSetting.get();
  }
}

Simply. There would be 5 types of custom settings:

• bool
• unsigned long
• long
• double
• const char*

The validation function's purpose is, as its name implies, meant to validate the entry. Imagine you want to accept a percentage, you'll want a unsigned long and you'll check in the validation function if the number is between 0 and 100, returning false if it's not the case. Then, the /config endpoint in the API would return a non-200 error-code.

In this case, the /device-info endpoint would return:

{
  "device_id":"52a8fa5d",
  "homie_version":"2.0.0",
  "firmware":{

  },
  "nodes":[

  ],
  "settings":[
    {
      "id":"enableRgb",
      "description":"Enable RGB output",
      "type":"boolean",
      "default":true
    }
  ]
}

And the config.json:

{
  "name": "The kitchen light",
  "device_id": "kitchen-light",
  "wifi": {
    "ssid": "Network_1",
    "password": "I'm a Wi-Fi password!"
  },
  "mqtt": {
    "host": "192.168.1.10",
    "port": 1883,
    "mdns": "mqtt",
    "base_topic": "devices/",
    "auth": true,
    "username": "user",
    "password": "pass",
    "ssl": true,
    "fingerprint": "CF 05 98 89 CA FF 8E D8 5E 5C E0 C2 E4 F7 E6 C3 C7 50 DD 5C"
  },
  "ota": {
    "enabled": true,
    "host": "192.168.1.10",
    "port": 80,
    "mdns": "ota",
    "path": "/custom_ota",
    "ssl": true,
    "fingerprint": "CF 05 98 89 CA FF 8E D8 5E 5C E0 C2 E4 F7 E6 C3 C7 50 DD 5C"
  },
  "settings": {
    "enableRgb": true
  }
}

Does it sound good?

[flaviostutz]

I really like it. Maybe we could create a special "node" ($configNode) to group those attributes. In that way we could use the same existing node structures to read, write and request changes (/set) to those parameters remotelly. The only difference to a regular node is that you will store those states "offline" so the device can work even without an Internet connection.

[marvinroger]

@flaviostutz yes, this is probably what we're going to do for #36.

[marvinroger]

I would like the API to be as fluent as possible, what approach do you prefer?

First

#include <Homie.h>

HomieSetting<bool> enableRgbSetting("enableRgb",  "Enable RGB output", true, true, validateEnableRgbSetting); // parameterName, parameterDescription, required, defaultValue, validationFunction

void validateEnableRgbSetting(bool entry) {
  return true; // a bool cannot be validated, here to illustrate the idea
}

void setup() {
  // Homie.setFirmware()... etc
  Homie.setup();
}

void loop() {
  if (Homie.isReadyToOperate()) {
    bool rgbEnabled = enableRgbSetting.get();
  }
}

Second

#include <Homie.h>

HomieSetting<bool> enableRgbSetting("enableRgb",  "Enable RGB output");  // parameterName, parameterDescription

void setup() {
  // Homie.setFirmware()... etc

  enableRgbSetting.setDefaultValue(true).isRequired().setValidator([](bool candidate) { return true; });

  Homie.setup();
}

void loop() {
  if (Homie.isReadyToOperate()) {
    bool rgbEnabled = enableRgbSetting.get();
  }
}

I prefer the second approach, as you don't need to read the API to understand the code. What do you think?

[flaviostutz]

I like the second approach mainly if there are good defaults. Something like a default "return true" for validator, a default "false" value for boolean data types and "false" for required etc.

Sent from my iPhone

On Aug 14, 2016, at 11:48, Marvin Roger notifications@github.com wrote:

I would like the API to be as fluent as possible, what approach do you prefer?

First

include

HomieSetting enableRgbSetting("enableRgb", "Enable RGB output", true, true, validateEnableRgbSetting); // parameterName, parameterDescription, required, defaultValue, validationFunction

void validateEnableRgbSetting(bool entry) { return true; // a bool cannot be validated, here to illustrate the idea }

void setup() { // Homie.setFirmware()... etc Homie.setup(); }

void loop() { if (Homie.isReadyToOperate()) { bool rgbEnabled = enableRgbSetting.get(); } } Second

include

HomieSetting enableRgbSetting("enableRgb", "Enable RGB output"); // parameterName, parameterDescription

void setup() { // Homie.setFirmware()... etc

enableRgbSetting.setDefaultValue(true).isRequired().setValidator([](bool candidate) { return true; });

Homie.setup(); }

void loop() { if (Homie.isReadyToOperate()) { bool rgbEnabled = enableRgbSetting.get(); } } I prefer the second approach, as you don't need to read the API to understand the code. What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[marvinroger]

Actually, here is what I got so far:

template <class T>
HomieSetting(const char* name, const char* description);
T get() const;
bool wasProvided() const;
HomieSetting<T>& setDefaultValue(T defaultValue);
HomieSetting<T>& setValidator(std::function<bool(T candidate)> validator);

A setting is mandatory by default, unless setDefaultValue() is called, it then becomes optional. If optional and the setting is provided, wasProvided() returns true and get() returns the provided value. If optional and the setting is not provided, wasProvided() returns false and get() returns the default value.

If the validator function is not provided, then any value is considered good.

I think it just works this way!

[marvinroger]

Done, it will be released in the v2.0.0 release. Docs will follow. :tada: