search

sidoh / esp8266_milight_hub

Replacement for a Milight/LimitlessLED hub hosted on an ESP8266
MIT License
937 stars 220 forks source link

EthernetClient build #215

Open PetricaM opened 6 years ago

PetricaM commented 6 years ago

Hi Chris,

Could you make a build for Wemos D1 with EthernetClient instead of WifiManager?

There is Wemos D1 R2 (compatible with Mini; I'm already using R2 as MiLight Hub and it is more stable than Mini): https://www.amazon.co.uk/XCSOURCE-Advanced-Development-Compatible-TE482/dp/B01FVKTL9O/ref=sr_1_2?ie=UTF8&qid=1518341067&sr=8-2&keywords=wemos+d1+r2&dpID=51P5rYbkbeL&preST=_SY300_QL70_&dpSrc=srch

This could be used together with W5100 ethernet shield (it is designed for Arduino Uno however it is pin to pin compatible with D1 R2): https://www.amazon.co.uk/tinxi%C2%AE-Ethernet-Shield-Network-Arduino/dp/B01C2Q70A2/ref=sr_1_1?s=electronics&ie=UTF8&qid=1518341410&sr=1-1&keywords=w5100+ethernet&dpID=51Z4Zh%252B2jSL&preST=_SY300_QL70_&dpSrc=srch

It should reduce latency and also free up RAM as it will not have to deal with the wifi management.

Thanks

sidoh commented 6 years ago

Interesting. Has someone already built an ESP8266 library for the Ethernet shield?

PetricaM commented 6 years ago

I think this should be it: https://github.com/esp8266/Arduino/tree/master/libraries/Ethernet

sidoh commented 6 years ago

Huh, I had thought I replied to this. Sorry for the late reply.

I think the issue here is that the entire ESP8266 network stack is tied to WiFi code. The libraries I'm using to handle HTTP, MQTT, and UDP are all using the WiFI libraries when you drill down far enough.

I'm very supportive of the idea, but I think this is a very large effort.

otto2704 commented 6 years ago

Hi, I would be interested in an Ethernet-Solution too. Does not matter, if this is ESP or some Arduino... When I can use a cable, I will use a cable ;-) I have developed some UNO sketch though which can control 4 zones (using fhem). But it is not able to receive things from the remotes....

sidoh commented 6 years ago

Have you guys used Ethernet with ESP8266? I'm curious what hardware you'd even use. I found this library for the ENC28J60, but it doesn't seem like it's used a ton:

https://github.com/kissste/esp8266-enc28j60-UIP-Ethernet

daandupau commented 6 years ago

Some googling gave me this: https://github.com/klarsys/esp_enc28j60 using this 10base-t Ethernet module: ENC28J60.

i am also interested in using your project in conjunction with an Ethernet connection instead of WiFi.

Streep commented 6 years ago

This is probably a silly remark, but is it possible to attach or bridge to the network by plugging the ESP8266 into a USB port on a networked linux box, e.g. a Raspberry Pi?

Or actually, why do we even need the ESP8266 at all? Can we somehow just attach a NRF24L01 variant to a Raspberry Pi or Arduino Ethernet or something else? (Again might be a silly thought on my part.)

sidoh commented 6 years ago

@daandupau -- you'll notice that this is for the ESP IoT SDK. This is not compatible with the platform this project is developed on top of, unfortunately.

The ask in this ticket is possible, it's just a really large task.

@Streep -- no, any platform capable of sending commands to the NRF will work. Probably out of scope for this project, though =)

Streep commented 6 years ago

no, any platform capable of sending commands to the NRF will work. Probably out of scope for this project, though =)

Would be great, right now it's a little user unfriendly with the extra step of adding the Wi-Fi password. Having a wired interface to the network would be an improvement. Also more responsive of course. I wish I had time figuring out something like this. :)

PetricaM commented 6 years ago

I did open the issue mainly due to the fact that, at the respective time, the gateways became unresponsive.

Meanwhile, I did managed to significantly improve the reliability of the MiLight network by doing the followings:

However, although adding ethernet to ESP8266 would bring some improvements to the table (by decreasing lag and eliminating a vector of attack), I think these would be minor and instead developing a wired solution for ESP32 would be a more logical path.

ESP32 is a beast compared to ESP8266 and it would be possible for a wired ESP32 MiLight gateway to achieve response times similar to Philips Hue.

Adding ethernet to ESP8266 would likely improve the response times; however, the board is not powerful enough to support 50-60 Hz processing.

WIFI only boards are still an option to improve coverage (to achieve something like a MiLight mesh network).

wertzui commented 3 years ago

Is there any progress on an Ethernet build?

pfink commented 3 years ago

Wouldn't it work "out-of-the box" if you just use https://github.com/henryk/openmili with a nRF51-DONGLE (pca10031) on any device that could run Python and has an Ethernet and USB interface (e.g. Raspberry Pi)? On the Raspberry, it should also be possible to directly connect a nRF module via SPI and use openmili.

sidoh commented 3 years ago

Henryk reverse engineered one of the earlier protocols, but his work didn't include the newer devices (I reversed this one).

Regardless of the core RF drivers, there's a ton of plumbing in this project which can't trivially be ported onto another platform.

It's certainly doable, would just be a fair amount of work, and would be a divergence. I'm happy to support anyone who wants to take this on, but not something I see myself doing in the foreseeable future.

pfink commented 3 years ago

I'll just give a try for the NRF24L01+ via SPI on the Raspberry Pi using openmili. Surely, it will be a very minimalistic approach not comparable to this project, as I personally just have milight V3 devices (which should be covered by Henryk I guess) and I don't really need the web-server and MQTT part. In case you have any hints for things that should be taken into consideration already in advance, feel free to share them, I'm thankful for any support. NRF24L01+ is already on it's way to me 👍

And yes, I totally understand that providing support for a wired bridge at the same level as this project does is a project on it's own with significant efforts. I also don't plan to integrate with your code base as I'd like to keep it simple and stupid.

pfink commented 3 years ago

By the way, if anyone is looking out for a wired possibility already now, you could take a look at my project https://github.com/pfink/rfled-server-python. This one has support for multiple wired bridges, but you still need to buy one milight bridge per 4 zones and do some hardware modifications on them. So now I try to go for the next step getting completely rid of the Milight hardware and the 4-zones per bridge restriction :)

pfink commented 3 years ago

Communication with the bulbs wasn't really flawless with other libraries I tried, so other than planned I tried to integrate with this one as it seems to be the best-maintained code supporting all relevant protocols. Will track these efforts in a seperate issue #701 as it is not really an EthernetClient build as suggested in this issue

Asino123 commented 2 years ago

I did open the issue mainly due to the fact that, at the respective time, the gateways became unresponsive.

Meanwhile, I did managed to significantly improve the reliability of the MiLight network by doing the followings:

  • split MQTT topics; initially I had the same topic for each light on both gateway and HA (one for commands and one for states) then I've changed the ones for the gateway and set an automation on HA to forward payloads back and forth; having the same topics on gateway and HA had the effect that sending a payload from the state topic to command topic would set the gateway in a loop and knocked it out;
  • using higher quality ESP8266 boards such as Wemos D1 R2 (pin to pin compatible with Arduino Uno) and which seem to provide stable enough power supply;
  • powering the boards through the 12V rail (instead of the microUSB connector);
  • adding capacitors between GND and VCC pins of the NRF24L01.

However, although adding ethernet to ESP8266 would bring some improvements to the table (by decreasing lag and eliminating a vector of attack), I think these would be minor and instead developing a wired solution for ESP32 would be a more logical path.

ESP32 is a beast compared to ESP8266 and it would be possible for a wired ESP32 MiLight gateway to achieve response times similar to Philips Hue.

Adding ethernet to ESP8266 would likely improve the response times; however, the board is not powerful enough to support 50-60 Hz processing.

WIFI only boards are still an option to improve coverage (to achieve something like a MiLight mesh network).

Hi, I'm following your directions to create a more stable milight hub. I bought the Wemos D1 R2 as you indicated and the improvements have been enormous. I would like to install the capacitors, could you tell me which one to buy and where to mount them please? Have a nice day.

PetricaM commented 2 years ago

Hi,

I used a 4.7uF 50V electrolytic capacitor (nothing fancy, have a box with all sorts of capacitors of different sizes and caps). It needs to go between GND and VCC pins of the module:

https://i.stack.imgur.com/WQW4o.png

LE: forgot to mention, the capacitor is polarized thus you need to use the correct wiring. The longer leg is the positive pin however if you remove it from an older electronics board you cannot differentiate between legs (in all cases look on the side with the dashes as it indicates the negative pin).

LE2: actually you could try with another capacitor to reverse polarity. It makes for a very exciting and also satisfying (but very brief) pop 😃

Asino123 commented 2 years ago
  • NRF24L01

:-) thank you very much for the reply. I ordered the capacitors, as soon as they arrive I install them. I would not like to take advantage of your availability but I would still ask you four questions: 1 - do you power the Wemos D1 at 12V DC? 2 - did you use the firmwar e for the D1 or the D1 PRO? 3 - does the NRF24L01 module receive power from the Wemos D1 or did you use an additional power supply? 4 - I don't understand what I should do on home assistant when "you divide MQTT topics; initially I had the same topic for each spy on both gateway and HA (one for commands and one for states) then I changed those for the gateway and setting an automation on HA to forward payloads back and forth; having the same gateway and HA arguments had the effect that sending a payload from the state argument to the command argument would set the gateway to a cycle and would have eliminated it; " Thanks in advance and sorry for the inconvenience.

PetricaM commented 2 years ago
  1. Yes, 12V barrel plug to power the module; I've seen you mentioned above the module you bought, but just to be sure: D1 R2 is a large board, similar in size to Arduino Uno; D1 mini and mini pro are the small boards;
  2. D1 mini firmware (D1 mini pro board has a slightly larger memory than non pro)
  3. Power from the Wemos board; you can use separate power to the NRF module but since 3.3V is not as readily available (compared to 5V), you would need to use a buck converter; anyway, even with separate power modules, the two boards need common grounding;
  4. This is not needed. Things have changed in HA from the time I wrote the comment.
Asino123 commented 2 years ago

Thanks for your advice and your availability. I will have to do another 3 hubs, you advise me to buy other wemos boards (https://www.amazon.it/ARCELI-ESP8266-sviluppo-Compatibile-programma/dp/B07J2QKNHB/ref=sr_1_1?__mk_it_IT=%C3%85M%C3 % 85% C5% BD% C3% 95% C3% 91 & crid = 228UAHSA49Q39 & keywords = d1 + r2 + wemos & qid = 1657107768 & sprefix = d1 + r2 + wemos% 2Caps% 2C74 & sr = 8-1) D1 R2 or do I buy some D1 mini pro? Good day

PetricaM commented 2 years ago

I don't think there would be large differences between D1 mini, D1 mini pro and D1 R2 so you might go with the cheapest version.

But, WTF? I've seen the prices on Amazon and they're huge. I've purchased the boards circa 2017 (over Aliexpress so I had to wait for about 2 months to arrive but they were like 1.25 USD for D1 mini and 2 USD for D1 R2...)

Asino123 commented 2 years ago

Thanks to the availability. Yes, I also buy from Aliexpress. This time I needed it urgently because my nodemcu V3 were giving me problems like milight hub.