Open mjkl-gh opened 3 years ago
Thanks for this. I understand what you are looking to do, and I can certainly see the benefits of such a pattern and I think your buffered stream approah sounds good.
Are you thinking of this as an API extension to the framework or a feature which may be configured within the UI in addition to the APIs to support clients of the serial stream?
Regardless, finding a decent approach here could certainly help with the other requirements such as remote logging. I'll be slow to reply during the working week - generally evenings only.
I never really saw the need for pushing Serial over WiFi to be honest. In my view it would be easier to collect the data, perform any crunching, normalizing and push out the enriched data to the Web UI (via a socket), MQTT (as payload) or dump into a remove syslog (as log lines). Like Rick I hit the end of ESP8266 and struggled with its limited heap memory and fragmentation so switched exclusively now to the ESP32 which is a blessing I can tell you. I'm slowing working on tweaking esp8266-react to use esp32_https_server which brings another level of security with signed certs.
btw @mjkl-gh I did a small project for the p1meter based on esp8266-react which I never finished, including the circuit design which I can share.
@proddy I think abandoning support for 8266 for certain features is probably a good idea for the framework. I can see why you would do that for given all the features you pack into your project.
Not wanting to get too off-topic on this thread. As it wasn't looking like TLS support is going to come to AsyncTCP any time soon - development there is more or less dead. Investigating a different set of libraries for the Web Server and MQTT (which also uses AsyncTCP and therefore can't support TLS either) was very high on my list - intrested to hear how you get on using an alternative WebServer lib - especially when it comes to timing critical stuff. I'm wondering a re-write on idf might be a better long term solution - their WebServer and MQTT implementations support TLS.
If this feature is too much for the esp8266, we should be prepared to say it's ESP32 only.
If this feature is too much for the esp8266, we should be prepared to say it's ESP32 only.
yup, agree. how about a spanking new repo called esp32-react and I'll help you port it over to IDF, using the standard libraries. If we run into limitations we can always switch to any of the alternatives out there for web & mqtt. It's been on my todo list for a while also.
I never really saw the need for pushing Serial over WiFi to be honest. In my view it would be easier to collect the data, perform any crunching, normalizing and push out the enriched data to the Web UI (via a socket), MQTT (as payload) or dump into a remove syslog (as log lines).
Totally agree in the case of an endproduct or endgoal. In that case serial over tcp usually doesn't make sense if that's all you want you're probably better off programming it with esp-link and call it a day. A project like that doesn't need a nice interface as it will probably be used only once.
However, as there are a ton of serial devices in the world it is a good point to get started. Using ser2net one can use an esp as a virtual serial device on a Linux server across the building. And first explore the serial connection using python libraries for serial devices out there before moving to a (self-written) C++ implementation.
In case it wasn't clear. ( I didn't mention it in this issue). This is less of a Feature request and more of a "Hey I'm developing this, maybe you're interested too." After that it became apparent you guys already had some ideas on this.
On the esp32 part. I'm all for it. I must admit the only esp8266 I've got came in devices (tuya smart bulbs etc etc). I myself haven't tried IDF yet what are the advantages?
I've not used it either other than a bit of tinkering. I'm imagining a few of the advantages might be:
Feature wise, it's really complete: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/index.html
However, it's not written like Arduino code where the primary objective is being multi-system and developer friendly. The SDK provides a more fully featured but lower level abstraction. So that probably would put many people off and would immediately reduce the ecosystem of libraries and tools at your disposal - so there's a significant trade off.
@proddy Yes, I think a collaborative effort and a clean(ish) slate would be good if going down the IDF route. I'm planning to spend some time this week learning the SDK and working with some of the SDK's examples. I see no reason why the SDK provided server, client and peripheral utilities wouldn't be sufficient. I'm not sure however if there will be many alternatives if working with IDF directly as the majority of the community maintained libraries are built for the Arduino core.
Send me an email if you are interested in discussing this - perhaps we could have a chat this weekend?
As promised:
Many thanks for all the pointers. Some stuff I've already seen, some looks very interesting. I will definetly look into it
Unfortunately, I would class myself as a C++ developer with just enough knowledge to know what not to start on. Anyhow, my goals are somewhat similar to esp-link. As in create a simple serial to tcp bridge (and back). My ideas were:
In my final application I want to utilize an m5stickc to read out my dutch utilities smart meter. All these smart meters come with a serial port which can be used to feed data into a smarthome application. I already have some working prototypes at friend's and family's houses.
The reason for requiring a serialservice is that I eventually also want to create an MQTT service for automatically logging the data as sensors to home assistant using autoconfigure. More,over include some internal sensors such as battery level. The reason for this is that some meters only supply 100ma from their ports so they need some efficiency tweaking.