search

letscontrolit / ESPEasy

Easy MultiSensor device based on ESP8266/ESP32
http://www.espeasy.com
Other
3.23k stars 2.2k forks source link

Feedback required: New sets of plugins/controllers #2590

Closed TD-er closed 2 years ago

TD-er commented 4 years ago

The last few weeks we're running out of the space available for the sketch. The test and dev builds now approach their max size so something has to change.

I think the sets of normal, test and dev does not really make sense anymore.

We already have some builds specific for some use case, like specific hardware or just a single purpose. For example the Sonoff POW as a specific build for off the shelf hardware and the IR builds for the IR RX/TX and AC control.

Maybe we can make more logical build sets.

For this we need some user input to get an idea of sensors often used together or so often they must be included in any build.

Other options are to disable some of the webserver parts if they are not needed.

So please let's get some proper lists of used plugins to redefine the build scripts.

Domosapiens commented 4 years ago

My applications are:

all supported by generic: LCD/OLED display, Touch,IR,RCWL detection, Dummy Devices, Analog, NTP and System measurement

reporting to: 2# Domoticz controllers via HTTP, P2P, Rules

fly74 commented 4 years ago

My plugins in use:

DS18B20 / BME280 SSD1306 /w. and wo. framed 7Segment Dummy Device Switch

P2P Controller Generic HTTP controller

Rules

uzi18 commented 4 years ago

maybe add just some kind of stats and send it periodically to server, so we will know minimal needed features, then extra stuff will be available for own/self build or with configurator for patreons

I don't use espeasy personally but help for my friends to use it :) some improvements was dedicated for them

stats: nettemp controller mqtt controller dallas gpio bme/bmx pcf lcd

Budman1758 commented 4 years ago

My 2 cents... A much more detailed tutorial on setting up the necessary components to self-compile the firmware with just the plugins needed for your specific use case, If it were a bit easier to understand the process to build your own versions that would reduce the need for 20 to 30 plus bin files every release. I don't think you will ever make a set of pre-built bins that will please everyone. While this would not completely solve the issue methinks it would help immensely. This way you could have some very generic builds for most setups and build a few device specific bins for stuff like Sonoffs etc.

ghtester commented 4 years ago

I am using or would like to start using in future these devices/features:

Analog input - internal Display - LCD2004 Display - OLED SSD1306 Energy (AC) - Eastron SDM120C/220T/230/630 Environment - DHT11/12/22 SONOFF 2301/7021 Environment - DS18b20 Environment - BMx280 GAS - SGP30 Generic - Dummy Device Generic - MQTT Import Generic - Pulse Counter Input - iButton Keypad - TTP229 Touch Light/Lux - BH1750 Output - DomoticzMQTT Helper Position - HC-SR04 RFID-PN532 Regulator - Level Control Switch Input - Rotary Encoder Switch Input - Switch

Infrared Receive Infrared Transmit

Controllers: Generic HTTP ThingSpeak Domoticz MQTT

Notifications: Email (SMTP)

BTW. on my primary node I am still running this (a bit older but great) ESP Easy release: GIT version: mega-20180311 Uptime: 99 days 0 hours 43 minutes Load: 29% (LC=11244) Free Mem: 14696 (3176 - parseTemplate3) Wifi RSSI: -87 dB :-)

2bontb commented 4 years ago

I use the following

Devices: Sonoff Basic (running 1M OTA) Sonoff S20 (running 1M OTA) Sonoff POW (running hard SONOFF POW)

Wemos + Sensors, having temp/hum/presure/light often combined with PIR and sometimes with Pulsecount (running 4M normal)

Wemos + dedicated for Itho (own 4M normal build and added itho from plugin playground [adapted to fit new web interface])

Wemos + display + 3 gpio's (running 4M normal)

All Sensors use : system info

Controlers: FHEM

Using: Syslog, NTP

Running on all systems Release mega-20190827

HV-NL commented 4 years ago

I am using the following combinations:

DS18B20
DS18B20
Dummy device
HTU21D
HTU21D
OLED SSD1306
Switch
HTU21D
OLED SSD1306
APDS9950 (stripped version, derived from APDS9960)
Switch
SHT30
DHT22
Switch
P1 WiFi Gateway

Many of these combinations use Rules All are communicating with Domoticz HTTP and many are writing results to syslog

My devices are

micropet commented 4 years ago

I would submit a simple personal config file where I can select the controllers and the plugins.

For people who can not compile themselves, a "normal" version should be enough.

If you want more, you have to worry about how to compile a file. Of course, a simple tutorial is very useful.

TD-er commented 4 years ago

@micropet This evening I've been fixing the good old memanalyzer.py file, also with this in mind. So now I know again how to generate a build from within Python having all setup from the command line.

The other way around is already possible, as can be seen in the config of the custom build. That one uses a Python script to define the plugins to use. Both could be used to read a file, but then you have to change the file again if you want to change the config. Another option is to read from a database, where a config is stored from someone clicking away :)

I think if we just allow to read from a file -if exists- with nothing else than a list of plugins and controllers, then anyone can make its own preferred set. We also have the custom.h file, but that's a bit harder to work with if you ask me.

micropet commented 4 years ago

So you mean a website that lists all plugins and controllers. Then I just do a ckeckmark to the required componenets. That would be great, of course.

I never understood how it works with the Python script. custem.h is also mysterious

I've always fiddled with define_plugin_sets.h.

TD-er commented 4 years ago

Just as an initial test, could you checkout the latest from the mega branch and go to tools/vagrant. There is a (very short) readme about what to install and then go to that folder with a terminal/shell. In Windows it can be CMD or Windows PowerShell and in Linux it can be any terminal.

Then type vagant up Please let me know if anything fails. If it is finished building something in the build folder, then you can type vagrant halt in the same window you typed vagrant up.

It is just preliminary, so I will add some support for reading TXT files defining what needs to be built, but this is not bad for a morning work I think.

LeeNX commented 4 years ago

@TD-er oh, I like this idea of trying to build in a VM! Almost like a single command to get a standard build tool-chain.

Current implementation looks to be possible a little broken. There were some red screen stuff scroll by, so I don't know what was broken just yet, but I did do the process in a screen recording, so I can upload what happen from my OSX machine.

I do seem to have four bins files thou

vagrant@vagrant:~/GitHub/letscontrolit/ESPEasy/dist/bin$ ls -la
total 23832
drwxrwxr-x 2 vagrant vagrant     4096 Sep  2 09:37 .
drwxrwxr-x 3 vagrant vagrant     4096 Sep  2 09:37 ..
-rw-rw-r-- 1 vagrant vagrant 16777216 Sep  2 09:37 blank_16MB.bin
-rw-rw-r-- 1 vagrant vagrant  1048576 Sep  2 09:37 blank_1MB.bin
-rw-rw-r-- 1 vagrant vagrant  2097152 Sep  2 09:37 blank_2MB.bin
-rw-rw-r-- 1 vagrant vagrant  4194304 Sep  2 09:37 blank_4MB.bin
-rw-rw-r-- 1 vagrant vagrant   276848 Sep  2 09:37 ESPEasy_2step_UploaderMega_1024.bin
vagrant@vagrant:~/GitHub/letscontrolit/ESPEasy/dist/bin$ date
Mon Sep  2 09:40:48 UTC 2019
vagrant@vagrant:~/GitHub/letscontrolit/ESPEasy/dist/bin$

Might it be worth moving the discussion to an issue of it's own, instead of high-jacking your Feedback request?

My feedback on what I use ESPEasy for/with ... Every cheap sensor that I can find that ESPEasy supports now ... ;-) .

Can you not possible pull the web site stats on what sensors people search for or click throu?

Thanks for all the awesome tools!

TD-er commented 4 years ago

Hmm, I think it may be interesting to see what went wrong here. I just tested it on Windows here, so that's why I did add the sed command in the provisioning to remove the CRLF and replace then by the Unix line endings.

I guess you're right in moving it to a separate issue about Vagrant.

LeeNX commented 4 years ago

@TD-er I don't know if anything went wrong, just saw some red messages, as I seem to have a build. Maybe it's just warnings.

Included me in the new issue, if you interest in any feedback I could give. I have text screen recording to upload some place, if that helps?

TD-er commented 4 years ago

Please continue in #2594 and I would like to see the screen recording. The red messages could be build errors, but it is best if you check the ZIP file in the (new created) build dir: tools/vagrant/build There should be one extra bin file like this: image

micropet commented 4 years ago

I think that will not work for me. I have Hyper-V installed on all computers. That seems to be incompatible with VirtualBox. I'll keep looking.

TD-er commented 4 years ago

Maybe Vagrant can also work together with other virtualization environments?

I chose Vagrant since it is the defacto product for running deployment tests during development. Can you run "Ubuntu on Windows" with Hyper-V active? (See Windows store for "Ubuntu") If so, then we can split the scripts so you can also run the same tools only start them with another command.

LeeNX commented 4 years ago

@micropet vagrant supports hyper-v - https://www.vagrantup.com/docs/hyperv/

Am I missing something?

TD-er commented 4 years ago

@LeeNX There is a warning about HyperV on the Vagrant site

LeeNX commented 4 years ago

I think the warning is about VirtualBox's vs Hyper-V Hypervisors, but from the link I provided, I would believe that vagrant natively supports M$ Hyper-V Hypervisor. Could be something else too. There are warnings about Hyper-V and VMWare too.

I have not tried Vagrant on Windows, so I am very much just an arm chair suggestor ... ;-)

TD-er commented 4 years ago

Not all builds of the bento/ubuntu-18.04 support HyperV, but this build does.

Maybe if you just change the relevant parts of the Vagrantfile into this:

Vagrant.configure("2") do |config|
  config.vm.box = "bento/ubuntu-18.04"
  config.vm.box_version = "201812.27.0"
end

So add the box_version line right after this one:

config.vm.box = "bento/ubuntu-18.04"

I have not explicitly set the provider, so I guess it could detect HyperV then.

micropet commented 4 years ago

I get the following error:

d:\Soft\ESPEasy-mega\tools\vagrant>vagrant up Vagrant failed to initialize at a very early stage:

There was an error while executing VBoxManage, a CLI used by Vagrant for controlling VirtualBox. The command and stderr is shown below. Command: ["--version"] Stderr: Der Befehl ""c:\Program Files\Oracle\VirtualBox\VBoxManage.exe"" ist entweder falsch geschrieben oder konnte nicht gefunden werden.

Stupid question: Does Virtualbox additionally need to be installed, or is Hyper-V sufficient?

And what should the Vagrant do? I thought we were talking about a website with boxes to tick the required plugins.

LeeNX commented 4 years ago

I am guessing, as I don't use Windows nor M$ Hyper-V, but I think you are not meant to have VirtualBox installed on a Windows Computer that has Hyper-V setup. At least, that is how I read - https://www.vagrantup.com/docs/hyperv/

LeeNX commented 4 years ago

@micropet - I think the vagrant discussion is better moved to #2594

Vagrant build system will let you build custom firmware and plugins, without having to do more than git clone and vagrant up in the correct folder. Later some basic config files will let you build custom firmware for your ESP device with only the plugin and controllers you need ... I think ... ;-)

micropet commented 4 years ago

Thank you.

TD-er commented 4 years ago

Just a quick reply, to let you know I will reply to this in the Vagrant issue :)

berbergh commented 4 years ago

To begin with, I need the test version for as long as some devices are not set to Normal. I noticed that of the mega-20190903, the 252 test core of 4M is gone.

So far, my devices are: Mostly a Wemos with:

I would like to get MQTT working, but it cuases severe crashes on all devices.

TD-er commented 4 years ago

To begin with, I need the test version for as long as some devices are not set to Normal. I noticed that of the mega-20190903, the 252 test core of 4M is gone.

If a build is not included in the ZIP file, it could very well be the binary size exceeded the limit for a sketch size. (if you flash one that's too large, it will corrupt the existing partitioning or just continue to crash) In the latest builds, the file name of the "_4M" builds has also changed to "_4M1M" to make it clear it is using 1 MB SPIFFS on 4 MB flash. This to differentiate the "_4M2M" builds which have 2 MB SPIFFS. Currently there is only one _4M2M build, but better make it consistent throughout all builds

About the MQTT crashes. Do you have any special demands? Have you tried lowering the timeout to 100 msec? What MQTT controller do you use?

I have MQTT running on all my nodes without issues, but that's all Domoticz MQTT connecting to Mosquito on a Pi.

berbergh commented 4 years ago

About the MQTT crashes. Do you have any special demands? Have you tried lowering the timeout to 100 msec?

No special demands at all. I have set time out to 100 msec now.

What` MQTT controller do you use? I have MQTT running on all my nodes without issues, but that's all Domoticz MQTT connecting to Mosquito on a Pi.

I have the same setup. Domoticz MQTT and Mosquito on a Pi.

In the latest builds, the file name of the "_4M" builds has also changed to "_4M1M" to make it clear it is using 1 MB SPIFFS on 4 MB flash. This to differentiate the "_4M2M" builds which have 2 MB SPIFFS. Currently there is only one _4M2M build, but better make it consistent throughout all builds

Which one should I use for Wemos 4MB?

TD-er commented 4 years ago

Well, right now only the IR related build has the 4M2M layout. (and still the old 4M1M) All others have the 4M1M layout, so I guess there is yet not even a choice here. Just go for 4M1M. The 4M2M is meant to allow for more room on SPIFFS, but it has not yet been tested. If you change from one to the other you will loose all settings, because the filesystem is recreated and formatted.

s0170071 commented 4 years ago

back to the initial request:

I use these pluigins: SHT25 temp/ humidity (my code, in the playground) SMA inverter, modbus (my code, in the playground) MT681 SML power meter (my code, in the playground) ViessmanVitotronic optical interface (my code, in the playground) IRTX

all with mqtt except the viessmann, that one uses http.

TD-er commented 4 years ago

Last night's build was somewhat of a disaster if you look at the bin files that made it into the zip. Only 1 "test" build and no "dev" build included. So I will split up these, but the question is: "how?"

shaman1010 commented 4 years ago

sonoff pow2 (AC meter), BME 180/280, MQTT, Dummy.

May be linux-way? Small base core (for minimum size of flash 512kB), and overlays ( https://github.com/pvvx/esp8266web ) Overlays - added from web interface esp8266. One device (DS, MQTT, BME, etc) = one overlay.

Or server with own builder. And checkboxes. ( https://wifi-iot.com/p/main/ )

TD-er commented 4 years ago

Or server with own builder.

See: https://github.com/letscontrolit/ESPEasy/issues/2594

TD-er commented 4 years ago

I was thinking about how to split the build sets. On the forum, it was suggested to also have a look at how Tasmota is handling build sets

What about the controllers? Is anyone using a MQTT controller with something other than generic/advanced HTTP? I guess Domoticz HTTP/MQTT should be in the same build set, since they share a lot of code.

My suggestion:

And use that including the existing normal/test/dev builds? Maybe also have one set of "metering" plugins, like interfacing with Modbus devices. And one for "playful " plugins like

Those are not useful for most users. Doesn't mean a build with these should only have these, so we may also need some "standard set" of plugins included in all builds (except for "switch only" builds)

Just for some idea of plugin/controller sizes the output of the memory analyzer run. The rather big ones are marked bold.

module cache IRAM init RAM r.o. RAM uninit RAM Flash ROM
CORE 29268 1584 2960 37096 568404
MQTT_ONLY 0 68 356 1192 35936
USE_SETTINGS_ARCHIVE 0 0 140 0 9664
WEBSERVER_RULES_DEBUG=1 0 -8 0 0 16
WEBSERVER_TIMINGSTATS 0 -8 4 -8 5216
src/_C001.ino 0 -8 8 8 5344
src/_C002.ino 0 68 40 1144 18592
src/_C003.ino 0 0 4 24 2400
src/_C004.ino 0 -8 0 16 3008
src/_C005.ino 0 -4 8 1144 10320
src/_C006.ino 0 -4 8 1144 10400
src/_C007.ino 0 -8 0 32 3056
src/_C008.ino 0 -8 4 8 5184
src/_C009.ino 0 64 36 8 7360
src/_C010.ino 0 -8 0 32 4112
src/_C011.ino 0 0 4 24 7792
src/_C012.ino 0 0 0 32 4512
src/_C013.ino 0 -8 0 16 1328
src/_C014.ino 0 -4 332 1152 19472
src/_C015.ino 0 0 1144 120 7712
src/_C016.ino 0 -8 0 32 5216
src/_C017.ino 0 72 40 24 4768
src/_C018.ino 404 -8 104 72 18928
src/_N001_Email.ino 0 -8 4 -8 4000
src/_N002_Buzzer.ino 0 -8 0 0 688
src/_P001_Switch.ino 0 -8 40 8 13696
src/_P002_ADC.ino 0 0 0 16 2496
src/_P003_Pulse.ino 220 -8 0 384 2512
src/_P004_Dallas.ino 0 -8 28 112 4352
src/_P005_DHT.ino 24 0 16 0 2064
src/_P006_BMP085.ino 0 0 0 16 2112
src/_P007_PCF8591.ino 0 0 0 0 480
src/_P008_RFID.ino 240 0 0 16 1472
src/_P009_MCP.ino 0 -8 16 0 10768
src/_P010_BH1750.ino 0 -8 0 0 2544
src/_P011_PME.ino 0 -8 0 0 4032
src/_P012_LCD.ino 0 8 64 0 5568
src/_P013_HCSR04.ino 0 -8 0 48 5856
src/_P014_SI7021.ino 0 -8 0 0 1744
src/_P015_TSL2561.ino 0 -8 0 16 4112
src/_P016_IR.ino 464 0 680 120 33648
src/_P017_PN532.ino 0 -8 0 64 2192
src/_P018_Dust.ino 0 -8 0 0 640
src/_P019_PCF8574.ino 0 0 16 0 10704
src/_P020_Ser2Net.ino 0 -8 8 24 3184
src/_P021_Level.ino 0 -8 0 16 2176
src/_P022_PCA9685.ino 0 -8 4 -8 8368
src/_P023_OLED.ino 0 -8 40 -8 5808
src/_P024_MLX90614.ino 0 -8 0 0 960
src/_P025_ADS1115.ino 0 0 16 0 3120
src/_P026_Sysinfo.ino 0 -8 0 0 3440
src/_P027_INA219.ino 0 8 4 24 2816
src/_P028_BME280.ino 128 8 28 32 7008
src/_P029_Output.ino 0 -8 0 0 512
src/_P030_BMP280.ino 128 0 0 64 3072
src/_P031_SHT1X.ino 0 -8 0 0 2704
src/_P032_MS5611.ino 0 -8 0 48 2640
src/_P033_Dummy.ino 0 0 0 0 2400
src/_P034_DHT12.ino 0 -8 0 0 736
src/_P035_IRTX.ino 76 -8 868 -8 21888
src/_P036_FrameOLED.ino 0 0 0 144 32672
src/_P037_MQTTImport.ino 0 -4 12 1200 12736
src/_P038_NeoPixel.ino 128 -8 20 -8 5968
src/_P039_Thermocouple.ino 0 0 12 16 1968
src/_P040_ID12.ino 0 -8 0 0 768
src/_P041_NeoClock.ino 128 -8 0 0 3360
src/_P042_Candle.ino 128 0 12 32 8672
src/_P043_ClkOutput.ino 0 0 0 0 1680
src/_P044_P1WifiGateway.ino 0 -8 28 64 4416
src/_P045_MPU6050.ino 0 -8 0 64 4464
src/_P046_VentusW266.ino 0 0 0 0 0
src/_P047_i2c-soil-moisture-sensor.ino 0 -8 0 0 3104
src/_P048_Motorshield_v2.ino 0 24 8 -8 6944
src/_P049_MHZ19.ino 404 0 44 16 9280
src/_P050_TCS34725.ino 0 -8 0 0 3344
src/_P051_AM2320.ino 0 -8 0 0 992
src/_P052_SenseAir.ino 404 -8 224 16 13312
src/_P053_PMSx003.ino 404 -8 12 16 7024
src/_P054_DMX512.ino 0 0 0 0 1696
src/_P055_Chiming.ino 0 -8 4 -8 3616
src/_P056_SDS011-Dust.ino 404 -8 12 16 7952
src/_P057_HT16K33_LED.ino 0 0 48 0 4192
src/_P058_HT16K33_KeyPad.ino 0 0 32 0 1376
src/_P059_Encoder.ino 432 64 16 48 3888
src/_P060_MCP3221.ino 0 -8 32 0 2240
src/_P061_KeyPad.ino 0 -8 64 0 2352
src/_P062_MPR121_KeyPad.ino 0 -8 16 0 1840
src/_P063_TTP229_KeyPad.ino 0 -8 4 -8 1808
src/_P064_APDS9960.ino 0 0 0 0 4944
src/_P065_DRF0299_MP3.ino 404 -8 28 16 4144
src/_P066_VEML6040.ino 0 0 16 0 2672
src/_P067_HX711_Load_Cell.ino 0 0 4 88 8224
src/_P068_SHT3x.ino 0 0 0 0 1776
src/_P069_LM75A.ino 0 -8 32 0 1104
src/_P070_NeoPixel_Clock.ino 0 -8 0 0 16
src/_P071_Kamstrup401.ino 404 0 16 16 7040
src/_P072_HDC1080.ino 0 -8 0 0 800
src/_P073_7DGT.ino 0 -8 120 -8 8704
src/_P074_TSL2591.ino 0 -8 28 0 4096
src/_P075_Nextion.ino 404 -8 76 16 10560
src/_P076_HLW8012.ino 432 0 8 24 8304
src/_P077_CSE7766.ino 0 -8 0 0 3280
src/_P078_Eastron.ino 404 -8 52 8 9840
src/_P079_Wemos_Motorshield.ino 0 0 0 0 2096
src/_P080_DallasIButton.ino 0 -8 0 16 2560
src/_P081_Cron.ino 0 56 608 16 8624
src/_P082_GPS.ino 432 0 136 24 17536
src/_P083_SGP30.ino 0 -8 0 0 1248
src/_P084_VEML6070.ino 0 -8 0 0 1472
src/_P085_AcuDC243.ino 532 0 240 16 12336
src/_P086_Homie.ino 0 -8 4 -8 3904
src/_P087_SerialProxy.ino 404 -8 12 16 8640
src/_P088_HeatpumpIR.ino 0 8 624 320 20752
TD-er commented 4 years ago

Just the log of last night's build.

Status Reason Env
SUCCESS 829456 < 1044464 Bytes. custom_ESP8266_4M1M
SUCCESS 1689376 < 1900544 Bytes. esp-wrover-kit_test_1M8_partition
SUCCESS 1689392 < 1900544 Bytes. esp32test_1M8_partition
SUCCESS 863296 < 892912 Bytes. normal_ESP8266_1M
SUCCESS 863664 < 892912 Bytes. normal_ESP8266_1M_VCC
SUCCESS 863248 < 892912 Bytes. normal_ESP8285_1M
SUCCESS 818192 < 892912 Bytes. normal_core_241_ESP8266_1M
SUCCESS 865088 < 1044464 Bytes. normal_WROOM02_2M
SUCCESS 865088 < 1044464 Bytes. normal_core_252_WROOM02_2M256
SUCCESS 864992 < 1044464 Bytes. normal_ESP8266_4M1M
SUCCESS 819344 < 1044464 Bytes. normal_core_241_ESP8266_4M1M
FAILED No file created. normal_core_252_ESP8266_16M
SUCCESS 871712 < 1044464 Bytes. normal_core_260_sdk222_alpha_ESP8266_4M1M
FAILED No file created. normal_core_260_sdk222_alpha_ESP8266_16M
SUCCESS 614016 < 616448 Bytes. minimal_core_242_ESP8266_1M_OTA
SUCCESS 614016 < 616448 Bytes. minimal_core_242_ESP8285_1M_OTA
FAILED too large 650064 > 616448 Bytes. minimal_core_252_ESP8266_1M_OTA
FAILED too large 650048 > 616448 Bytes. minimal_core_252_ESP8285_1M_OTA
SUCCESS 839584 < 892912 Bytes. minimal_IRext_ESP8266_1M
FAILED No file created. minimal_IRext_ESP8266_4M1M
SUCCESS 840944 < 1044464 Bytes. minimal_IRext_ESP8266_4M2M
SUCCESS 956512 < 1044464 Bytes. normal_IRext_no_rx_ESP8266_4M2M
SUCCESS 1012000 < 1044464 Bytes. test_core_260_sdk222_alpha_ESP8266_4M1M
SUCCESS 1012704 < 1044464 Bytes. test_core_260_sdk3_alpha_ESP8266_4M1M
FAILED No file created. test_core_260_sdk222_alpha_ESP8266_16M
SUCCESS 1013904 < 1044464 Bytes. test_ESP8266_4M_VCC
SUCCESS 1004544 < 1044464 Bytes. dev_ESP8266_4M1M
SUCCESS 710672 < 1044464 Bytes. hard_SONOFF_POW_4M1M
FAILED No file created. hard_other_POW_ESP8285_1M
FAILED No file created. hard_Shelly_1_2M256
FAILED No file created. hard_Ventus_W266

The "No file created." is reported when at build the file size already exceeds the set value in platformio.ini. (or another build error)

jimmys01 commented 4 years ago

minimal_IRext_ESP8266_4M1M minimal_IRext_ESP8266_4M2M Those are practicaly the same build but the one compiled and the other not.. Also in my pc the latest code builds fine.

TD-er commented 4 years ago

I've noticed the same. It could be the full path in which the build is made may just be the tipping point. We can switch those to core 2.6.0, since that one has the brand new feature of flash string de-duplication. That may save a few kByte in binary size. Perhaps a lot more on the IR lib.

Edit: The "minimal" 4M1M build may be checking against 1M minimal size.

TD-er commented 4 years ago

I pinned the issue, since it is really important to have a proper definition of builds. Almost all testand devbuilds no longer fit the sketch size.

clumsy-stefan commented 4 years ago

As I anyway use my own set of plugins, I can't really comment here, but one thing I noticed is that even when only using the one controller plugin I need (C009 FHEM) and one Plugin (P001 Switch) compiling it with Core 2.6.0 I already get over the maximum size for 1M untis for using the two-step OTA. So therefore my vote would be to add some more stringent BUILD_MINIMAL_OTA or additional definitions which strip out everything not needed (eg. icons, infos, debugs, etc.) and so it fits again in small (1M) builds.

Grovkillen commented 4 years ago

The new GUI engine (which is not yet available for beta) will allow for external hosting* of the GUI so that would be ideal for the 1M units.

*Meaning you can surf to a big brother (4M) unit and from that one control the 1M unit(s).

TD-er commented 4 years ago

I have been ding some static code analysis and the C009 controller does appear quite high on the resources used list. The do_process_c009_delay_queue call alone uses 7.6 kByte. It also uses JSON, which needs an external library.

Plugin_001 does use 11.2 kB

TD-er commented 4 years ago

Should the minimal build also have the web-log present?

Grovkillen commented 4 years ago

Should the minimal build also have the web-log present?

No I'd say that's not in the scope of those small units.

clumsy-stefan commented 4 years ago

Should the minimal build also have the web-log present?

No I'd say that's not in the scope of those small units.

Agree, especially if it saves significantly. It's still possible to use syslog for logging/debugging though

mackowiakp commented 4 years ago

Since I am probably the oldest here (almost 60 years) and I started my computer education in 1980, I see that the whole ESP project has reached the border with which I have encountered many times in IT history. Please, do not treat this as rude, but I think the question is not right. The problem is not which device/sensor to choose for a particular distribution. The problem is how to get FW containing all the drivers needed in a particular application, ignoring all the others, even the most popular ones. Here comes a comparison with the history of the Novell NetWare system. Version 1.XX (based on the XNS protocol - probably no one remembers that it is the first Ethernet protocol) only supported specific types of hardware. Of course, with time new types of new equipment arrived. Therefore, version 2.XX introduced a system build/compilation at the installation stage. During this compilation, it was possible to add (from diskette) device drivers supplied by various manufacturers. This process required some knowledge and was not universal. Any hardware change required a new build/compilation. And adding too many drivers resulted in exceeding the computer resources, for example, memory. I think the ESP easy project is at this stage. To have your own set of sensors, we basically need to compile our own version of FW. At the end of the last century, Novell introduced version 3.xx. It was probably the first system with dynamically loaded kernel and driver modules. Something that is common today was a revolution at the time. So I think the future is a similar solution in the case of ESP. At the flash stage, we select from the future version of esptool, using checkboxes, a list of necessary sensors. And the ESPeasy "kernel" contains only what is directly on the PCB, for example, nodeMCU. If the number of drivers exceeds the device resources, esptool will inform us. And besides, the story of Arduino does not end today. Device resources will increase, not decrease. The number and types of sensors will also increase. Therefore, the use of a system of dynamically loaded modules seems even more necessary. These are my - maybe a little too long - comments on the choice of sensors.

Grovkillen commented 4 years ago

@mackowiakp I think you're 100% correct in your analysis of the problem we're facing. That is why we plan on making a "compile-as-you-go" (online) service. But until we have time for that we want to create a set of commonly used sets of firmware releases. And we will most likely still provide pre-compiled versions for development and Swiss-army-knife units.

TD-er commented 4 years ago

I totally agree with your post, but sadly there is no possibility for our setup (using C++ in the way we use it) to load components at run time. For that we should move to another language like MicroPython. (see upyEasy project)

As you may have seen, I was working on a Vagrant setup to ease the process of self-compiling and I do think there's the future. Let the node decide what it needs. But we're not there yet. So until then we should make a solid set of application driven plugins. Maybe it is already sufficient to split at controller level, like MQTT and non-MQTT, since those are the biggest controllers. Also LoRaWAN and network related controllers will hardly ever be used together.

mackowiakp commented 4 years ago

@ Grovkillen .But remember. Swiss-army-knife has 23854 units, or more. Except the one You just need. Murphy is alive!

Misiu commented 4 years ago

As you may have seen, I was working on a Vagrant setup to ease the process of self-compiling and I do think there's the future.

I this that this will be THE biggest feature yet🎉 User picks the board (device) he has (ideally a list of devices like Wemos D1, Shelly 1, etc), he selects the features he needs and he clicks the "COMPILE" button. This is the feature that should be available only for active supporters (patreon users only?). The goals would be completed in no time! :)

I think that there should be a list of devices that the user can select from because it isn't very intuitive what bin we should select now. Take a look at espurna releases page I think it's easier to select bin file named shelly2.bin than to know what chip the device has inside. The configurator should allow selecting the device and based on that it would select the correct chip. What do You think?