Custom Vagrant compilation failed

[ghtester]

After the latest source & Custom.h changes I am not able to compile the Custom firmware using Vagrant anymore. The compilation log ends with:

default: Compiling .pio/build/custom_IR_ESP8266_4M1M/FrameworkArduino/umm_malloc/umm_malloc.cpp.o default: Compiling .pio/build/custom_IR_ESP8266_4M1M/FrameworkArduino/umm_malloc/umm_poison.c.o default: Archiving .pio/build/custom_IR_ESP8266_4M1M/libFrameworkArduino.a default: Indexing .pio/build/custom_IR_ESP8266_4M1M/libFrameworkArduino.a default: Linking .pio/build/custom_IR_ESP8266_4M1M/ESP_Easy_mega_20220801_custom_IR_ESP8266_4M1M.elf default: /home/vagrant/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: .pio/build/custom_IR_ESP8266_4M1M/ESP_Easy_mega_20220801_custom_IR_ESP8266_4M1M.elf section .text1' will not fit in regioniram1_0_seg' default: collect2: error: ld returned 1 exit status default: ** [.pio/build/custom_IR_ESP8266_4M1M/ESP_Easy_mega_20220801_custom_IR_ESP8266_4M1M.elf] Error 1 default: ========================= [FAILED] Took 686.98 seconds ========================= default: Environment Status Duration default: ---------------------- -------- ------------ default: custom_IR_ESP8266_4M1M FAILED 00:11:26.984 default: ==================== 1 failed, 0 succeeded in 00:11:26.984 ==================== default: Error: Option '-e' requires an argument. default: Error: Option '-e' requires an argument. default: Error: Option '-e' requires an argument. default: Error: Option '-e' requires an argument. default: Error: Option '-e' requires an argument. default: Created /tmp/tmp.QWN1BI9Flg default: Created /tmp/tmp.QWN1BI9Flg/ESPEasy_collect_dist default: ### Creating zip archives default: default: zip error: Nothing to do! (/home/vagrant/GitHub/letscontrolit/ESPEasy/ESPEasy_ELF_files_20220801_vagrant.zip) default: ### Created ESPEasy_ELF_files_20220801_vagrant.zip default: cp: default: cannot stat 'build_output/bin/': No such file or directory default: ### Created ESPEasy_ESP82xx_20220801_vagrant.zip default: cp: default: cannot stat 'build_output/bin/ESP32' default: : No such file or directory

What should I do to fix the issue please?

[TD-er]

default: /home/vagrant/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: .pio/build/custom_IR_ESP8266_4M1M/ESP_Easy_mega_20220801_custom_IR_ESP8266_4M1M.elf section .text1' will not fit in regioniram1_0_seg'

This is the main issue. Sadly it is unclear which plugin may be causing this, but there is some plugin which is now using extra of the iRAM. Typically this is used by functions marked with IRAM_ATTR (or ICACHE_ATTR) or volatile variable declaration.

The Dallas plugin(s) have been changed recently which may attribute to this.

What you can do is to disable one or more plugins and see if it fits again.

The problem is that we have a really small amount of iRAM, so this is some build issue we really need to watch out for.

[tonhuisman]

Are you compiling a specific PR? I've seen PR 4161 (P014) fail with exactly that error, where other PR are building just fine (on Github Actions).

[ghtester]

Thank you for the quick response, I tried to disable the Dallas plugin & compiled again but it did not help. @tonhuisman No, just the source with already merged changes. But in my plugin set the P014 is enabled. I'll try to disable it.

Huh, I see the Dallas plugin is still enabled, maybe I did not save the Custom.h properly?!? Need to recompile again...

[tonhuisman]

Have you cleared your Vagrant environment before rebuilding? (Haven't used Vagrant much, but I seem to remember it keeps older files as much as possible) As continuing on an existing/previous build could also be the cause for this error, AFAICS.

[ghtester]

Yes that's what I am always doing, building the Vagrant environment from scratch by script (so the virtual machine is always destroyed and created before the build). Everything was working fine until the latest changes were introduced. The latest successful custom build I made was dated 20220726. No changes on my side except the Custom.h update with latest changes. Now I am trying to recompile with Dallas [P004] plugin enabled and with SI7021 [P014] disabled.

Edit - the compilation with Dallas [P004] plugin enabled and with SI7021 [P014] disabled failed again. Now trying to compile with Dallas disabled and SI7021 enabled.

[ghtester]

Hmm, the last attempt failed as well... :-( Just found the warning: # warning "Custom.h has '#define USE_RTTTL' to be replaced with '#define FEATURE_RTTTL 1 So I have updated the Custom.h accordingly and going to try the compilation again with both Dallas and SI7021 disabled

[ghtester]

Unfortunately I still have the same issue... :-( Hopefully I won't need to go through all plugins and test disabling them? The compilation takes some time so it would be a long-time and annoying process... Any help tip please?

[tonhuisman]

You can of course build separate builds locally using VSCode and PlatformIO, as documented here (instructions are based on Windows but are mostly usable on MacOS and Linux). Then it is much easier to find and fix problems in problematic builds, and, somewhat depending on the speed of the computer used, it should cost much less time for a single build-run.

[ghtester]

Thanks for the hint but I won't go this way. So far the Vagrant environment was working fine for my needs. It's a pity that with latest code changes it does not work for me anymore but I believe I can survive for now with older build...

[tonhuisman]

If you are 'only' building a Custom.h build, then why run all builds using Vagrant? Or you could create a pio_envlist.txt file to only build the env's you actually need.

Can you share (only) the list of enabled plugins and FEATURE_* settings from your Custom,h file?

[ghtester]

I already have the pio_envlist.txt and I am building just one build listed above - custom_IR_ESP8266_4M1M. The plugin list is this: ####################################################################################################### ########### Plugins ####################################################################################################### */

define FEATURE_SERVO 0 // Uncomment and set to 0 to explicitly disable SERVO support

define USES_P001 // Switch

define USES_P002 // ADC

define USES_P003 // Pulse

define USES_P004 // Dallas

define USES_P005 // DHT

// #define USES_P006 // BMP085

define USES_P007 // PCF8591

define USES_P008 // RFID

// #define USES_P009 // MCP

define USES_P010 // BH1750

// #define USES_P011 // PME

define USES_P012 // LCD

define USES_P013 // HCSR04

define USES_P014 // SI7021

// #define USES_P015 // TSL2561

define USES_P017 // PN532

// #define USES_P018 // Dust

define USES_P019 // PCF8574

define USES_P020 // Ser2Net

// #define USES_P021 // Level // #define USES_P022 // PCA9685

define USES_P023 // OLED

// #define USES_P024 // MLX90614 // #define USES_P025 // ADS1115

define USES_P026 // SysInfo

define USES_P027 // INA219

define USES_P028 // BME280

define USES_P029 // Output

// #define USES_P031 // SHT1X // #define USES_P032 // MS5611

define USES_P033 // Dummy

define USES_P034 // DHT12

// #define USES_P036 // FrameOLED

define USES_P037 // MQTTImport

// #define P037_MAPPING_SUPPORT 1 // Enable Value mapping support // #define P037_FILTER_SUPPORT 1 // Enable filtering support // #define P037_JSON_SUPPORT 1 // Enable Json support // #define USES_P038 // NeoPixel // #define USES_P039 // Environment - Thermocouple

// #define USES_P040 // RFID - ID12LA/RDM6300 // #define USES_P041 // NeoClock // #define USES_P042 // Candle

define USES_P043 // ClkOutput

// #define USES_P044 // P1WifiGateway // #define USES_P045 // MPU6050 // #define USES_P046 // VentusW266 // #define USES_P047 // I2C_soil_misture // #define USES_P048 // Motoshield_v2

define USES_P049 // MHZ19

// #define USES_P050 // TCS34725 RGB Color Sensor with IR filter and White LED

define USES_P051 // AM2320

// #define USES_P052 // SenseAir // #define USES_P053 // PMSx003 // #define USES_P054 // DMX512 // #define USES_P055 // Chiming // #define USES_P056 // SDS011-Dust // #define USES_P057 // HT16K33_LED // #define USES_P058 // HT16K33_KeyPad // #define USES_P059 // Encoder

// #define USES_P060 // MCP3221

define USES_P061 // Keypad

define USES_P062 // MPR121_KeyPad

define USES_P063 // TTP229_KeyPad

// #define USES_P064 // APDS9960 Gesture // #define USES_P065 // DRF0299 // #define USES_P066 // VEML6040 // #define USES_P067 // HX711_Load_Cell

define USES_P068 // SHT3x

// #define USES_P069 // LM75A

// #define USES_P070 // NeoPixel_Clock // #define USES_P071 // Kamstrup401 // #define USES_P072 // HDC1080 // #define USES_P073 // 7DG // #define USES_P074 // TSL2591 // #define USES_P075 // Nextion // #define USES_P076 // HWL8012 in POW r1 // #define USES_P077 // CSE7766 in POW R2

define USES_P078 // Eastron Modbus Energy meters

// #define USES_P079 // Wemos Motoshield

define USES_P080 // iButton Sensor DS1990A

// #define USES_P081 // Cron // #define USES_P082 // GPS

define USES_P083 // SGP30

// #define USES_P084 // VEML6070 // #define USES_P085 // AcuDC24x

define USES_P086 // Receiving values according Homie convention. Works together with C014 Homie controller

define USES_P087 // Serial Proxy

// #define USES_P088 // HeatpumpIR

define USES_P089 // Ping

// #define USES_P090 // CCS811 // #define USES_P091 // SerSwitch // #define USES_P092 // DLbus // #define USES_P093 // MitsubishiHP // #define USES_P094 // CULReader // #define USES_P095 // ILI9341 // #define USES_P096 // eInk // #define USES_P097 // ESP32Touch // #define USES_P098 // // #define USES_P099 // XPT2046 touchscreen

// #define USES_P100 // DS2423 counter

define USES_P101 // WakeOnLan

// #define USES_P102 // PZEM004Tv3 // #define USES_P103 // Atlas Scientific EZO Sensors (pH, ORP, EZO, DO) // #define USES_P104 // MAX7219 dotmatrix

define USES_P105 // AHT10/20/21

// #define USES_P106 // BME680 // #define USES_P107 // Si1145 // #define USES_P109 // ThermoOLED

// #define USES_P110 // VL53L0X Time of Flight sensor // #define USES_P111 // RF522 RFID reader // #define USES_P112 // AS7265x // #define USES_P113 // VL53L1X ToF // #define USES_P114 // VEML6075 // #define USES_P115 // MAX1704x // #define USES_P116 // ST77xx // #define USES_P117 // SCD30 // #define USES_P118 // Itho ventilation control // #define USES_P119 // ITG3205 Gyro // #define USES_P120 // ADXL345 Acceleration / Gravity // #define USES_P124 // I2C MultiRelay // #define USES_P125 // ADXL345 SPI Acceleration / Gravity

define USES_P126 // 74HC595 Shift register

// #define USES_P127 // CDM7160 // #define USES_P132 // INA3221 // #define USES_P133 // LTR390 UV

// #define USES_P128 // NeoPixelBusFX // #define P128_USES_GRB // Default // #define P128_USES_GRBW // Select 1 option, only first one enabled from this list will be used // #define P128_USES_RGB // #define P128_USES_RGBW // #define P128_USES_BRG // #define P128_USES_RBG // #define P128_ENABLE_FAKETV 0 // Enable(1)/Disable(0) FakeTV effect, dis$

// Special plugins needing IR library

define USES_P016 // IR

// #define P016_SEND_IR_TO_CONTROLLER false //IF true then the JSON replay solution is transmited back to the condroller.

define USES_P035 // IRTX

// #define P016_P035_Extended_AC // The following define is needed for extended decoding of A/C Messages and or using standardised //common arguments for controlling all deeply supported A/C units // #define P016_P035_USE_RAW_RAW2 //Use the RAW and RAW2 encodings, disabling it saves 3.7Kb // #define USES_P088 // Heatpump IR // #define USES_P108 // DDS238-x ZN Modbus energy meters

/ ####################################################################################################### ########### Controllers ####################################################################################################### /

// #define USES_C001 // Domoticz HTTP

define USES_C002 // Domoticz MQTT

// #define USES_C003 // Nodo telnet

define USES_C004 // ThingSpeak

define USES_C005 // Home Assistant (openHAB) MQTT

// #define USES_C006 // PiDome MQTT // #define USES_C007 // Emoncms

define USES_C008 // Generic HTTP

// #define USES_C009 // FHEM HTTP // #define USES_C010 // Generic UDP

define USES_C011 // Generic HTTP Advanced

// #define USES_C012 // Blynk HTTP

define USES_C013 // ESPEasy P2P network

define USES_C014 // homie 3 & 4dev MQTT

// #define USES_C015 // Blynk // #define USES_C016 // Cache controller // #define USES_C017 // Zabbix // #define USES_C018 // TTN/RN2483

/ ####################################################################################################### ########### Notifiers ####################################################################################################### /

define USES_N001 // Email

define USES_N002 // Buzzer

//#define LIMIT_BUILD_SIZE

define FEATURE_RTTTL 1 // to support RTTTL command in Custom build !!!

endif // ESPEASY_CUSTOM_H

Also I have an old definition at beginning (maybe it should be removed):

define DCORE_POST_3_0_0

[tonhuisman]

Also I have an old definition at beginning (maybe it should be removed):

define DCORE_POST_3_0_0

For that #define to work, it should be named CORE_POST_3_0_0, the original -D prefix comes from the Python scripts, or from a partial compiler command-line.

---

Your issue is a combination of trying to squeeze in all plugins you need for all ESP's you have installed, and including the IR libraries and plugins, that library uses a lot of iRam. Even when disabling the IR plugins (P016/P035), the libraries still use too much of iRam. We've tried to solve that before, but it's getting harder and harder. Btw: Building that set (No IR) using the Custom ESP8266 4M1M env works as intended.

My suggestion would be to maintain 2 Custom.h files, 1 non-IR that has all plugins needed for the units that don't have IR, and a separate IR build that includes IR stuff, but only the set of plugins used on those ESP's.

Or replace all ESP8266's by 16MB ESP32's 😁 that way you can have all plugins (MAX env's) available in all units.

[TD-er]

Do you really need C014? That code really needs a lot of clean-up and I started on it a few times but gave up as I may spend really a lot of time but will not have any test environment here to test it.

That code really is an extreme form of cut-paste-paste-paste-paste-paste-paste and not even "the good code". Every time I see that code appear in my code changes that may affect code all over the project I am tempted to stop even make that change as the code of that plugin really looks like Pandora might have left some boxes here.

[tonhuisman]

Turning off P008 could temporarily fix your build, we're already working on improvements there.

[ghtester]

@tonhuisman The IR is a must for me. I am using it on all nodes. It's a pity it became a bit complicated in near past, for my needs the command table in the IRRX plugin is useless as I am using Rules instead. ESP32 is an overkill for my needs. I also don't like to use several builds with different plugin sets... Thanks for the info about P008, I'll give it a try... @TD-er In fact I really don't need C014, thanks for the comment, I am going to disable it as well. I know it must be a hard job, fighting with Pandora boxes... ;-) Thank you both for help!

[TD-er]

ESP32 is an overkill for my needs. I also don't like to use several builds with different plugin sets...

Well based on your plugin list, I don't think it is overkill. Also it is more power efficient compared to ESP8266.

I will have a look at all the functions that now are using IRAM to see whether they all need it. Also I have recently added a lot more #include "../../ESPEasy_common.h" in files as they did need it but hadn't. This common file also manages the "translation" of the old ICACHE_RAM_ATTR to IRAM_ATTR to make sure any left-overs of the old attributes are translated and vice verse when compiling new IRAM_ATTR code with older SDKs.

But if it now shows up as a problem, this could mean it wasn't compiled with these attributes before and then we can ask ourselves whether it was needed at all. (or whether you actually needed that plugin) N.B. not having these includes in the right place may lead to unpredictable builds as the build order of things becomes very important, which is random.

[ghtester]

Well, I'll consider the reduction of number plugins (I really don't currently own all the enabled HW devices/sensors). I thought ESP32 would consume more power than ESP8266... Also the dimensions of ESP8266 are more suitable and I already have them (in opposite to ESP32's :-)) Thanks again for all information shared. I'm not sure if I understood well the remarks about compilation attributes but I'll double check the structure of my Custom.h with the current Custom-sample.h...

BTW. I can confirm the successful compilation with both P008 and C014 disabled.

[ghtester]

Just tested the compilation again with enabled P008 and C014. As it was successful, I am closing the issue. Thanks to all involved developers for a great work.