search

Blackymas / NSPanel_HA_Blueprint

This allows you to configure your complete NSPanel via Blueprint with UI and without changing anything in the code
1.33k stars 247 forks source link

Error while compiling V4.02 #1116

Closed lecrams closed 9 months ago

lecrams commented 10 months ago

When I try to compile V4.02 I get the following error! What can be wrong INFO ESPHome 2023.9.0 INFO Reading configuration /config/esphome/esp-panel-2.yaml... INFO Detected timezone 'Europe/Amsterdam' WARNING GPIO4 is a Strapping PIN and should be avoided. Attaching external pullup/down resistors to strapping pins can cause unexpected failures. See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins INFO Generating C++ source... INFO Compiling app... Processing nspanel2 (board: esp32dev; framework: arduino; platform: platformio/espressif32@5.4.0)

HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash

My yaml is this: substitutions:

CHANGE ME

device_name: "nspanel2" wifi_ssid: "xxx" wifi_password: "xxxxxx"

CHANGE ME
DO NOT CHANGE ANYTHING!

nextion_update_url: "http://192.168.178.87:8123/local/nspanel.tft"

nextion_update_url: "http://192.168.178.87:8123/local/nspanel.tftā€œ

packages:

download esphome code from Github

remote_package: url: https://github.com/Blackymas/NSPanel_HA_Blueprint ref: main files: [nspanel_esphome.yaml] refresh: 300s

DO NOT CHANGE ANYTHING!
edwardtfn commented 10 months ago

Looks like your compiler is crashing during the process. This could be related to ESPHome trying to reserve more memory than available in your system. There are cases like this reported (in other projects) by people using like RPi3 with 1Gb RAM...

I would suggest the following to make sure you have the cleanest environment before start compiling:

  1. Clean the build files - On the ESPHome dashboard, find your device, click on the 3-dot menu and then in "Clean build files"
  2. Add the following to your ESPHome settings (the one you shared above - can be at the end of the file): https://github.com/Blackymas/NSPanel_HA_Blueprint/blob/main/docs/en/customization.md#compiling-esphome-on-lower-powered-machines
  3. Reboot host - Not only rebooting Home Assistant, but also rebooting the host system to make sure it free up as much memory as possible.
  4. Try to install again.
lecrams commented 10 months ago

I have 4 Gb on my odroid N2. Despite this amount of memory your suggestions worked on on eof the panel that I have. Another one still has the still the problem described above.

Any other suggestions to clean up memory?

edwardtfn commented 10 months ago

Mmmm Memory shouldn't be an issue with your system. Do you have any special setting with this panel? Is the error happening at exactly same point?

padrekopre commented 10 months ago

I have similar problem with compiling :

https://pastebin.com/kJ9Gq8nW

Running Clean Builds not helping, any one know whats the problem ?

edwardtfn commented 10 months ago

Which version of ESPHome compiler/add-on you have? This requires v2023.5.0 at least.

lecrams commented 10 months ago

I am using ESPhome V 2023.9.1

edwardtfn commented 10 months ago

Yeah, I understood that, but the logs provided by @padrekopre shows a different issue and looks like outdated ESPHome in that case.

For your case I'm working right now to clean up the code a bit. I will let you know and will ask you to try again when I get some progress.

edwardtfn commented 10 months ago

By the way, you can compile v3.4.1 with no issues, right?

lecrams commented 10 months ago

Yes compiling V3,4.1 did not have any issues

iw0der commented 10 months ago

Hi guys, I have RPI3 with 1GB RAM and after version 2023.5.5 I can no longer update the panel; if I try now with version 2023.9.3 I get the following error message during compilation even after updating the blueprint to the latest version (4.0.2); I've tried a bit of everything, I've deleted, I've recompiled but it always ends up like this:

INFO ESPHome 2023.9.3
INFO Reading configuration /config/esphome/nspanel-zn.yaml...
INFO Detected timezone 'Europe/Rome'
Processing nspanel-zn (board: esp32dev; framework: arduino; platform: platformio/espressif32@5.4.0)
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Dependency Graph
|-- AsyncTCP-esphome @ 2.0.1
|-- WiFi @ 2.0.0
|-- FS @ 2.0.0
|-- Update @ 2.0.0
|-- ESPAsyncWebServer-esphome @ 3.1.0
|-- DNSServer @ 2.0.0
|-- ESPmDNS @ 2.0.0
|-- ArduinoJson @ 6.18.5
|-- WiFiClientSecure @ 2.0.0
|-- HTTPClient @ 2.0.0
Compiling .pioenvs/nspanel-zn/src/main.cpp.o
xtensa-esp32-elf-g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
*** [.pioenvs/nspanel-zn/src/main.cpp.o] Error 1
========================= [FAILED] Took 52.94 seconds =========================

Over the months I have tried with the various versions and always ended up with a compilation error on the main; I hoped that some update could solve it but I see that there is no solution yet... at least within reach of us, rookies :-)

I had already had a similar problem in the past but I solved it by updating the blueprint and add compile_process_limit... this time I'm stuck, I hope to read here some tips to regain control, thanks!

73, Arturo.

lecrams commented 10 months ago

For me it sometimes helped rebooting HA and doing a clean build after that. Otherwise you also can try to build it in a PC environment

edwardtfn commented 10 months ago

I agree with the suggestions from @lecrams, but before you go for that, may I ask you to try the latest code from dev. That one isn't fully operational and I don't think you should use that in the end, but what I'm doing there is trying to clean-up things the make it less resource hunger. If you can give a try, just change in your settings, in the remote_package area, from ref: main to ref: dev and try compiling, then send here your results (successfully or not).

After that, change back to ref: main to go back to the stable code and I have some recommendations for that case also:

  1. Add the following to your yaml (more details here): 
    esphome:
compile_process_limit: 1
  2. On ESPHome dashboard, on your device, click the 3-dot menu and select "Clean build files"
  3. Before trying to compile, restart you Home Assistant host system, to make sure you have a fresh system
  4. Then try to compile again

By the way, are you aware you can still install v3.4.1 just by changing that to ref: v3.4.1? It might work in your case if you don't succeed with the clean machine compilation.

iw0der commented 10 months ago

I agree with the suggestions from @lecrams, but before you go for that, may I ask you to try the latest code from dev.

Tried everything (dev, v3.4.1, v3.4, beta), always the same error; probably if I could go back with the EspHome version I would be able to make the old version work... but calculating the 7-8 hundred seconds just for each compilation and the long restart times, I have already invested a lot of time :-( and I'll stop here for the moment also to not mess up everything else that works and is surviving the continuous updates of Home Assistant!

Thank you, Arturo.

lecrams commented 10 months ago

Thx for the effort anyway. I will then stick to building it on a PC. Only one step extra to get a binary!

iw0der commented 10 months ago

Hi!

I try again, even after having done a bit of cleaning and after each update, but nothing... it always ends up like this:

HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Dependency Graph
|-- AsyncTCP-esphome @ 2.0.1
|-- WiFi @ 2.0.0
|-- FS @ 2.0.0
|-- Update @ 2.0.0
|-- ESPAsyncWebServer-esphome @ 3.1.0
|-- ESPmDNS @ 2.0.0
|-- ArduinoJson @ 6.18.5
|-- WiFiClientSecure @ 2.0.0
|-- HTTPClient @ 2.0.0
Compiling .pioenvs/nspanel-zn/src/main.cpp.o
xtensa-esp32-elf-g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
*** [.pioenvs/nspanel-zn/src/main.cpp.o] Error 1
========================= [FAILED] Took 71.94 seconds =========================

Now I was thinking ...:

73, Arturo.

edwardtfn commented 10 months ago

How complex could it be to create a lite version of the blueprint that survives compilation on low-resource hosts?

I'm almost there. :) I'm working on breaking things on parts, so it would be easier to select the parts you want and this will make possible this "lite" version you mentioned. Also, I'm working on supporting ESP-IDF framework, which is a bit lighter (hopefully also when compiling). I've succeeded with this, but in the end some of my recent changes broke the upload TFT, so I have to go back to this.

By the way, it would be nice if you could try the new code from DEV with esp-idf famework. This can be done with something like this (remember that won't work to transfer TFT, but I can propose something else for that if you can compile like this):

substitutions:
  ###### CHANGE ME START ######
  device_name: "YOUR_NSPANEL_NAME" 
  wifi_ssid: !secret wifi_ssid
  wifi_password: !secret wifi_password

  nextion_update_url: "http://homeassistant.local:8123/local/nspanel_eu.tft"

  ##### CHANGE ME END #####

packages:
  remote_package:
    url: https://github.com/Blackymas/NSPanel_HA_Blueprint
    ref: dev
    files:
      - nspanel_esphome.yaml # Core package
      # - nspanel_esphome_advanced.yaml # activate advanced (legacy) elements - can be useful for troubleshooting
      # - nspanel_esphome_addon_climate_cool.yaml # activate for local climate (cooling) control
      # - nspanel_esphome_addon_climate_heat.yaml # activate for local climate (heater) control
    refresh: 300s

##### My customization - Start #####
esphome:
  compile_process_limit: 1

esp32:
  board: esp32dev
  framework:
    type: esp-idf
##### My customization - End #####

The changes are on ref: dev and in the "My customization" area.

Please let me know if that worked.

A rookie who would like to update the firmware via PC what steps should he follow, and does this make sense?

You have to install Python and ESPHome in your PC and then use that to compile. There are some videos about this, like this one: https://youtu.be/a3iay-g1AsI?t=65&si=eA1GbT5Y3LM9ZSwo (only the topic related to compiling with your PC, as you still wanna to connect it to HA you don't have to follow the full video instructions, which are about something else).

it's time to abandon the RPI3 in favor of a 4 or perhaps soon the 5; despite everything else does it work smoothly?

Well, I believe the problem is more in the lack of RAM than the platform itself, but probably the best way is to move to an entirely new system with more memory. However, if you can compile ESPHome with your PC you probably can save that money for now and live with your RPi3 as this is probably OK for everything else in Home Assistant.

lecrams commented 10 months ago

I will not be able to test today! I will find some time tomorrow.

Since for me it is not a problem getting ESPhome up and running on a PC (I am a embedded SW engineer) it doesn't mean it is so obvious for everyone. However I think with the youtube link it should be possible for everyone to get a PC based ESPhome environment.

Not sure if it is indeed a lack of RAM. I have 4 Gb of RAM and still I get the error.

edwardtfn commented 10 months ago

You are probably right. 4Gb shouldn't be a problem.

iw0der commented 10 months ago

I'm almost there. :) I'm working on breaking things on parts...

A lot of work but I think it's a good idea. Great!

I've succeeded with this, but in the end some of my recent changes broke the upload TFT, so I have to go back to this.

I understand, sometimes it doesn't take much to get into trouble... the other day I was damned because I had used a strndup() instead of memcpy() inside a function that manipulated a list... it was under my nose but It took me a while to realize that the problem came from that...

By the way, it would be nice if you could try the new code from DEV with esp-idf famework.

No luck, now the compilation doesn't start at all...

INFO ESPHome 2023.10.1
INFO Reading configuration /config/esphome/nspanel-zn.yaml...
INFO Updating https://github.com/Blackymas/NSPanel_HA_Blueprint@dev
INFO Detected timezone 'Europe/Rome'
Failed config

http_request: [source /data/packages/891b50f5/nspanel_esphome_addon_upload_tft.yaml:14]

  This feature is incompatible with esp-idf.
  id: httpclient

...I'll try something later; Now, I'm busy.

73, Arturo.

iw0der commented 9 months ago

Hi!

In the meantime, I'll try to update the code and launch the rebuild; always in a hurry and with increasingly negative results... recently I have noticed two notable things:

and in any case it always ends +/- like this:

-- Found Git: /usr/bin/git (found version "2.30.2") 
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- The ASM compiler identification is GNU
-- Found assembler: /data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc -- broken
-- Configuring incomplete, errors occurred!
See also "/data/build/nspanel-zn/.pioenvs/nspanel-zn/CMakeFiles/CMakeOutput.log".
See also "/data/build/nspanel-zn/.pioenvs/nspanel-zn/CMakeFiles/CMakeError.log".

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
CMake Error at /data/cache/platformio/packages/tool-cmake/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message):
  The C compiler

    "/data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /data/build/nspanel-zn/.pioenvs/nspanel-zn/CMakeFiles/CMakeTmp

    Run Build Command(s):/data/cache/platformio/packages/tool-ninja/ninja cmTC_7c766 && [1/2] Building C object CMakeFiles/cmTC_7c766.dir/testCCompiler.c.obj
    [2/2] Linking C executable cmTC_7c766
    FAILED: cmTC_7c766 
    : && /data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc -mlongcalls -Wno-frame-address   CMakeFiles/cmTC_7c766.dir/testCCompiler.c.obj  -o cmTC_7c766   && :
    /data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/../libexec/gcc/xtensa-esp32-elf/8.4.0/liblto_plugin.so: error loading plugin: /data/cache/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/../libexec/gcc/xtensa-esp32-elf/8.4.0/liblto_plugin.so: cannot open shared object file: No such file or directory
    collect2: error: ld returned 1 exit status
    ninja: build stopped: subcommand failed.

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  /data/cache/platformio/packages/framework-espidf/tools/cmake/project.cmake:298 (__project)
  CMakeLists.txt:3 (project)

======================== [FAILED] Took 1731.72 seconds ========================

73, Arturo.

edwardtfn commented 9 months ago

Sorry for the slow reply here... The past days have been quite busy. :(

So, it looks like some issue in your ESPHome installation. Have you tried reinstalling the Add-on or doing a deeper cleaning like this? In both cases, please backup first. šŸ˜‰

lecrams commented 9 months ago

After the cleanup I can now compile without any error. Thx for the suggestions. Hopefully it will not reoccur with a next release of NSpanel

iw0der commented 9 months ago

no luck... :-(

...cleaned up those few files present, uninstalled esphome, restarted home assistant, re-installed esphome and restored the yaml files, then started the compilation:

INFO ESPHome 2023.10.3
INFO Reading configuration /config/esphome/nspanel-zn.yaml...
INFO Updating https://github.com/Blackymas/NSPanel_HA_Blueprint@dev
INFO Detected timezone 'Europe/Rome'
INFO Generating C++ source...
INFO Compiling app...
Processing nspanel-zn (board: esp32dev; framework: espidf; platform: platformio/espressif32@5.4.0)
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - framework-espidf @ 3.40405.230623 (4.4.5) 
 - tool-cmake @ 3.16.9 
 - tool-ninja @ 1.10.2 
 - toolchain-esp32ulp @ 2.35.0-20220830 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Installing ESP-IDF's Python dependencies
Collecting urllib3<2
  Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Collecting cryptography<35.0.0,>=2.1.4
  Downloading cryptography-3.4.8.tar.gz (546 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:

As mentioned before, one of the recurring errors is "fatal error: ffi.h: No such file or directory" and immediately after "ERROR: Could not build wheels for cffi which use PEP 517 and cannot be installed directly"; then after 1900 seconds of effort it ends without a firmware image ready to send to the device!

73, Arturo.

edwardtfn commented 9 months ago

Please try this:

  remote_package:
    url: https://github.com/Blackymas/NSPanel_HA_Blueprint
    ref: dev
    files:
      - nspanel_esphome_core.yaml
      # - nspanel_esphome.yaml # Core package
      # - nspanel_esphome_advanced.yaml # activate advanced (legacy) elements - can be useful for troubleshooting
      # - nspanel_esphome_addon_climate_cool.yaml # activate for local climate (cooling) control
      # - nspanel_esphome_addon_climate_heat.yaml # activate for local climate (heater) control
    refresh: 300s

This is excluding TFT upload engine.

iw0der commented 9 months ago

same result :-(

logs_nspanel-zn_run.txt

edwardtfn commented 9 months ago

@iw0der, this is definitely something with your ESPHome environment:

  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /root/.platformio/penv/.espidf-4.4.5/bin/python /root/.platformio/penv/.espidf-4.4.5/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-kvuufh0x/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --prefer-binary -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4'
Building wheels for collected packages: cffi
    Building wheel for cffi (PEP 517): started
    Building wheel for cffi (PEP 517): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /root/.platformio/penv/.espidf-4.4.5/bin/python /tmp/tmpld38torh_in_process.py build_wheel /tmp/tmpplvaq0s2
         cwd: /tmp/pip-install-lv5ie24v/cffi_5b212990943146049bbfb97815256e3f
    Complete output (55 lines):
    /tmp/pip-build-env-_mydfpkt/overlay/lib/python3.9/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
    !!

            ********************************************************************************
            The license_file parameter is deprecated, use license_files instead.

            By 2023-Oct-30, you need to update your project and remove deprecated calls
            or your builds will no longer be supported.

            See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
            ********************************************************************************
creating build/temp.linux-armv7l-cpython-39/src/c
    arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/.platformio/penv/.espidf-4.4.5/include -I/usr/include/python3.9 -c src/c/_cffi_backend.c -o build/temp.linux-armv7l-cpython-39/src/c/_cffi_backend.o
    src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory

*************************************************************
* Looking for ffi.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:ffi.h"
* Web  > https://registry.platformio.org/search?q=header:ffi.h
*
*************************************************************

       15 | #include <ffi.h>
          |          ^~~~~~~
    compilation terminated.

...

This last yaml I've shared will be installing the bare minimum for this project. Certainly much lighter than the previous versions (3.x), so I don't think the issue is related to the NSPanel project's files being too much for your environment.

Have you tried compiling in your computer instead?

iw0der commented 9 months ago

this is definitely something with your ESPHome environment

ok, now it's difficult for me to try anything else, also because ultimately everything else works like a charm; at this point I'll wait to migrate to new hardware (RPi4 or maybe RPi5 if they do the porting) to see if I won't be left with the problem anymore...

Have you tried compiling in your computer instead?

No, maybe I'll do more; I'll try again by setting up an HA instance under my virtualizer (XEN) but now I have no idea if this is possible/compatible... I'll have to check, maybe I'll come back with some news later.

At the moment, Thanks for all...

73, Arturo.

PS: perhaps the problem could resolve itself shortly; I understand that it has been present on various systems for a while and that it has also been addressed on EspHome pull #5473

iw0der commented 9 months ago

Hi Guys.

Now, in doubt whether there is already a solution in the development repository, I'm also trying that version of ESPHOME... but too bad, the blueprint code immediately fails!

INFO ESPHome 2023.11.0-dev
INFO Reading configuration /config/esphome/nspanel-zn.yaml...
INFO Detected timezone 'Europe/Rome'
Failed config

and then, at this point:

    - service: notification_show
      variables: 
        label: string

        [text] is an invalid option for [variables].
        text: string

73, Arturo.

edwardtfn commented 9 months ago

You are running a dev version of ESPHome... Have you asked at their dev channel about any breaking change related to this?

iw0der commented 9 months ago

Have you asked at their dev channel about any breaking change related to this?

Maybe... based on this esphome/esphome#5473 !?

edwardtfn commented 9 months ago

I don't see anything on esphome/esphome#5473 that could be breaking the variables on scripts.

edwardtfn commented 9 months ago

Have you asked at their dev channel about any breaking change related to this?

Maybe... based on this esphome/esphome#5473 !?

Ah, ok... now I got your point.