Installation Issues and New Guide

Ok guys i have read the other issues, i have read the tries that @edwardtfn is doing and i thank him very much for his efforts, but we seriously need to update the guides and find a way to be able to make this entire process as painless as possible.

At the moment i am trying to flash a few NEW nspanels. Yesterday it took me 2 hours (yes 2 hours) to manage and flash the TFT to one of the new nspanels... i have no idea what worked and what not.

Today i am trying to do the same. Same configuration.... it doesn't work. Yesterday the "beta" worked, today it doesn't. the "main" never worked.

the entire TFT nextion process is a pain in the butt and after the last ESP updates.... it almost NEVER works at the new devices. you have to sacrifice a goat and hope for the best.

Let me try and open one more issue, find a solution and maybe update the guides? (The videos in the "HOW TO" are old and should be the main reference)

here is some of the log file

[13:29:25][D][main:732]: Button pressed: Update TFT display [13:29:25][D][script.upload_tft:086]: Try #1 [13:29:25][D][script.nextion_status:2653]: Nextion status: [13:29:25][D][script.nextion_status:2654]: Is detected: True [13:29:25][D][script.nextion_status:2655]: Is setup: False [13:29:25][D][script.upload_tft:088]: Setting Nextion protocol reparse mode to passive [13:29:25][D][main:2562]: Exit reparse [13:29:25][D][script.upload_tft:070]: Wait 2 seconds [13:29:27][D][nextion_upload:130]: Nextion TFT upload requested [13:29:27][D][nextion_upload:131]: url: http://192.168.10.66:8123/local/nspanel_eu.tft [13:29:27][D][nextion_upload:178]: TFT file size: 7247932 [13:29:27][D][nextion_upload:190]: Updating Nextion [13:29:29][D][nextion_upload:218]: Upgrade response is [] [13:29:29][E][nextion_upload:223]: Preparation for tft update failed 0 "" [13:29:29][D][nextion_upload:254]: Restarting Nextion [13:29:31][D][script.upload_tft:093]: Turn off Nextion [13:29:31][D][switch:016]: 'nsp-kidroom Nextion display - Power' Turning OFF. [13:29:31][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state OFF [13:29:31][D][script.upload_tft:070]: Wait 3 seconds [13:29:34][D][script.upload_tft:096]: Turn on Nextion [13:29:34][D][switch:012]: 'nsp-kidroom Nextion display - Power' Turning ON. [13:29:34][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state ON [13:29:34][D][script.upload_tft:070]: Wait 10 seconds [13:29:44][D][script.upload_tft:086]: Try #2 [13:29:44][D][script.nextion_status:2653]: Nextion status: [13:29:44][D][script.nextion_status:2654]: Is detected: True [13:29:44][D][script.nextion_status:2655]: Is setup: False [13:29:44][D][script.upload_tft:088]: Setting Nextion protocol reparse mode to passive [13:29:44][D][main:2562]: Exit reparse [13:29:44][D][script.upload_tft:070]: Wait 2 seconds [13:29:46][D][nextion_upload:130]: Nextion TFT upload requested [13:29:46][D][nextion_upload:131]: url: http://192.168.10.66:8123/local/nspanel_eu.tft [13:29:46][D][nextion_upload:178]: TFT file size: 7247932 [13:29:46][D][nextion_upload:190]: Updating Nextion [13:29:48][D][nextion_upload:218]: Upgrade response is [] [13:29:48][E][nextion_upload:223]: Preparation for tft update failed 0 "" [13:29:48][D][nextion_upload:254]: Restarting Nextion [13:29:49][D][script.upload_tft:093]: Turn off Nextion [13:29:49][D][switch:016]: 'nsp-kidroom Nextion display - Power' Turning OFF. [13:29:49][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state OFF [13:29:49][D][script.upload_tft:070]: Wait 3 seconds [13:29:52][D][script.upload_tft:096]: Turn on Nextion [13:29:52][D][switch:012]: 'nsp-kidroom Nextion display - Power' Turning ON. [13:29:52][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state ON [13:29:52][D][script.upload_tft:070]: Wait 10 seconds [13:30:02][D][script.upload_tft:086]: Try #3 [13:30:02][D][script.nextion_status:2653]: Nextion status: [13:30:02][D][script.nextion_status:2654]: Is detected: True [13:30:02][D][script.nextion_status:2655]: Is setup: False [13:30:02][D][script.upload_tft:088]: Setting Nextion protocol reparse mode to passive [13:30:02][D][main:2562]: Exit reparse [13:30:03][D][script.upload_tft:070]: Wait 2 seconds [13:30:04][D][nextion_upload:130]: Nextion TFT upload requested [13:30:04][D][nextion_upload:131]: url: http://192.168.10.66:8123/local/nspanel_eu.tft [13:30:05][D][nextion_upload:178]: TFT file size: 7247932 [13:30:05][D][nextion_upload:190]: Updating Nextion [13:30:07][D][nextion_upload:218]: Upgrade response is [] [13:30:07][E][nextion_upload:223]: Preparation for tft update failed 0 "" [13:30:07][D][nextion_upload:254]: Restarting Nextion [13:30:08][D][script.upload_tft:093]: Turn off Nextion [13:30:08][D][switch:016]: 'nsp-kidroom Nextion display - Power' Turning OFF. [13:30:08][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state OFF [13:30:08][D][script.upload_tft:070]: Wait 3 seconds [13:30:11][D][script.upload_tft:096]: Turn on Nextion [13:30:11][D][switch:012]: 'nsp-kidroom Nextion display - Power' Turning ON. [13:30:11][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state ON [13:30:11][D][script.upload_tft:070]: Wait 10 seconds [13:30:21][E][script.upload_tft:100]: TFT upload failed. [13:30:21][D][script.upload_tft:101]: Turn off Nextion [13:30:21][D][switch:016]: 'nsp-kidroom Nextion display - Power' Turning OFF. [13:30:22][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state OFF [13:30:22][D][script.upload_tft:070]: Wait 2 seconds [13:30:23][D][script.upload_tft:104]: Turn on Nextion [13:30:23][D][switch:012]: 'nsp-kidroom Nextion display - Power' Turning ON. [13:30:23][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state ON [13:30:23][D][script.upload_tft:106]: Restarting esphome [13:30:23][D][script.upload_tft:070]: Wait 1 seconds [13:30:24][D][button:010]: 'nsp-kidroom Restart' Pressed. [13:30:24][I][restart.button:012]: Restarting device... [13:30:25][I][app:134]: Rebooting safely... [13:30:25][D][esp32.preferences:114]: Saving 1 preferences to flash... [13:30:25][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed WARNING nsp-kidroom: Connection error occurred: [Errno 104] Connection reset by peer INFO Processing unexpected disconnect from ESPHome API for nsp-kidroom WARNING Disconnected from API INFO Successfully connected to nsp-kidroom in 0.030s INFO Successful handshake with nsp-kidroom in 0.044s [13:30:59][D][api:102]: Accepted 192.168.10.66 [13:30:59][D][api.connection:1089]: Home Assistant 2023.11.3 (192.168.10.66): Connected successfully [13:30:59][D][time:045]: Synchronized time: 2023-12-01 13:30:59 [13:30:59][D][main:090]: System clock synchronized [13:30:59][E][uart:015]: Reading from UART timed out at byte 0! [13:30:59][W][nextion:077]: Nextion is not connected! [13:30:59][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:30:59][W][component:215]: Components should block for at most 20-30ms. [13:31:00][E][uart:015]: Reading from UART timed out at byte 0! [13:31:00][W][nextion:077]: Nextion is not connected! [13:31:00][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:00][W][component:215]: Components should block for at most 20-30ms. [13:31:00][E][uart:015]: Reading from UART timed out at byte 0! [13:31:00][W][nextion:077]: Nextion is not connected! [13:31:00][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:00][W][component:215]: Components should block for at most 20-30ms. [13:31:01][E][uart:015]: Reading from UART timed out at byte 0! [13:31:01][W][nextion:077]: Nextion is not connected! [13:31:01][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:01][W][component:215]: Components should block for at most 20-30ms. [13:31:02][E][uart:015]: Reading from UART timed out at byte 0! [13:31:02][W][nextion:077]: Nextion is not connected! [13:31:02][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:02][W][component:215]: Components should block for at most 20-30ms. [13:31:02][E][uart:015]: Reading from UART timed out at byte 0! [13:31:02][W][nextion:077]: Nextion is not connected! [13:31:02][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:02][W][component:215]: Components should block for at most 20-30ms. [13:31:03][E][uart:015]: Reading from UART timed out at byte 0! [13:31:03][W][nextion:077]: Nextion is not connected! [13:31:03][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:03][W][component:215]: Components should block for at most 20-30ms. [13:31:04][E][uart:015]: Reading from UART timed out at byte 0! [13:31:04][W][nextion:077]: Nextion is not connected! [13:31:04][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:04][W][component:215]: Components should block for at most 20-30ms. [13:31:04][E][uart:015]: Reading from UART timed out at byte 0! [13:31:04][W][nextion:077]: Nextion is not connected!

...

[13:31:19][D][main:2562]: Exit reparse [13:31:19][E][uart:015]: Reading from UART timed out at byte 0! [13:31:19][W][nextion:077]: Nextion is not connected! [13:31:19][W][component:214]: Component nextion.display took a long time for an operation (0.11 s). [13:31:19][W][component:215]: Components should block for at most 20-30ms. [13:31:20][D][nextion:066]: 0x1A error ignored during setup [13:31:20][I][nextion:083]: Nextion is connected

[13:31:20][C][nextion:124]: Device Model: NX4832F035_011C [13:31:20][C][nextion:125]: Firmware Version: 51 [13:31:20][C][nextion:126]: Serial Number: E0AF340135FFA277 [13:31:20][C][nextion:127]: Flash Size: 16777216 [13:31:20][C][nextion:128]: Wake On Touch: True [13:31:20][C][nextion:139]: Start Up Page : 8 [13:31:20][W][component:214]: Component nextion.display took a long time for an operation (0.09 s). [13:31:20][W][component:215]: Components should block for at most 20-30ms. [13:31:28][D][nextion:265]: Manually set nextion report ready [13:31:28][D][nextion:343]: Nextion is setup [13:31:28][D][script.boot_sequence:2575]: Starting boot sequence [13:31:28][D][script.boot_sequence:2576]: Wait for TFT version [13:31:28][D][script.nextion_status:2653]: Nextion status: [13:31:28][D][script.nextion_status:2655]: Is setup: True [13:31:28][D][script.nextion_status:2653]: Nextion status: [13:31:28][D][script.nextion_status:2655]: Is setup: True

[13:31:28][E][nextion:274]: Nextion queue is empty! [13:31:59][D][sensor:094]: 'ntc_source': Sending state 1.54900 V with 2 decimals of accuracy [13:31:59][D][resistance:039]: 'resistance_sensor' - Resistance 9907.9Ω [13:31:59][D][sensor:094]: 'resistance_sensor': Sending state 9907.93945 Ω with 1 decimals of accuracy [13:31:59][D][ntc:026]: 'nsp-kidroom Temperature' - Temperature: 25.2°C [13:31:59][D][sensor:094]: 'nsp-kidroom Temperature': Sending state 25.20828 °C with 1 decimals of accuracy [13:32:00][W][nextion:719]: Received unknown event from nextion: 0x55 [13:32:00][W][nextion:394]: Nextion reported variable name invalid!

this is driving me nuts. why doesn't the beta work today!

here is what happens when i press update TFT

[13:33:02][D][script.upload_tft.upload_tft_arduino:365]: Nextion TFT upload requested [13:33:02][D][script.upload_tft.upload_tft_arduino:366]: url: http://192.168.10.66:8123/local/nspanel_eu.tft [13:33:03][D][script.upload_tft.upload_tft_arduino:367]: baud_rate: 115200

[13:33:03][D][script.upload_tft.upload_tft_arduino:404]: Requesting URL: http://192.168.10.66:8123/local/nspanel_eu.tft [13:33:03][D][script.upload_tft.upload_tft_arduino:436]: Updating Nextion [13:33:03][D][script.upload_tft.send_nextion_command:170]: Sending: whmi-wris 7247932,115200,1 [13:33:03][D][script.upload_tft.upload_tft_arduino:460]: Waiting for upgrade response [13:33:05][D][script.upload_tft.upload_tft_arduino:465]: Upgrade response is [55.AA.87.13.00.7B.22.48.4D.49.5F.69.6E.74.65.72.66.61.63.65.22.3A.32.7D.93.57.00 (27)] [13:33:05][D][script.upload_tft.upload_tft_arduino:470]: Preparation for tft update failed 85 "U\xaa\x87" [13:33:05][D][script.upload_tft.uploadend:238]: Completed: 0 [13:33:06][D][script.upload_tft.uploadend:248]: Nextion TFT upload will try again [13:33:06][D][script.upload_tft:767]: Turn off Nextion [13:33:06][D][switch:016]: 'nsp-kidroom Nextion display - Power' Turning OFF. [13:33:06][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state OFF [13:33:06][D][script.upload_tft:156]: Wait 2 seconds [13:33:08][D][script.upload_tft:770]: Turn on Nextion [13:33:08][D][switch:012]: 'nsp-kidroom Nextion display - Power' Turning ON. [13:33:08][D][switch:055]: 'nsp-kidroom Nextion display - Power': Sending state ON [13:33:08][D][script.upload_tft:156]: Wait 5 seconds [13:33:13][D][script.upload_tft:765]: Try #3 [13:33:13][D][script.upload_tft.upload_tft_arduino:365]: Nextion TFT upload requested [13:33:13][D][script.upload_tft.upload_tft_arduino:366]: url: http://192.168.10.66:8123/local/nspanel_eu.tft [13:33:13][D][script.upload_tft.upload_tft_arduino:367]: baud_rate: 115200

[13:33:13][D][script.upload_tft.upload_tft_arduino:404]: Requesting URL: http://192.168.10.66:8123/local/nspanel_eu.tft [13:33:13][D][script.upload_tft.upload_tft_arduino:436]: Updating Nextion [13:33:13][D][script.upload_tft.send_nextion_command:170]: Sending: whmi-wris 7247932,115200,1 [13:33:13][D][script.upload_tft.upload_tft_arduino:460]: Waiting for upgrade response [13:33:15][D][script.upload_tft.upload_tft_arduino:465]: Upgrade response is [55.AA.87.13.00.7B.22.48.4D.49.5F.69.6E.74.65.72.66.61.63.65.22.3A.32.7D.93.57.00 (27)] [13:33:15][D][script.upload_tft.upload_tft_arduino:470]: Preparation for tft update failed 85 "U\xaa\x87"

here is the yaml...

substitutions:

###### CHANGE ME START ######

  device_name: "nsp-kidroom" 
  wifi_ssid: !secret wifi_ssid 
  wifi_password: !secret wifi_password

  #nextion_update_url: "http://192.168.3.60:8123/local/nspanel_eu_ted.tft" # URL to local tft File
  #nextion_update_url: "http://192.168.3.60:8123/local/nspanel_eu_411.tft" # URL to local tft File
  #nextion_update_url: "http://10.10.1.128:8123/local/nspanel_blank.tft" # URL to local tft File
  nextion_update_url: "http://192.168.10.66:8123/local/nspanel_eu.tft" 

  #nextion_update_url: "https://raw.githubusercontent.com/Blackymas/NSPanel_HA_Blueprint/main/nspanel_eu.tft" # URL to Github
  #nextion_update_url: "https://github.com/Blackymas/NSPanel_HA_Blueprint/raw/main/custom_configuration/nspanel_blank.tft" # URL to Github

##### CHANGE ME END #####

###### USE THIS ONLY FOR YOUR FIRST TFT UPLOAD, ONCE IT WORKED, REMOVE THESE LINES ######
#external_components:
#  - source: github://pr#5016
#    components: [nextion]
#    refresh: 1h
###############################################################################

packages:
  remote_package:
    url: https://github.com/Blackymas/NSPanel_HA_Blueprint
    ref: main
    files:
      - nspanel_esphome.yaml # Core package
      # - nspanel_esphome_addon_climate_cool.yaml # activate for local climate (cooling) control
      # - nspanel_esphome_addon_climate_heat.yaml # activate for local climate (heater) control
    refresh: 300s

##### My customization - Start #####
esp32:
  framework:
    type: esp-idf

#external_components:
#  - source: github://pr#5825
#    components: [nextion]
#    refresh: 1s

##### My customization - End #####

why doesn't the beta work today!

I wish I know the answer. 😞

In fact, there are many things that makes this upload hard. One is related to the limited memory on the ESP32, and I've worked a lot to reduce the memory needed to transfer the file, but also on removing not core code, so other things won't be fighting for that limited memory. Another problem is the active reparse mode used by Sonoff (and by some Tasmota implementations) which is not really supported by ESPHome. We have used some old PRs created in the past, but those PRs have always being rejected by ESPHome team (which I understand their reasons) and working with not released PRs are always a fragile solution, as other things changes breaking that old code. Earlier this week this happened again and I had to spend hours working in a new solution, which is the one in main now, but as that one is not perfect, I still working on the solution in beta. I've also got some PRs recently approved on ESPHome, but those may not come before ESPHome 2023.12 which is at least 3 weeks far from now. And those might not be included, so I cannot count on those.

So, what I am doing now on beta is to remove all those references to non released code. Still some to go, but I think we have improved.

Basically, the fight is to detect when the display is on reparse mode and try to exit it. We can send an "Exit reparse" command, but the display (Nextion) will leave it for a few seconds and then restart itself with the reparse mode enable again.

What changed on beta from yesterday to now was the timing between those actions. Those are all included in the boot sequence which could take up to 3 minutes. On that boot sequence, if the display is not detected for 20s after the boot, an exit reparse command is sent. If it still not detected after 45s after that (this was 20s yesterday), it will restart the display. Another thing I've changed is, now the system tests if the display is on reparse before the upload, and send an "exit reparse" if that is the case. The code from yesterday was always sending this call and this was messing up with displays already out of reparse.

I come back with some suggestions in another message in a minute.

Please try this code:

substitutions:

###### CHANGE ME START ######

  device_name: "nsp-kidroom" 
  wifi_ssid: !secret wifi_ssid 
  wifi_password: !secret wifi_password

  #nextion_update_url: "http://192.168.3.60:8123/local/nspanel_eu_ted.tft" # URL to local tft File
  #nextion_update_url: "http://192.168.3.60:8123/local/nspanel_eu_411.tft" # URL to local tft File
  nextion_update_url: "http://10.10.1.128:8123/local/nspanel_blank.tft" # URL to local tft File
  #nextion_update_url: "http://192.168.10.66:8123/local/nspanel_eu.tft" 

  #nextion_update_url: "https://raw.githubusercontent.com/Blackymas/NSPanel_HA_Blueprint/main/nspanel_eu.tft" # URL to Github
  #nextion_update_url: "https://github.com/Blackymas/NSPanel_HA_Blueprint/raw/main/custom_configuration/nspanel_blank.tft" # URL to Github

##### CHANGE ME END #####

packages:
  remote_package:
    url: https://github.com/Blackymas/NSPanel_HA_Blueprint
    ref: beta
    files:
      - nspanel_esphome.yaml # Core package
      # - nspanel_esphome_addon_climate_cool.yaml # activate for local climate (cooling) control
      # - nspanel_esphome_addon_climate_heat.yaml # activate for local climate (heater) control
    refresh: 1s

##### My customization - Start #####
esp32:
  framework:
    type: esp-idf

#external_components:
#  - source: github://pr#5825
#    components: [nextion]
#    refresh: 1s

##### My customization - End #####

This is what I've changed in your original yaml:

I've pointed it to beta with refresh of 1s, so we make sure your compiler will look for the latest code all the time. It's changing fast on the past couple of days.
I've changed to nspanel_blank.tft as that is much smaller and will reduce the chances of failure if the transfer starts. That will get rid of reparse mode in definitive and the following transfers (nspanel_eu.tft) will be easier.
I've completely removed the reference to PR#5016 even if that was commented out. That PR will fail compiling and is not valid anymore.

Install that, and look at the logs when your device starts... it will take a while until Nextion is setup (as it is in reparse mode). During this time, you will see messages like Is setup: False. When Nextion gets connected, you should see something like this (from your log):

[13:31:20][I][nextion:083]: Nextion is connected
[13:31:20][C][nextion:123]: Nextion:
[13:31:20][C][nextion:124]: Device Model: NX4832F035_011C
[13:31:20][C][nextion:125]: Firmware Version: 51
[13:31:20][C][nextion:126]: Serial Number: E0AF340135FFA277
[13:31:20][C][nextion:127]: Flash Size: 16777216
[13:31:20][C][nextion:128]: Wake On Touch: True
[13:31:20][C][nextion:139]: Start Up Page : 8

At that point, upload tft should work. But it can take up to 3 minutes to get there. If after this time you cannot see that Nextion connected or Is setup: True, click Upload TFT anyways, but I think its likely to fail, which will end up with another boot and restart the whole process.

Please try that and let me know your results. Logs are never too much. 😉

i just changed the main to beta (didn't change anything else for a try)

and it works now! i can see the tft uploading (it will take a while but it is uploading!)

seriously edward thank you very much for being active and helpful. i will finish with the panels and i will check the links so that i can buy you a few beers man. (no more than two. you need to be sober if you have to fix these things)

(it will take a while but it is uploading!)

It shouldn't take more than half an hour, so it should be about to end now, otherwise you have something else to look at.

no no it took much less than 30 minutes. i just connected the other one, flashed it and now the TFT is uploading. 1st try! i wanted to cry.

4.20 started. let's see when it finishes

yeap. 10 minutes exactly!

Nice. I'm closing this one just to make my life a bit easier, but feel free to reopen if you still having issues.

Blackymas / NSPanel_HA_Blueprint

Installation Issues and New Guide #1368