hassio-addons / addon-node-red

Node-RED - Home Assistant Community Add-ons
https://addons.community
MIT License
525 stars 120 forks source link

Very slow communication between HA and Node Red #1923

Open lucode opened 1 month ago

lucode commented 1 month ago

Problem/Motivation

I have no idea why the communication between HA and Node Red is suddenly so slow, that it takes some times more then 3 seconds to execute a service call.

Expected behavior

Normally this should happen in milliseconds.

Actual behavior

The service call stays for seconds in sending mode: image The App consumes a lot of CPU power 40-55% I switched to warn mode in the logs and see that that are many missing cache entries for entities. I guess that might be the problem, but I don't now how to clear the cache. Restarting does not help in my case. https://community.home-assistant.io/t/refresh-node-red-entities-cache/218466

Steps to reproduce

Unfortunately I have no idea how to reproduce it.

Proposed changes

Sometimes it helps a bit for a certain period to restart Node-Red.

Logs

-----------------------------------------------------------
 Add-on: Node-RED
 Flow-based programming for the Internet of Things
-----------------------------------------------------------
 Add-on version: 18.0.4
 You are running the latest version of this add-on.
 System: Home Assistant OS 12.4  (amd64 / qemux86-64)
 Home Assistant Core: 2024.7.4
 Home Assistant Supervisor: 2024.08.0
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-nginx: starting
s6-rc: info: service init-customizations: starting
s6-rc: info: service init-customizations successfully started
s6-rc: info: service init-nodered: starting
s6-rc: info: service init-nginx successfully started

up to date, audited 85 packages in 4s

19 packages are looking for funding
  run `npm fund` for details

5 high severity vulnerabilities

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
npm notice
npm notice New minor version of npm available! 10.7.0 -> 10.8.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2
npm notice To update run: npm install -g npm@10.8.2
npm notice
s6-rc: info: service init-nodered successfully started
s6-rc: info: service nodered: starting
s6-rc: info: service nodered successfully started
s6-rc: info: service nginx: starting
s6-rc: info: service nginx successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
[15:20:15] INFO: Starting Node-RED...

> start
> node $NODE_OPTIONS node_modules/node-red/red.js --settings /etc/node-red/config.js
....
for entityId: input_boolean.light_simulation_mode
10 Aug 15:05:09 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:06:09 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:07:09 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:08:10 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:09:10 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:10:11 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:11:11 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:11:38 - [error] [server-state-changed:a4a128a4fac4307a] ConfigError: Entity ID is required
10 Aug 15:11:38 - [error] [server-state-changed:91432a0e5b7708f4] ConfigError: Entity ID is required
10 Aug 15:11:38 - [error] [server-state-changed:1f047f57ee7e0a9e] ConfigError: Entity ID is required
10 Aug 15:12:11 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:13:11 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:14:11 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:15:11 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:15:21 - [error] [server-state-changed:a4a128a4fac4307a] ConfigError: Entity ID is required
10 Aug 15:15:21 - [error] [server-state-changed:91432a0e5b7708f4] ConfigError: Entity ID is required
10 Aug 15:15:21 - [error] [server-state-changed:1f047f57ee7e0a9e] ConfigError: Entity ID is required
10 Aug 15:16:11 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
10 Aug 15:17:12 - [error] [api-current-state:Light Simulation mode] InputError: Entity could not be found in cache for entityId: input_boolean.light_simulation_mode
Richy22B commented 1 month ago

Node Red has worked perfectly for years but I'm getting a similar issue which is preventing flows working. Like you said, restarting Node Red allows it to work for a while but then mine crashes again.

My Log...


Add-on: Node-RED Flow-based programming for the Internet of Things

Add-on version: 18.0.4 You are running the latest version of this add-on. System: Home Assistant OS 12.4 (amd64 / generic-x86-64) Home Assistant Core: 2024.8.1 Home Assistant Supervisor: 2024.08.0

Please, share the above information when looking for help or support in, e.g., GitHub, forums or the Discord chat.

s6-rc: info: service base-addon-banner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service base-addon-log-level: starting s6-rc: info: service fix-attrs successfully started Log level is set to NOTICE s6-rc: info: service base-addon-log-level successfully started s6-rc: info: service legacy-cont-init: starting s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service init-nginx: starting s6-rc: info: service init-customizations: starting s6-rc: info: service init-customizations successfully started s6-rc: info: service init-nodered: starting s6-rc: info: service init-nginx successfully started

up to date, audited 358 packages in 2s

80 packages are looking for funding run npm fund for details

15 high severity vulnerabilities

To address issues that do not require attention, run: npm audit fix

To address all issues possible (including breaking changes), run: npm audit fix --force

Some issues need review, and may require choosing a different dependency.

Run npm audit for details. npm notice npm notice New minor version of npm available! 10.7.0 -> 10.8.2 npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2 npm notice To update run: npm install -g npm@10.8.2 npm notice s6-rc: info: service init-nodered successfully started s6-rc: info: service nodered: starting s6-rc: info: service nodered successfully started s6-rc: info: service nginx: starting s6-rc: info: service nginx successfully started s6-rc: info: service legacy-services: starting s6-rc: info: service legacy-services successfully started

start node $NODE_OPTIONS node_modules/node-red/red.js --settings /etc/node-red/config.js

11 Aug 18:58:13 - [info]

Welcome to Node-RED

11 Aug 18:58:13 - [info] Node-RED version: v4.0.2 11 Aug 18:58:13 - [info] Node.js version: v18.20.3 11 Aug 18:58:13 - [info] Linux 6.6.33-haos x64 LE 11 Aug 18:58:13 - [info] Loading palette nodes 11 Aug 18:58:13 - [info] Node-RED Contrib Theme Collection version: v4.0.7 11 Aug 18:58:14 - [info] Dashboard version 3.6.5 started at /endpoint/ui 11 Aug 18:58:14 - [info] Settings file : /etc/node-red/config.js 11 Aug 18:58:14 - [info] Context store : 'default' [module=memory] 11 Aug 18:58:14 - [info] User directory : /config/ 11 Aug 18:58:14 - [warn] Projects disabled : editorTheme.projects.enabled=false 11 Aug 18:58:14 - [info] Flows file : /config/flows.json 11 Aug 18:58:14 - [info] Server now running at http://xxxxxxxxxxxxxxxx/ 11 Aug 18:58:14 - [info] Starting flows 11 Aug 18:58:14 - [info] Started flows 11 Aug 18:58:14 - [error] [api-current-state:Is Sentry Mode On?] InputError: Entity could not be found in cache for entityId: switch.bfg_sentry_mode 11 Aug 18:58:14 - [error] [api-current-state:Open Plan Motion] InputError: Entity could not be found in cache for entityId: binary_sensor.shellymotionsensor_60a423872950_motion 11 Aug 18:58:14 - [error] [api-call-service:Turn Off Someone Was Home] NoConnectionError 11 Aug 18:58:14 - [error] [api-current-state:Logan Bedroom Temp] InputError: Entity could not be found in cache for entityId: climate.logans_thermostat 11 Aug 18:58:14 - [error] [api-current-state:Was Someone At Home Today?] InputError: Entity could not be found in cache for entityId: input_boolean.someone_was_home_today 11 Aug 18:58:14 - [error] [api-current-state:What Is Temp Top Of Buffer?] InputError: Entity could not be found in cache for entityId: sensor.shellyplus2pm_c049ef8662b4_temperature_3 11 Aug 18:58:14 - [error] [api-current-state:Solar Peak Today] InputError: Entity could not be found in cache for entityId: sensor.power_highest_peak_time_today 11 Aug 18:58:14 - [error] [api-current-state:Solar Peak Tomorrow] InputError: Entity could not be found in cache for entityId: sensor.power_highest_peak_time_tomorrow 11 Aug 18:58:14 - [error] [api-current-state:Heat Pump Heating Status] InputError: Entity could not be found in cache for entityId: binary_sensor.stiebel_eltron_isg_is_heating 11 Aug 18:58:14 - [error] [api-current-state:Is Hot Water On?] InputError: Entity could not be found in cache for entityId: binary_sensor.stiebel_eltron_isg_is_heating_boiler 11 Aug 18:58:14 - [error] [api-current-state:Is Heating On?] InputError: Entity could not be found in cache for entityId: binary_sensor.stiebel_eltron_isg_is_heating 11 Aug 18:58:14 - [error] [api-current-state:3pm Temp Lock] InputError: Entity could not be found in cache for entityId: input_boolean.3pm_temp_lock 11 Aug 18:58:14 - [error] [api-current-state:Outside Light Relay] InputError: Entity could not be found in cache for entityId: switch.shelly1_e8db84ab5efc 11 Aug 18:58:14 - [error] [api-current-state:Shelly PV Output] InputError: Entity could not be found in cache for entityId: sensor.shellyem_c45bbe796c9a_channel_1_power 11 Aug 18:58:16 - [error] [api-current-state:Buffer Target Temp] InputError: Entity could not be found in cache for entityId: sensor.stiebel_eltron_isg_target_temperature_buffer 11 Aug 18:58:19 - [info] [server:Home Assistant] Connecting to http://supervisor/core 11 Aug 18:58:19 - [info] [server:Home Assistant] Connected to http://supervisor/core

fspuhler commented 4 weeks ago

I have also problems with the performance. But in my case, the call service ends with an error.

start node $NODE_OPTIONS node_modules/node-red/red.js --settings /etc/node-red/config.js

14 Aug 07:29:59 - [info]

Welcome to Node-RED

14 Aug 07:29:59 - [info] Node-RED version: v4.0.2 14 Aug 07:29:59 - [info] Node.js version: v18.20.3 14 Aug 07:29:59 - [info] Linux 6.6.31-haos-raspi arm64 LE 14 Aug 07:29:59 - [info] Loading palette nodes 14 Aug 07:30:00 - [info] Node-RED Contrib Theme Collection version: v4.0.8 14 Aug 07:30:04 - [info] Dashboard version 3.6.5 started at /endpoint/ui 14 Aug 07:30:05 - [info] Settings file : /etc/node-red/config.js 14 Aug 07:30:05 - [info] Context store : 'default' [module=memory] 14 Aug 07:30:05 - [info] User directory : /config/ 14 Aug 07:30:05 - [warn] Projects disabled : editorTheme.projects.enabled=false 14 Aug 07:30:05 - [info] Flows file : /config/flows.json 14 Aug 07:30:05 - [info] Server now running at http://127.0.0.1:46836/ 14 Aug 07:30:05 - [warn]


Your flow credentials file is encrypted using a system-generated key.

If the system-generated key is lost for any reason, your credentials file will not be recoverable, you will have to delete it and re-enter your credentials.

You should set your own key using the 'credentialSecret' option in your settings file. Node-RED will then re-encrypt your credentials file using your chosen key the next time you deploy a change.

[07:30:05] INFO: Starting NGinx... 14 Aug 07:30:05 - [info] Starting flows 14 Aug 07:30:06 - [info] Started flows 14 Aug 07:30:06 - [info] [mqtt-broker:7ebca5059d842f66] Connected to broker: HA_itself@mqtt://localhost:1883 14 Aug 07:30:10 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 07:30:10 - [info] [server:Home Assistant] Connected to http://supervisor/core 14 Aug 07:35:06 - [info] [server:Home Assistant] Connection closed to http://supervisor/core 14 Aug 07:35:11 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 07:35:11 - [info] [server:Home Assistant] Connected to http://supervisor/core 14 Aug 07:37:22 - [info] Stopping modified flows 14 Aug 07:37:22 - [info] Stopped modified flows 14 Aug 07:37:22 - [info] Updated flows 14 Aug 07:37:22 - [info] Starting modified flows 14 Aug 07:37:22 - [info] Started modified flows 14 Aug 07:45:11 - [info] [server:Home Assistant] Connection closed to http://supervisor/core 14 Aug 07:45:17 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 07:45:17 - [info] [server:Home Assistant] Connected to http://supervisor/core 14 Aug 07:50:46 - [info] [server:Home Assistant] Connection closed to http://supervisor/core 14 Aug 07:50:46 - [error] [api-call-service:Bad Lenny Light Off] Error: Unrecognized error: {"type":"result","success":false,"error":{"code":3,"message":"Connection lost"}} 14 Aug 07:50:51 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 07:50:51 - [info] [server:Home Assistant] Connected to http://supervisor/core 14 Aug 07:56:54 - [info] [server:Home Assistant] Connection closed to http://supervisor/core 14 Aug 07:56:59 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 07:56:59 - [info] [server:Home Assistant] Connected to http://supervisor/core 14 Aug 07:59:58 - [info] [server:Home Assistant] Connection closed to http://supervisor/core 14 Aug 08:00:04 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 08:00:04 - [info] [server:Home Assistant] Connected to http://supervisor/core 14 Aug 08:07:37 - [error] [api-call-service:EG Flur Light Off] Error: Unrecognized error: 3 14 Aug 08:07:37 - [error] [api-call-service:EG Flur Light Off] Error: Unrecognized error: 3 14 Aug 08:07:56 - [info] [server:Home Assistant] Connection closed to http://supervisor/core 14 Aug 08:08:01 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 08:08:01 - [info] [server:Home Assistant] Connected to http://supervisor/core 14 Aug 08:14:29 - [info] [server:Home Assistant] Connection closed to http://supervisor/core 14 Aug 08:14:35 - [info] [server:Home Assistant] Connecting to http://supervisor/core 14 Aug 08:14:35 - [info] [server:Home Assistant] Connected to http://supervisor/core

This is an issue since the upgrade to the newest version. Thanks for taking a look on this.

lucode commented 4 weeks ago

You should set your own key using the 'credentialSecret' option in your settings file. Node-RED will then re-encrypt your credentials file using your chosen key the next time you deploy a change.

I did change my secret to a new one with !secret node-red.key and after restarting yaml files and Node-Red I did edit something and deploy in Node-Red. That works fine, but my conclusion at the moment is that it is all the same. Still high CPU consumption and all the missing entities in cache. As long there is no feasible way to rebuild the cache the only thing that may work is to backup everything and reinstalling the Addon.

fspuhler commented 4 weeks ago

Backuped everthing and reinstalled nodered addon. Now it works fine. Thanks for helping me. My issue is solved

lucode commented 4 weeks ago

@fspuhler Nice it did work for you. Unfortunately in my case it didn't help either to reinstall the addon.

I did the following:

  1. Download the whole node-red folder with SFTP from /addon_configs/xxxx_node-red
  2. Backup all the flows and the config file yaml
  3. Uninstalling the addon
  4. Deleting the node-red folder under addon_configs
  5. Reinstalling the addon
  6. Editing the configuration file (copy-paste from backup)
  7. Starting the addon
  8. Adding the missing nodes to palette (from file package.json in node-red backup files)
  9. Restore (import) the the flows from backup file.

But I still have a high CPU load any missing entities in cache.

Eventually I do looking in the wrong direction and the issue is related to the HA side of the communication between Nod-Red and HA?