Cannot or extremley difficult to import flows from local drive.

[wrmacj]

Problem/Motivation

NodeRed goes non responsive and or has great difficulty importing or exporting flows if there are more than a few “full” flow backups in \addon_configs\a0d7b954_nodered\lib\flows

I am tying it restore flows from backup(Import or export).
I try to do an all flows export daily when I am doing a lot of coding. When I select import or export and then select the local drive, most of the time nothing appears to be happening except CPU usage goes up. I am not able to select a file because none are displayed. This problem is not restricted to import. I cannot export either. If I select export, it never displays the files in the directory.

If I check the CPU usage during these times it is around 35%-45%. Most times I have to restart NRed? It does not help but at least I can get NRed working again. NRed goes non responsive. If I open another browser to NRed it just spins but does not open or error.

My workaround - If I want to export all flows for the day I just copy \addon_configs\a0d7b954_nodered\flows.json and rename it into \addon_configs\a0d7b954_nodered\lib\flows directory with the date. Importing a single flow from an all flows export does not work and I have opened another ticket for that.

(Why the issue was filed) Extremely difficult to export or import flows(Backup or restore), IF there are more than a few full flow exports already in the local folder, it is almost impossible.

Expected behaviour

(What you expected to happen) I would to be able to do a daily export of all flows to a named file IE. 24.2.4_allFlows.json When I select Import/Export and then select the Local drive I expect it to show me the files in that folder and be able to select one to restore or review the files I have exported so I can pick a filename to export to.

Actual behaviour

If there are files currently in \addon_configs\a0d7b954_nodered\lib\flows NRed CPU goes to 35%-40% and pretty much goes completely non-responsive for at least a few minutes.
If I open a new IE tab to NRed it just spins and may ...open in 3-5min but may not at all. Most of the time it will not even show the squiggly bars that are supposed to imply it is searching the local folder. If I remove the previous flow exports from addon_configs\a0d7b954_nodered\lib\flows it will allow me to do a new backup. (What actually happened)

Steps to reproduce

The easiest way is to just copy your \addon_configs\a0d7b954_nodered\flows.json a couple of times into \addon_configs\a0d7b954_nodered\lib\flows with different names ie. Do 9 files of about 500k each. 1.json 2.json 3.json 4,json An example of my flows directory follows so you get an idea of size. No do ctrl I or E to import or export. This should demonstrate the problem. Then remove all files but one and it should respond as you would expect.

This is what I have in my lb directory 12/12/2023 12:25 PM 114,326 23.12.12_All_Flows.json 12/12/2023 12:25 PM 114,326 23.1212_All_Flows.json 01/24/2024 02:15 PM 612,294 24.1.24_AllFlows.json 01/26/2024 09:21 PM 609,233 24.1.26_Stuff might be missing_AllFlows.json 01/27/2024 02:24 PM 611,401 24.1.27a_AllFlows.json 01/27/2024 11:20 PM 611,386 24.1.27b_AllFlows.json 01/28/2024 05:26 AM 635,395 24.1.28a_AllFlows.json 01/30/2024 05:51 AM 674,167 24.1.30a_AllFlows.json 01/31/2024 05:35 AM 686,014 24.1.30b_AllFlows.json 12/03/2023 02:08 PM 6,397 Example Flow.json 12/12/2023 12:06 PM 5,541 Purge and Repack HA database.json 12/12/2023 10:07 PM 3,313 Toggle Desk Light Charger.json 12/12/2023 11:59 AM 6,761 Water_Tempurature.json 12/12/2023 11:55 AM 6,927 Water_Tempurature_old.json 12/06/2023 05:06 PM 23,136 Zones_notify.json

Logs and SW versions

Core 2024.1.6 Supervisor 2023.12.1 Operating System 11.4 Frontend 20240104.0 Node-Red 17.0.4 I tried to do an import at around 01:51:00 The screen finally showed the files around 01:54 5 Feb 01:50:52 - [info] [server:BMAC] Connection closed to http://supervisor/core 5 Feb 01:50:52 - [info] [server:BMAC] Connecting to http://supervisor/core 5 Feb 01:50:52 - [info] [server:BMAC via http] Connection closed to http://192.168.10.11:8123 5 Feb 01:50:52 - [info] [server:BMAC via http] Connecting to http://192.168.10.11:8123 5 Feb 01:50:52 - [info] [server:BMAC] Connected to http://supervisor/core 5 Feb 01:50:52 - [info] [server:BMAC via http] Connected to http://192.168.10.11:8123 5 Feb 01:50:52 - [debug] [server:BMAC] States Loaded 5 Feb 01:50:52 - [debug] [server:BMAC via http] States Loaded 5 Feb 01:50:54 - [debug] [server:BMAC] Services Loaded 5 Feb 01:50:54 - [debug] [server:BMAC via http] Services Loaded 5 Feb 01:50:54 - [debug] [server:BMAC] Integration: loaded 5 Feb 01:50:54 - [debug] [ha-entity-config:Recirculation] Registering switch node with Home Assistant 5 Feb 01:50:54 - [debug] [ha-entity-config:Recirculation Stop] Registering button node with Home Assistant 5 Feb 01:50:54 - [debug] [server:BMAC] HA State: running 5 Feb 01:50:54 - [debug] [server:BMAC via http] Integration: loaded 5 Feb 01:50:54 - [debug] [server:BMAC via http] HA State: running 5 Feb 01:55:22 - [info] [server:BMAC] Connection closed to http://supervisor/core 5 Feb 01:55:22 - [info] [server:BMAC] Connecting to http://supervisor/core 5 Feb 01:55:22 - [info] [server:BMAC via http] Connection closed to http://192.168.10.11:8123 5 Feb 01:55:22 - [info] [server:BMAC via http] Connecting to http://192.168.10.11:8123 5 Feb 01:55:22 - [info] [server:BMAC] Connected to http://supervisor/core 5 Feb 01:55:22 - [info] [server:BMAC via http] Connected to http://192.168.10.11:8123 5 Feb 01:55:22 - [debug] [server:BMAC] States Loaded 5 Feb 01:55:22 - [debug] [server:BMAC via http] States Loaded 5 Feb 01:55:23 - [debug] [server:BMAC via http] Services Loaded 5 Feb 01:55:23 - [debug] [server:BMAC] Services Loaded 5 Feb 01:55:23 - [debug] [server:BMAC via http] Integration: loaded 5 Feb 01:55:23 - [debug] [server:BMAC via http] HA State: running 5 Feb 01:55:23 - [debug] [server:BMAC] Integration: loaded 5 Feb 01:55:23 - [debug] [ha-entity-config:Recirculation] Registering switch node with Home Assistant 5 Feb 01:55:23 - [debug] [ha-entity-config:Recirculation Stop] Registering button node with Home Assistant 5 Feb 01:55:23 - [debug] [server:BMAC] HA State: running

And example of what I have in my addon_configs\a0d7b954_nodered\lib directory

This is what is in the NRed log when I am trying to import/export If I have just two file sin the dir It takes about 2 minutes before the contents of the directory are displayed. When all of these files are in the directory it might come back after 5 min or may not come back at all

(If you have a proposed change, workaround or fix, describe the rationale behind it) Let me know what else I can provide

[wrmacj]

The following are a couple screen shots of cpu activity with one, two three and six files in the flows folder. When there were 6 files in the folder Nred import never finished opening the folder and displaying files after 10 min so it would now be imposible to import any files at all. I selectect export and the same thing, I I am no longer able to import or export any flows unless I restart. The cpu behaviour during export is the same as when I selected import with 6 files. Normal operation cpu and mem activity.

[wrmacj]

Doubling the cpus' and Ram did not help. The system sat over night and still did not produce a file list when import was selected. This is the cup usage with 8 cores and 8G of ram.

[github-actions[bot]]

There hasn't been any activity on this issue recently, so we clean up some of the older and inactive issues. Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thanks!