After update to 16.0.1 all flows gone

Bloodpack commented 10 months ago

Did update from 15.x to 16.0 and then to 16.0.1 and all my flows are gone. tried to restore backup from 15.x and nodered does not start anymore.

EDIT: Read about the solution here: https://github.com/hassio-addons/addon-node-red/issues/1736#issuecomment-1812437543

kalatos86 commented 10 months ago

Hello,

I was also impacted by this bug. What I did I used the backup and rollback to 15.1.1 version. Unfortunately my /addon_configs/ catalog was empty. Now installed 16.0.2 NodeRed is working but all my flows gone. What I did wrong?

svenf01 commented 10 months ago

Hi I'm sorry. Im updated from prior 16 to 16.0.1 with lost of flows. Now i updated to 16.0.2 and checked the /addon_config/ is emty. I have a samba backup that includes the folder /data/node-red with also the flow.json. Where i have to copy that data?

PITT72 commented 10 months ago

Maybe this could help some of you. I restored version Node-Red 15.1. from HA Backup. Since the /config/node-red folder is empty (because of the previous upgrade to version 16), Node-Red cannot start. I downloaded/copied the last HA Full Backup to my local machine, extracted it, and copied the content from /data/config/node-red back to the /config/node-red folder on my HA VM. Now Node-Red 15.1 is starting and working again. Upgrade to 16.0.2 worked now without losing the flows.

nivek1612 commented 10 months ago

Hi I'm sorry. Im updated from prior 16 to 16.0.1 with lost of flows. Now i updated to 16.0.2 and checked the /addon_config/ is emty. I have a samba backup that includes the folder /data/node-red with also the flow.json. Where i have to copy that data?

This worked for me But I always take a full HA backup before any changes, that's the backup when restored will correct the directory changes which I think is the issue

Restore complete full HA backup taken before any upgrade to 16.0.x (overwrite everything) Install 16.0.2 All works

PITT72 commented 10 months ago

If you don't restore the complete HA backup, make sure the folder \addon_configs\something_nodered is empty, before you try a new upgrade attempt to version 16.0.2.

LsVzqz commented 10 months ago

After a full backup of Home Assistant (with version 15.1.1), node red longer starts. I try starting: Then I start the addon, get the following, and after some time, it goes back to the start dialog.

Anyone else seen this issue?

EDIT: I restarted my VM itself (in my case, prox mox) and it starts up again. I will wait to upgrade to 16.x :) Thanks 3ative from discord.

Gadund commented 10 months ago

@LsVzqz

me too

MarcoL1975 commented 10 months ago

If you don't restore the complete HA backup, make sure the folder \addon_configs\something_nodered is empty, before you try a new upgrade attempt to version 16.0.2.

Restored the 15.1.1 backup, copied the needed files to \config\node-red from HA Backup. Started 15.1.1 which is working. But still the new 16.0.2 is installed under \addon_configs !? How can i fix this?

Jalle19 commented 10 months ago

The cool thing is though after you have upgraded to v16.0.2, it will be.

As of v16, the backup of your flows, will be backuped with the Node-RED add-on backup 🤩

It is the whole reason for these new features, and this migration in the first place.

That explains why all this trouble was worth it! Thanks!

frenck commented 10 months ago

I did that, but still the new 16.0.2 is installed under \addon_configs !? How can i fix this?

@MarcoL1975 That is the new location. That is the goal of this change. The configuration will now be included in add-on backups and no longer be in the Home Assistant config folder. This is also listed on the release notes.

../Frenck

MarcoL1975 commented 10 months ago

I did that, but still the new 16.0.2 is installed under \addon_configs !? How can i fix this?

@MarcoL1975 That is the new location. That is the goal of this change. The configuration will now be included in add-on backups and no longer be in the Home Assistant config folder. This is also listed on the release notes.

../Frenck

Ow duhhh sorry, all fine then now :) Thanks!

aircooledbeemer commented 10 months ago

Many thanks for the fast delivery of the fix!

dpgh947 commented 10 months ago

managed to restore and get working and now upgraded to 16.0.2, all now working. That was a worrying hour!

I am still confused over the /config /homeassistant directory/link changes. The samba change caught me out twice the other day (when it went in and then was reverted), and now I have just had to change some stuff in node red because I was referencing files under /config and now node red doesn't know about /config and I have had to change these references to /homeassistant.

It is confusing because docs (and backup etc.) still talk about config folder and /config, but node red now doesn't know about it.

Is /config going to disappear altogether?

rellerton commented 10 months ago

I struggled getting the partial restore to work as well. Not sure why, I've used it several times before, but it did not roll back the node red version. Before I went for a full restore I did the following:

Stopped the running 16.0.2 version of the add on
Backed up/removed the node red folder in /addon_configs/
Pulled a copy from yesterday from my GitHub repo of the /config/nodered folder back in to the /config folder. Just lucky I had this handy.
Started the add on. It appears to have redone the migration and everything is back to working without issue.

Thanks for the quick fix Frenck!

davecpearce commented 10 months ago

After a full backup of Home Assistant (with version 15.1.1), node red longer starts. I try starting: Then I start the addon, get the following, and after some time, it goes back to the start dialog.

Anyone else seen this issue?

I am having the same issue. I restored from a backup with NR 15.x and NR starts but then crashed, no info in the NR log. I cannot start Node Red either. Since I have backups, I tried deleting the NR addon and re-installing 16.0.2 and I get this error but I am not sure what the issue is:

Failed to install add-on The command '/bin/bash -o pipefail -c apk add --no-cache --virtual .build-dependencies build-base=0.5-r3 linux-headers=6.3-r0 py3-pip=23.1.2-r0 python3-dev=3.11.4-r0 && apk add --no-cache git=2.40.1-r0 icu-data-full=73.1-r1 nginx=1.24.0-r6 nodejs=18.16.0-r1 npm=9.6.6-r0 openssh-client=9.3_p1-r3 patch=2.7.6-r10 && npm install --no-audit --no-fund --no-update-notifier --omit=dev --unsafe-perm && npm rebuild --build-from-source @serialport/bindings-cpp && npm cache clear --force && echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && patch -d /opt/node_modules/node-red-dashboard -p1 -i /tmp/node-red-dashboard-show-dashboard.patch && apk del --no-cache --purge .build-dependencies && rm -fr /etc/nginx /root/.cache /root/.npm /root/.nrpmrc /tmp/*' returned a non-zero code: 64

LsVzqz commented 10 months ago

After a full backup of Home Assistant (with version 15.1.1), node red longer starts. I try starting: Then I start the addon, get the following, and after some time, it goes back to the start dialog. Anyone else seen this issue?

I am having the same issue. I restored from a backup with NR 15.x and NR starts but then crashed, no info in the NR log. I cannot start Node Red either. Since I have backups, I tried deleting the NR addon and re-installing 16.0.2 and I get this error but I am not sure what the issue is:

Failed to install add-on The command '/bin/bash -o pipefail -c apk add --no-cache --virtual .build-dependencies build-base=0.5-r3 linux-headers=6.3-r0 py3-pip=23.1.2-r0 python3-dev=3.11.4-r0 && apk add --no-cache git=2.40.1-r0 icu-data-full=73.1-r1 nginx=1.24.0-r6 nodejs=18.16.0-r1 npm=9.6.6-r0 openssh-client=9.3_p1-r3 patch=2.7.6-r10 && npm install --no-audit --no-fund --no-update-notifier --omit=dev --unsafe-perm && npm rebuild --build-from-source @serialport/bindings-cpp && npm cache clear --force && echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && patch -d /opt/node_modules/node-red-dashboard -p1 -i /tmp/node-red-dashboard-show-dashboard.patch && apk del --no-cache --purge .build-dependencies && rm -fr /etc/nginx /root/.cache /root/.npm /root/.nrpmrc /tmp/*' returned a non-zero code: 64

I didn't uninstall it, but I did just restart my VM to get it to start. Maybe restore again and just restart the VM?

andyc1701 commented 10 months ago

Hello,

I am sorry if this is a dumb question, and I have been up and down this thread a few times now. How do I upgrade to 16.02? In the Addon's section of my HA install, it states v16.01 is only available.

PITT72 commented 10 months ago

@andyc1701 A HA restart should do the trick.

andyc1701 commented 10 months ago

@PITT72 Thanks, I work in IT and should have known better. That did the trick, thank you.

toyota-axel commented 10 months ago

Hi, I also had the majority of avbove mentioned problems, did a full rollback to my nightly VM backup. Will wait until everything has cleared out. (16.0.3 maybe?)

frenck commented 10 months ago

Will wait until everything has cleared out. (16.0.3 maybe?)

It is cleared out with v16.0.2

alexruffell commented 10 months ago

Restored flows.json and packages.json from timeshift backups and all good now. Yay for backups!

I did not want to risk losing anything else I changed since yesterday as I put a lot of work into it so I downloaded my backup from yesterday 1am, and fished out the two files that @stu1811 mentioned. I replaced them in the new add_on folder and rebooted (I had already installed all updates). All is good now. Thanks to all!

clownjoy commented 10 months ago

Hi, I'm struggling little bit with that using VM haos

restored to NR15.0.0 Not sure whether the stop on NR was required, but it was late when I realized that I should.
cleaned the /addon_config/* so the directory was clean
attempt to start NR15.0.0 has failed due to: ```

start node $NODE_OPTIONS node_modules/node-red/red.js --settings /etc/node-red/config.js

Error loading settings file: /etc/node-red/config.js


4. attempt to upgrade seems to be successful and  I could start NR, but without my flows :( 

any hints please  ? thanks

frenck commented 10 months ago

@clownjoy Please read: https://github.com/hassio-addons/addon-node-red/issues/1736#issuecomment-1812437543

ckglobalroaming commented 10 months ago

OPEN COMMENT @frenck Dear Frenck, thanks for all that you do for the community. Sorry that this little adventure tonight has inconvenienced so many people, these things happen. Its 12:30am most of my house is impaired or not working as i have a lot of NR in my setup. but i have a real IT day job to to in not too many hours. What is quite supprising to me is the way you are yelling at us in this thread. It is my subscription to nabu that affords salaries to keep this great project running. When people forget their project roots and their community that can be problematic. I will wait for this little train wreck to crash out tonight and see if there is a clear set of instructions to recover my flows tomorrow night after i have worked hard in my real IT day job. Thanks for the read and hope you have a better day and consider the importance of simple people like myself that pay the homeassistant bills. Appreciate your efforts. Best regards Craig.

Caligo82 commented 10 months ago

OPEN COMMENT Dear Frenck, thanks for all that you do for the community. Sorry that this little adventure tonight has inconvenienced so many people, these things happen. Its 12:30am most of my house is impaired or not working as i have a lot of NR in my setup. but i have a real IT day job to to in not too many hours. What is quite supprising to me is the way you are yelling at us in this thread. It is my subscription to nabu that affords salaries to keep this great project running. When people forget their project roots and their community that can be problematic. I will wait for this little train wreck to crash out tonight and see if there is a clear set of instructions to recover my flows tomorrow night after i have worked hard in my real IT day job. Thanks for the read and hope you have a better day and consider the importance of simple people like myself that pay the homeassistant bills. Appreciate your efforts. Best regards Craig.

Sorry, but I don't feel like frenck was yelling at anyone in this thread. Quite the opposite in my humble opinion. I feel like this bug was handled considerably better than the problem with the prematurely updated node-red websocket recently. It was also fixed in a reasonable amount of time. If you have your backups set up, no problem really. I've been using Homeassistant for a long time and stuff like this happens from time to time. Nothing that wouldn't happen in a corporate environment as well to be fair. And since this is all free software (apart from that nabu casa bit) I feel like this criticism is not justified.

clownjoy commented 10 months ago

@clownjoy Please read: #1736 (comment)

dear frenck, that's the guide I'm referring to. Did I missed something please ?

frenck commented 10 months ago

What is quite supprising to me is the way you are yelling

Are you ok? I'm not yelling, I've put some larger header in place at some point, to prevent flooding things or things to be missed. I understand the panic (and handled it as a high critical issue), but stopping a flood of comments coming in, is helping keeping things visible.

It is my subscription to nabu that affords salaries to keep this great project running.

Maybe you missed it, but this add-on (and the community add-ons project) is not by Nabu, and I just do that extra in my free spare time.

I will wait for this little train wreck to crash out tonight and see if there is a clear set of instructions to recover my flows tomorrow night after i have worked hard in my real IT day job

There is, but it seems like you didn't take time to read this thread, but instead think this comment helps in a way.

Thanks for the read and hope you have a better day and consider the importance of simple people like myself that pay the homeassistant bills.

Wow. Maybe you should actually read and think really good about what you wrote all there. I dare you to find anyone else addressing things like this so quickly.

Please... please. Uninstall this add-on and don't use it again. I don't need your entitled attitude anywhere near any of my personal projects. It seems the best choice for you anyways.

I'm happy to see many people do understand what was happening and do support, which is definitely what I needed today ❤️

../Frenck

frenck commented 10 months ago

@clownjoy Please read: #1736 (comment)

dear frenck, that's the guide I'm referring to. Did I missed something please ?

It sounds like you didn't restore the add-on AND Home Assistant config backups (both are needed).

dersch81 commented 10 months ago

Fixed and works perfectly after Backup of HA and then going from NodeRed Addon 15.x to 16.0.2

Shponzo commented 10 months ago

Fix worked here also! Good reminder of testing backups from time to time. :-) Thank you Frenck for the quick fix!

andrewrogers101 commented 10 months ago

Another option is to extract flows.json from backup using 7zip or similar and import flows into NR 16.02. Open flows.json in notepad and copy into clipboard in NR import feature.

IliasEagle commented 10 months ago

OPEN COMMENT @frenck Dear Frenck, thanks for all that you do for the community. Sorry that this little adventure tonight has inconvenienced so many people, these things happen. Its 12:30am most of my house is impaired or not working as i have a lot of NR in my setup. but i have a real IT day job to to in not too many hours. What is quite supprising to me is the way you are yelling at us in this thread. It is my subscription to nabu that affords salaries to keep this great project running. When people forget their project roots and their community that can be problematic. I will wait for this little train wreck to crash out tonight and see if there is a clear set of instructions to recover my flows tomorrow night after i have worked hard in my real IT day job. Thanks for the read and hope you have a better day and consider the importance of simple people like myself that pay the homeassistant bills. Appreciate your efforts. Best regards Craig.

Dear @ckglobalroaming, I share your frustration as I too got in the unpleasant position of losing most of my "must have" automations. Please, see Frenk's position. He is in a position where a single mistake can upset a lot of people. As he already shared, he is very well aware of the impact this bug had and I cannot begin to think how frustrated he feels about it. Let's not forget that this is an open source project that most people contribute totally free of charge. Those of us who pay for the Nabu-Case subscription actually receive a very good service for remote access. The fact that Paulus uses those payments to maintain a core team of permanent contributors is no more than a gift back to us. So Frenk or any other developer owes us nothing. @frenck try to not take things personally (I am sure that's not easy). Your work among many others is a real gem for the rest of us.

p.s. I totally forgot we are talking about an add-on. That is a totally selfless contribution to the community. Respect @frenck!!!

IliasEagle commented 10 months ago

Another option is to extract flows.json from backup using 7zip or similar and import flows into NR 16.02. Open flows.json in notepad and copy into clipboard in NR import feature.

Yep. I did just that. The only other thing was that some of the pallette extensions had to be reinstalled but that took no more than a couple of minutes.

dm7500 commented 10 months ago

OPEN COMMENT @frenck Dear Frenck, thanks for all that you do for the community. Sorry that this little adventure tonight has inconvenienced so many people, these things happen. Its 12:30am most of my house is impaired or not working as i have a lot of NR in my setup. but i have a real IT day job to to in not too many hours. What is quite supprising to me is the way you are yelling at us in this thread. It is my subscription to nabu that affords salaries to keep this great project running. When people forget their project roots and their community that can be problematic. I will wait for this little train wreck to crash out tonight and see if there is a clear set of instructions to recover my flows tomorrow night after i have worked hard in my real IT day job. Thanks for the read and hope you have a better day and consider the importance of simple people like myself that pay the homeassistant bills. Appreciate your efforts. Best regards Craig.

If you work in IT, then you'll know the stress that a botched deployment can put on a person. It happens, it sucks, but you fix and move on. Frenck did an awesome job fixing the issue, documenting it, and trying to get others sorted as well.

At the end of the day, what he does for this community is amazing. Yes, he works for Nabu, but the add-ons he maintains is on his own free time. Honestly, I see less instability with the Home Assistant/add-on stack then I do with most enterprise level applications that I manage on a daily basis. Not a bad deal for free software.

Please, don't bring a holier-then-though attitude here. It's a simple botched deployment, that is easily fixed with little more then a good backup (you got those right?) and some patience.

jermanoid commented 10 months ago

Whelp. my last full backup was several months ago. Luckily I use node-red project mode and keep my flows in git. So I still have a backup of sorts. Still lost a couple changes but nothing i can't fix easily

mgroen63 commented 10 months ago

Okay, restored the last full backup from 5 days ago and have the old version now installed. In Home Assistant Node-Red is showing as not running and cannot be started. However when I open Node-Red in the browser I can see my flows and everything is working. Hows do I fix this?

frenck commented 10 months ago

@mgroen63 Seems like you are running into an Home Assistant Supervisor bug, where the statuses of containers get out of sync; rebooting your machine will solve that.

raidenlal commented 10 months ago

Took last night's backup, restored "Home Assistant" and "Node-RED", installed pending Node-RED update (16.0.2) and everything is back to normal. For some reason I didn't need to empty the folder.

Thanks for the quick fix.

steuerzentrale commented 10 months ago

@mgroen63 Seems like you are running into an Home Assistant Supervisor bug, where the statuses of containers get out of sync; rebooting your machine will solve that.

can confirm that. thanks a LOT! this did the trick for me, I spent hours of restoring backups forth and back. here is what worked for me: restored from a complete backup where NR was on 15.1.1, then rebooted the machine.

cdk222 commented 10 months ago

Hi Frenck

Need some help if i could. I had a copy of my flows that i saved on the weekend. So I did the following: Restore a backup (node-red v15 only) Clear out the Node-RED folder in /addon_configs/ Updated to 16.0.2 Imported flows Updated the pallet Initially the flows loaded fine how ever when I manually triggered an automations I received a 502 error and Node Red crashes and wont restart. Attempt to restart shows this in my logs

mgroen63 commented 10 months ago

That did the trick! Thanks! I was going around in circles thinking restarting Home Assistant was enough. The Reboot was the solution to get it working again!

dm7500 commented 10 months ago

I restored to last nights backup, and NR is running again at 15.1.1. I don't see a /addons_config folder in my Samba share. Am I missing something?

frenck commented 10 months ago

@dm7500 Make sure you updated to the latest Samba add-on.

dm7500 commented 10 months ago

@dm7500 Make sure you updated to the latest Samba add-on.

That did it, thanks! Looks like my /addon_configs directory is clear. Upgrading NR to the latest version now.

acmeGR commented 10 months ago

Hello. I noticed the issue with the flows and then I tried to restore NR to previous versions. I still didn't have the flows and so I checked this issue here. I tried to make a restore of NR and HA and it is stuck at "Partial backup 2023-11-12 05:00:00"

From Samba I have access to below folders. Any syggestions? Should I unplug my NUC and put it back to power? How much time should it take to make the restore?

supremelucifer commented 10 months ago

Download a full backup and in the rar fle go to: .\homeassitant.tar.gz\data\node-red and download flows.json. Import it back in Node Red and everything is working again. Tested it without all the hassle

derwolff commented 10 months ago

Ok after a few hectic minutes I am able to restart NR by download flow.json from the old configuration folder and import it. But my Alexa automation doesn't work anymore (node-red-contrib-alexa-cakebaked) and a new autentification was not possible. You must also copy the alexa-cookie.txt file into the new configuration folder and change the path in the Node.

craigmdennis commented 10 months ago

I managed to restore my flows from a core backup. It's events like this that force people to implement better backup hygiene (myself included). Thanks for the fix so quickly @frenck

I managed to restore my flows in the following way:

restore node-red partial backup for v15
downloaded latest core backup and extracted Home Assistant Core/homeassistant/data/node-red/
copied everything to config/node-red/
Nothing showed in node-red when it started
updated to 16.0.2
managed to start node-red but errors regarding missing node types (and unable to install them due to errors)
exported all flows from node-red itself
uninstalled node-red
deleted everything I could find
rebooted home assistant
reinstalled node-red
imported flows from file
installed missing nodes
had some issues with lat and long and the Home Assistant Zone (warnings when I tried to publish flows) but published anyway and the errors went away.

There is probably a faster way now more people have shared.

NOW RUNNING A FULL BACKUP :)

MarcusTseng commented 10 months ago

I got another condition here, which is that I could not find the folder /Home Assistant Core/homeassistant/data/node-red/ neither the flow.json file in anywhere of my backup files from august till this morning. Is my node-red flows doomed?

hassio-addons / addon-node-red

After update to 16.0.1 all flows gone #1736