Restoring Channels on RTL fails, Error Code 404

silvester235 commented 3 years ago

Mynodebtc v.0.2.27 Lightning v0.12.1 RTL v0.10.1- beta

I uploaded the backup file "Backup-Channel-All.bak" to /mnt/hdd/mynode/rtl_backup/restore

Backup file shows as restore option in the RTL GUI.

However when I choose restore I receive "Error Code 404, Error Message Unknown Error, API URL ./api/lnd/channels/backup/restore/hannel:All"

Screenshots attached.

Do you have any idea how to restore the channels?

ShahanaFarooqui commented 3 years ago

@silvester235, Please note that the file name to restore all channels backup should be channel-all.bak NOT Backup-Channel-All.bak. RTL, by default, creates the backup file as channel-all.bak.

The user should NOT rename any of the backup files created by RTL as the UI distinguishes these backups by their file names. We expect these files to be copied as is, in the 'restore' folder for proper channel restoration.

Also, please remove the trailing slash from the channelBackupPath in your RTL-Config.json. The value should be /mnt/hdd/mynode/rtl_backup in your case. Although, it is not the reason of current failure, it might cause other issues in the future.

silvester235 commented 3 years ago

@ShahanaFarooqui,

thanks for your reply. I removed the trailing slash as advised.

I tried both file names channel-all.bak and later Backup-Channel-All.bak however in both cases the result is an 404 error (in the API URL is in both cases the first letter missing).

I have this problem with Mynodebtc & RTL installation. Currently I am setting up Raspiblitz & RTL and will try again with my channel-all.bak file.

ShahanaFarooqui commented 3 years ago

@silvester235,

If RTL finds a file named channel-all.bak in the restore folder, it shows the Restore All button on top of the table NOT as a row in the table.

The retore page table should only show the individual channel backup files showing their channel point values. In below example, my restore folder contains 3 files, I can either choose to restore a single channel from the table or all channels by clicking Restore All button. However, trying to restore all channels from the first row of the table (hannel:All), will throw 404 Unknown Error.

ShahanaFarooqui commented 3 years ago

Removing the issue from the release milestone. For now, we are considring it as a troubleshooting issue not a bug.

silvester235 commented 3 years ago

@ShahanaFarooqui thanks for the update

on my latest Mynodebtc & RTL installation the "RESTORE ALL" button didn't show even if the channel-all.bak file was stored at restore folder location.

Only the "RESTORE" button under "Actions"

I am currently setting up Raspiblitz and see if it works. Thanks for your clarification.

ShahanaFarooqui commented 3 years ago

I failed to reproduce the issue at my end, where channel-all.bak file exists but the 'Restore All' button does not show up.

Please take your time and let us know if the problem persists.

RhinoAK commented 3 years ago

@silvester235, Please note that the file name to restore all channels backup should be channel-all.bak NOT Backup-Channel-All.bak. RTL, by default, creates the backup file as channel-all.bak.

The user should NOT rename any of the backup files created by RTL as the UI distinguishes these backups by their file names. We expect these files to be copied as is, in the 'restore' folder for proper channel restoration.

Also, please remove the trailing slash from the channelBackupPath in your RTL-Config.json. The value should be /mnt/hdd/mynode/rtl_backup in your case. Although, it is not the reason of current failure, it might cause other issues in the future.

When the backup is created it does in fact give the file name of Backup-Channel-All.bak at which point you're able to download it to your computer for safe keeping. I've been able to reproduce the same issue. I've even gone as far as having to create the restore directory, copying the file from pc to admin user home directory /home/admin via SCP, renamed the file to channel-all.bak, mv over to /mnt/hdd/mynode/rtl_backup/restore/ resulting in the same error. I thought it was an issue with permissions so I then assigned the restore permissions for the bitcoin user. Doing so still resulted in the same issue "Error Code 404"

ShahanaFarooqui commented 3 years ago

@RhinoAK & @silvester235 Thank you for the details. I was finally able to reproduce the issue after realizing that you were copying the downloaded file into restore folder. I kept testing it by copying the file directly from the backup folder. These were 3 main issues which are fixed and merged with PR #650.

1: We were renaming the file from channel-all.bak to Backup-Channel-All.bak while downloading the file. Now, the downloaded file will also be named as channel-all.bak to avoid any future confusion.

2: The file name check was case-sensitive making Channel-All.bak different from channel-all.bak. We have fixed the issue by

Making the file names case in-sensitive AND
Adding both file names (channel-all & backup-channel-all) as valid restorable files.

3: The file generated by 'Backup All' button (channel-all.bak) was utf-8 encoded and was able to restore the channels successfully. However, the file downloaded by 'Download All' button (Backup-Channel-All.bak) was in utf-8 BOM encoding and was failing to restore these channels. The fix will now download the file in utf-8 format only.

Please feel free to test the fix from Release-0.10.2 branch or it will be released soon with release v0.10.2.

silvester235 commented 3 years ago

@ShahanaFarooqui

Thanks for the update.

I have only the file Backup-Channel-All.bak downloaded by 'Download All' button in utf-8 BOM encoding.

"However, the file downloaded by 'Download All' button (Backup-Channel-All.bak) was in utf-8 BOM encoding and was failing to restore these channels."

Does it mean I will not be able to restore? Or ist here a way to change the format from utf-8 BOM to utf8?

ShahanaFarooqui commented 3 years ago

A lot of text editors can change the encoding very easily. For example, notepad++.

I successfully restored from my Backup-Channel-All.bak file after renaming it to ‘channel-all.bak’ and changing the encoding from ‘utf-8 BOM’ to ‘utf-8’ via notepad++. Please note that these edits are required till v0.10.1.

lazyrudi commented 1 year ago

I have a similar problem right now. The backup file created on my old node (name „Pinsel") has the name "channel-all-5.bak". Five, because I had 5 channels. I copied the file to the "Restore" folder of my new node (name „PinselNeu"). I get this shown in the RTL-Backup-Restore, but during the execution the error message 400 is displayed. Where is the error?

ShahanaFarooqui commented 1 year ago

Number 5 is not related to your number of channels. This is the node index taken from your RTL-Config.json file. We create backup files for all LND nodes configured in the application and name them uniquely with their node index.

Also the error message is indicating that this file doesn’t contain all channels backup. It is a single channel backup file.

lazyrudi commented 1 year ago

Thanks for the quick reply.

I am sure that I made a Backup of all 5 Channels using the "Backup All“ function

I don’t have a screenshot of the latest Channel balance but it should look like this:

This is the backup file.

How can I restore the channels?

Regards Rudi

Am 29.10.2022 um 18:14 schrieb ShahanaFarooqui @.***>:

Number 5 is not related to your number of channels. This is the node index taken from your RTL-Config.json file. We create backup files for all LND nodes configured in the application and name them uniquely with their node index.

Also the error message is indicating that this file doesn’t contain all channels backup. It is a single channel backup file.

— Reply to this email directly, view it on GitHub https://github.com/Ride-The-Lightning/RTL/issues/614#issuecomment-1295887496, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2JFQPPWSXGXOIE3RV7H6D3WFVEPDANCNFSM4YLDNVYQ. You are receiving this because you commented.

ShahanaFarooqui commented 1 year ago

Ensure that your backup file location is correct.
Ensure that your backup file is corresponding to the correct node index.
Do NOT rename any file before copying it into the restore folder.
If the restore folder has single channel backup file then RTL will show it in the grid on restore page.
If the folder has multi channel backup file, then it will show it on the top of restore page.
Restore selected or all channels according to your preference.

I am unable to check your backup file but for 5 channels backup, it should looks similar to this:

{
   "single_chan_backups":{
      "chan_backups":[
         {
            "chan_point":{
               "funding_txid_bytes":"Viv...Edg=",
               "output_index":0
            },
            "chan_backup":"qjo...XZE"
         },
         {
            "chan_point":{
               "funding_txid_bytes":"85s...Iz0=",
               "output_index":0
            },
            "chan_backup":"F6r...hSP"
         },
         {
            "chan_point":{
               "funding_txid_bytes":"KpS...evI=",
               "output_index":0
            },
            "chan_backup":"2M7...fxA=="
         },
         {
            "chan_point":{
               "funding_txid_bytes":"Uu/...wQQ=",
               "output_index":1
            },
            "chan_backup":"Rz9...WQ6"
         },
         {
            "chan_point":{
               "funding_txid_bytes":"/dd...3dI=",
               "output_index":0
            },
            "chan_backup":"kDj...JJJ"
         }
      ]
   },
   "multi_chan_backup":{
      "chan_points":[
         {
            "funding_txid_bytes":"Viv...Edg=",
            "output_index":0
         },
         {
            "funding_txid_bytes":"85s...Iz0=",
            "output_index":0
         },
         {
            "funding_txid_bytes":"KpS...evI=",
            "output_index":0
         },
         {
            "funding_txid_bytes":"Uu/...wQQ=",
            "output_index":1
         },
         {
            "funding_txid_bytes":"/dd...3dI=",
            "output_index":0
         }
      ],
      "multi_chan_backup":"E8S...TAV"
   }
}

Could you please confirm your lnd & RTL versions also? I just tested the backup & restore feature with RTL 0.13.1 and LND 0.15.3 and it worked without any issue.

lazyrudi commented 1 year ago

Ok, have renamed the file and started the RESTORE in RTL. Unfortunately it doesn’t work, showing an Error Code 500. What con I do?

Am 29.10.2022 um 19:58 schrieb ShahanaFarooqui @.***>:

Ensure that your backup file location is correct. Ensure that your backup file is corresponding to the correct node index. Do NOT rename any file before copying it into the restore folder. If the restore folder has single channel backup file then RTL will show it in the grid on restore page. If the folder has multi channel backup file, then it will show it on the top of restore page. Restore selected or all channels according to your preference. — Reply to this email directly, view it on GitHub https://github.com/Ride-The-Lightning/RTL/issues/614#issuecomment-1295919199, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2JFQPJNC3HAOBI3P5RA6T3WFVQS7ANCNFSM4YLDNVYQ. You are receiving this because you commented.

ShahanaFarooqui commented 1 year ago

The file should NOT be renamed.
It is not possible to suggest anything else with these little details.

lazyrudi commented 1 year ago

Sorry but I am no IT professional and need some more help.

I thought the node index is the -5 which I removed from the name. OK that was wrong. So I renamed the file name back to channel-all-5.bak. But what ist the node index, where can I find it where can I change it in the backup file?

But what ist the node index, where can I find it and what do I have to do to make it "corresponding to the correct node index"?

Is there any problem because I gave the new node a new name?

The backup file now has the old name is located in the right directory

When I start RTL Backup the Restore File/Channel Point shows all:5 .

When I run the restore function I again get the Error Code 400

What is wrong?

Regards Rudi.

Am 30.10.2022 um 15:20 schrieb ShahanaFarooqui @.***>:

The file should NOT be renamed. It is not possible to suggest anything else with these little details. — Reply to this email directly, view it on GitHub https://github.com/Ride-The-Lightning/RTL/issues/614#issuecomment-1296271722, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2JFQPIXNQNKNWCZ4TAAYQLWFZ74VANCNFSM4YLDNVYQ. You are receiving this because you commented.

lazyrudi commented 1 year ago

Ok I found the node indes Datei (rtl-config.json)

But how is it going to corresponding with the backup file?

Sorry but I am no IT professional and need some more help.

I thought the node index is the -5 which I removed from the name. OK that was wrong. So I renamed the file name back to channel-all-5.bak. But what ist the node index, where can I find it where can I change it in the backup file?

But what ist the node index, where can I find it and what do I have to do to make it "corresponding to the correct node index"?

Is there any problem because I gave the new node a new name?

The backup file now has the old name is located in the right directory

When I start RTL Backup the Restore File/Channel Point shows all:5 .

When I run the restore function I again get the Error Code 400

What is wrong?

Regards Rudi.

Am 30.10.2022 um 15:20 schrieb ShahanaFarooqui @. @.>>:

The file should NOT be renamed. It is not possible to suggest anything else with these little details. — Reply to this email directly, view it on GitHub https://github.com/Ride-The-Lightning/RTL/issues/614#issuecomment-1296271722, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2JFQPIXNQNKNWCZ4TAAYQLWFZ74VANCNFSM4YLDNVYQ. You are receiving this because you commented.

saubyk commented 1 year ago

Hi @lazyrudi please respond on the discussion page below: https://github.com/Ride-The-Lightning/RTL/discussions/1125

Ride-The-Lightning / RTL

Restoring Channels on RTL fails, Error Code 404 #614