cloudback / issue-tracker

User-reported issues for Cloudback
https://cloudback.it/
9 stars 8 forks source link

Download for S3 Glacier Archive is not functional. #29

Closed Vetavize-Robert closed 1 month ago

Vetavize-Robert commented 3 months ago

Describe the bug I get an error if I try and download an archive from S3 Glacier.

To Reproduce Steps to reproduce the behavior:

  1. Go to the dashboard https://app.cloudback.it/
  2. Click the download icon for a S3 Glacier archive
  3. See error

Expected behavior Archive downloads

Screenshots

image

image

Desktop (please complete the following information):

arxange1 commented 3 months ago

Dear @Vetavize-Robert,

We are currently experiencing a partial service downgrade due to recent infrastructure changes. To resolve this issue, we will be shutting down the entire site today. We will notify you as soon as the issue is resolved.

Additionally, you can check the service health reports on the status page:

arxange1 commented 3 months ago

Dear @Vetavize-Robert, the maintenance is done. the system is fully operational, please check

arxange1 commented 3 months ago

The issue with S3 Glacier Archive downloads is unrelated to maintenance. The problem is due to an incorrect error message being displayed. Please note that retrieving an S3 Glacier Archive can take up to 48 hours, so instant downloads are not supported for this type of storage. We are currently working on a fix to display the correct message.

arxange1 commented 3 months ago

The fix is implemented, please check

Vetavize-Robert commented 2 months ago

It does not look like it is resolved.

image

While I understand given that it is S3 Glacier there is no way to make it an instant download, I do not see a path to get a download through your service. Perhaps an email with a link to the download when available or another notification?

arxange1 commented 2 months ago

Dear @Vetavize-Robert, thank you for understanding the nature of the S3 Glacier and the associated download delays. The current setup doesn’t allow for downloads, and we’re sorry for any inconvenience this causes.

To better assist you, please provide more details on how you plan to use the download feature and why the S3 Glacier dashboard does not meet your needs.

Vetavize-Robert commented 2 months ago

S3 Glacier offers the lowest storage cost for our backups that are still 100% under our control. While we only anticipate downloading in an emergency event, we do need to test that our processes work from time to time. We would like to download the file, recover it to a new GitHub, and check to make sure that everything works. Since the zip file is password protected (your service has the password), I would expect that the download would queue, then when the download is ready we would get a link to the zip file with the appropriate password.

arxange1 commented 2 months ago

To address the password issue, we can add a "Send password via email" option to the "Cannot download backup archive" dialog, as a quick fix. Please let us know if that will work for you.

To address the testing needs, you can restore the backup to a new test repository to ensure everything works as expected, instead of downloading the archive manually.

Also, please note that in case of an emergency, initiating the download process from S3 Glacier can take up to 48 hours, which might not be ideal for urgent situations.

Vetavize-Robert commented 2 months ago

Adding the password via email is a good stopgap solution. The likelihood that an event would occur that would require us to get a backup from your solution is unlikely, so keeping our storage costs as low as possible is our goal. The idea behind S3 was that if your service ever went down, we have a copy under our control that we can restore. This does bring up the point that we should probably store the backup in S3 without a password just in case your service goes down and we cannot get the password.

Here is our backup mentality with your service:

  1. Primary Path: a. Store backup via Cloudback US b. Retrieval is fast and on a site not associated with our primary service
  2. Secondary Path: a. S3 Glacier Backup b. Storage is cheap c. Backup is under our control d. Long term storage for future audits (7 year retention period)

Initially we thought that composite storage would allow us to select Cloudback as the Primary and S3 as the secondary. It does not allow us to do that, so for the time being I am just going to take S3 snapshots manually.

arxange1 commented 2 months ago

Thank you for providing the details. We will allow Cloudback-managed storage to serve as a primary target in a composite storage setup. Consequently, we anticipate the release of two new features:

We will notify you once these features are available.

Vetavize-Robert commented 2 months ago

Thank you! You are doing an excellent job.

arxange1 commented 2 months ago

Dear @Vetavize-Robert, We implemented an option to choose Cloudback storage as a primary target in a composite storage setup. Once configured, it will allow you to download an archive and receive a password email. Please check.

A "send password" link for S3 Glacier will be released within a month. We will let you know as soon as it is released.

Vetavize-Robert commented 2 months ago

The new composite configuration is great and works as intended. Thank you for the enhancement.

arxange1 commented 1 month ago

Dear @Vetavize-Robert, A "send password" link for S3 Glacier is implemented, please check.

Vetavize-Robert commented 1 month ago

Download works, Slight translation issue:

The storage does not support instant downloads. You can request the password to access the archive contents after downloading it from your storage by your own:

Should be: Replace This storage type with the type of storage, for example "S3", or just use "this storage type" if you want to keep it generic. "by your own" should be "on your own".

This storage type does not support instant downloads. You can request the password to access the archive contents after downloading it from your storage on your own:

Thinking about it more you may want something like this for clarity:

This storage type does not support instant downloads. You will need to download the archive from your own storage solution.

For security reasons this backup is password protected. You will receive an email with the password after clicking Send Password button.

[Send Password]

The received email has the minimum necessary information for the filename and password. It may be nice to also have the repo name. If your user has a few repos and requests in a row, the repo name will help untangle things.

arxange1 commented 1 month ago

Dear @Vetavize-Robert,

We have implemented the suggested changes. Please review them. We greatly appreciate your contribution.