Closed kylos101 closed 4 days ago
@kylos101 Few questions related to "users must be able to access their most recent backup for a workspace regardless of workspace status":
1. During the
stopping
state, would the system be able to distinguish a backup that was done as a result of it from a previous one?2. From what I understand/recall we store the last 4 backups. Would we be able to provide the WebApp with the links and corresponding timestamps of all of them?
automate deployment of GCP storageClasses as part of cluster creation operation (specify discard mount option)
this is not required for XFS.
installer: allow to specify storageClass in gitpod.yaml
this can be optional for the first iteration
@sagor999 as a heads up, I added a few observability tasks. One of the first ones we'll need (if it doesn't already exist) is the ability to inspect backups and restores now being done with TAR. For example, this way we can measure duration for both.
@sagor999 @jenting are there any more integration tests that need to be added for new code we've written? In other words, I see you've fixed existing tests, but wanted to double check for new test needs. For example, one test I can think of, would be a test that kills a pod, relies on a process to backup the orphaned PVC, and then assert that the PVC is gone (because it was snapshotted).
Currently has this issue affecting PVC epic: Sometimes workspace attempts to start with PVC feature enabled
Question: How would someone who ran out of hours get their data back? (re: https://github.com/gitpod-io/gitpod/pull/14393) Contact support? It'd be better if they could self-serve.
Question: Will this change prevent us to download a single file in the workspace? (Will the "Download..." button in the right-click menu of a file still available?) Sometimes, I need to update my artifact on another server by downloading the artifact from Gitpod server and upload it to my server manually.
Question: Will this change prevent us to download a single file in the workspace? (Will the "Download..." button in the right-click menu of a file still available?)
No, this is about downloading the workspace content backup. You can still download individual files from your running workspace depending on how you connect to it. E.g. with Vs Code, just drag and drop.
Maybe this issue should be part of this epic to not lose my workspace's content on a regular basis: https://github.com/gitpod-io/gitpod/issues/11183
Update: Blocker functional issues, and significantly increased workspace startup times were found on the current technical design. 😞
After internal discussions, given backup success ratio is high and stable following adjacent improvements, and that the implementation of the new design will be considerably faster to do after https://github.com/gitpod-io/gitpod/issues/11416, we have decided to pause this effort until then.
PS: @6uliver I believe the root cause of that issue is different from the context of this one. I will follow-up on that one there. 🙏
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Summary
Better protect user data
Context
Sometimes a workspace, node, or workspace cluster fail and the user data cannot be backed up to cloud storage, resulting in data loss. A related incident for a global outage. A related RFC where we are discussing solutions.
Value
By better handling user data, users will trust that even if the Gitpod service is unavailable, once it is online, they will not lose data.
Acceptance criteria
User data is persisted in such a way that even if there is a workspace, node, or cluster failure, the data is accessible to be backed up at a later time.
Tasks
Ops:
(specifydiscard
mount option)workspace-clusters
Design:
Product changes:
Tests:
Bug
Should solve:
Day 2: