Open slingshotvfx opened 4 months ago
Thanks so much for reporting! I am also very sorry to see that files were deleted, and hope that these docs on data recovery with GitButler can help getting them back. Unfortunately, I don't know much about how virtual branches are applied, and can't even theorise about possible causes for the data-loss described here.
But, having recently worked on the watcher
part I can say that the theory of GB causing this itself seems likely.
After all, when files are changed, it will recalculate a lot of things and read files in the process. This happens in parallel to the ongoing changes in the working tree, preventing the removal of files.
In theory, filesystem events should be bundled into 100ms time-slices, but judging from the log this doesn't really happen here as these events are apparently more frequent than that. But even if it would work, then the changing operation could still not take longer than 100ms, or else the recalculation would already kick in. And even if this window would be larger than 100ms, there is always a chance that a change takes longer than that.
The only way to truly prevent 'stepping onto ones feet' is to assure reads are never interleaved with writes, something that cannot currently be done but of course is possible.
@Byron thanks for the link, fortunately I've not lost anything unrecoverable so far but I wasn't aware of that data recovery documentation which I'm sure will come in handy.
I'm not a rust guy but I'm happy to test future fixes or help in anyway I can.
Here's another person on discord with (possibly) the same issue causing problems with changes being locked: https://discord.com/channels/1060193121130000425/1206670506271707156/1230554414511689830
Just wanted to update this thread for newer builds. I find the latest version of gitbutler to be a lot more stable on windows, so really appreciate the effort there! Unfortunately I still run into this failed rmdir
error and loss of data pretty regularly in gitbutler v0.12.9
. Possibly the same or related to #3601
I captured the filesystem activity during these errors with ProcessMonitor. This was clicking the workspace "update" button after merging a simple PR in github.
It seems gitbutler is deleting one particular file, \ingest\handler.py
, then trying to delete the entire ingest
folder, which fails (possibly because it's not empty, as there are lots of other files in there besides handler.py
which was the only one deleted)
I am not familiar with the internals of gitbutler to understand why it needs to delete handler.py or the whole ingest folder. Do you have any ideas about why this is happening?
The update I'm trying to apply doesn't touch the ingest folder or handler.py at all (though handler.py does have uncommitted changes in another virtual branch) and there are no other attempts from gitbutler to delete any other files or folders in the entire repository.
The logs have changed slightly, the error now says failed to checkout index, this should not have happened, we should have already detected this
, if that helps at all.
2024-07-12T00:29:54.581892Z INFO update_base_branch: crates\gitbutler-tauri\src\virtual_branches.rs:152: new project_id=7954371c-c675-4582-b0d3-ec6d6201f378
2024-07-12T00:29:54.595111Z INFO update_base_branch:create_snapshot: crates\gitbutler-oplog\src\oplog.rs:289: new project_id=7954371c-c675-4582-b0d3-ec6d6201f378 self=Project { id: 7954371c-c675-4582-b0d3-ec6d6201f378, title: "slingshot-main", description: None, path: "C:\\Users\\Geoff\\Documents\\slingshot-main", preferred_key: SystemExecutable, ok_with_force_push: true, api: None, gitbutler_data_last_fetch: None, gitbutler_code_push_state: None, project_data_last_fetch: Some(Fetched { timestamp: SystemTime { intervals: 133652176575391519 } }), omit_certificate_check: None, snapshot_lines_threshold: None, ignore_project_semaphore: false }
2024-07-12T00:29:54.668369Z INFO update_base_branch:create_snapshot: crates\gitbutler-oplog\src\oplog.rs:289: close time.busy=73.2ms time.idle=28.1µs project_id=7954371c-c675-4582-b0d3-ec6d6201f378 self=Project { id: 7954371c-c675-4582-b0d3-ec6d6201f378, title: "slingshot-main", description: None, path: "C:\\Users\\Geoff\\Documents\\slingshot-main", preferred_key: SystemExecutable, ok_with_force_push: true, api: None, gitbutler_data_last_fetch: None, gitbutler_code_push_state: None, project_data_last_fetch: Some(Fetched { timestamp: SystemTime { intervals: 133652176575391519 } }), omit_certificate_check: None, snapshot_lines_threshold: None, ignore_project_semaphore: false }
2024-07-12T00:29:54.677097Z INFO update_base_branch:workdir: crates\gitbutler-branch\src\diff.rs:150: new project_id=7954371c-c675-4582-b0d3-ec6d6201f378 commit_oid=50cca8f9b37dad2a53d1796cdf6b1c56066b9a3d
2024-07-12T00:29:54.723870Z INFO update_base_branch:workdir: crates\gitbutler-branch\src\diff.rs:150: close time.busy=46.7ms time.idle=24.0µs project_id=7954371c-c675-4582-b0d3-ec6d6201f378 commit_oid=50cca8f9b37dad2a53d1796cdf6b1c56066b9a3d
2024-07-12T00:29:54.848308Z ERROR update_base_branch: crates\gitbutler-tauri\src\virtual_branches.rs:152: error=Error(failed to checkout index, this should not have happened, we should have already detected this
Caused by:
failed rmdir - 'C:/Users/<redacted>/Documents/<redacted>/ingest/' is locked: Access is denied.
; class=Os (2); code=Locked (-14)) project_id=7954371c-c675-4582-b0d3-ec6d6201f378
2024-07-12T00:29:54.848322Z INFO update_base_branch: crates\gitbutler-tauri\src\virtual_branches.rs:152: close time.busy=266ms time.idle=27.8µs project_id=7954371c-c675-4582-b0d3-ec6d6201f378
I am not familiar with the internals of gitbutler to understand why it needs to delete handler.py or the whole ingest folder. Do you have any ideas about why this is happening?
Thanks for the update!
As far as I know, GitButler only alters the worktree either when discarding individual files as triggered by the user explicitly, or when running a worktree checkout or reset which is handled by git2
. And that's where I am unsure of how it works exactly, and if it's Windows-proven already.
I also think that gitoxide
can help here as it's tested on Windows natively, forcing it to not rely on the flexibility of Unix filesystems.
gitbutler 0.11.3 for windows very often has issues with locked files/access denied errors on my machine. Running windows 11.
When these errors pop up, it permanently deletes the file(s) that were erroring, resulting in a loss of data.
This might be a dupe of https://github.com/gitbutlerapp/gitbutler/issues/3412, but I'm opening a new issue because I have no explorer windows, command prompts, or VS Code instances open, and file locksmith shows no handles/processes with locks on the files that show up in the error logs.
Is it possible gitbutler is locking/blocking it's own access to files somehow?
Here's an example:
deleting a branch after it's merged
In this example,
filesystem.py
was deleted and no longer exists on the disk.There might actually be two issues here. I think the
ERROR set_base_branch: crates\gitbutler-tauri\src\virtual_branches.rs:138: error=Error(failed to checkout tree
call is destructively removing changed files before erroring and not putting them back. But I'm also not sure where theAccess is denied
error is coming from in the first place.I've also experienced this same issue when unapplying branches, updating the workspace, and switching back to gitbutler (after checking out the main branch elsewhere)