Add optional simple backup to autosaves to address additional failure modes and data protection

[CAM-Gerlach]

Problem Description

While the changes outlined in #8237 should hopefully address the problem of Spyder blanking save files, there are other threats to users' data aside from Spyder's own writes to the file and it crashing and loosing data, such as other applications changing the file, version control system bugs or accidents, conflicting writes, hardware failure, and (most importantly) user error, wishing to go back to an older version not (yet) saved in version control.

While the current autosave system does provide some fairly limited protection against these threats, there are wide swaths of cases that aren't covered that a simple versioned autosave backup system would address. Furthermore, if users save more frequently than the autosave interval (as is often a good habit to avoid the very data loss autosave is intended to prevent), haven't made changes to the file, or have only made a change within the autosave interval, not even one autosave would exist to recover from.

Proposal

Therefore, I propose adding a basic backup system that users could opt into in the preferences that would be a simple extension of the autosave system, and would include the following changes:

• Change the default save interval to 120 seconds, since every 60 seconds is pretty high as a default (LibreOffice, MSOffice etc. default to every 5 minutes) and generates a lot of extra I/O. especially for large files, and would also generate a lot of backup files
• Append a timestamp to each autosave file name when autosaving instead of directly replacing the old one, as well as the name of the directory in which the file is found (to make it easier to identify manually in the file browser and less likely to produce a sequentially numbered duplicate) and save each in its own directory inside the backup dir (this would occur regardless of the user-facing "Backup" preferences option, to keep the implementation cleaner and also makes it a little easier for the user).
• Offer a checkbox option in the preferences, off by default and a sub-option of the autosave checkbox, Preserve backups of the last __<N_backups>__ autosaves for each file.
• If said option is enabled, don't delete the previous autosave file after saving the new one (manually or autosave), but rather after successfully saving the new autosave, delete the N oldest file(s) in the directory where N = (number_of_files_in_dir + 1) - N_backups and N > 0.
• Also, if backup is enabled, make the autosave still save a copy after autosave_interval minutes have passed since the last autosave, even if no unsaved changes currently exist to the file, so long as the file has unsaved changes relative to the autosave (which probably can be conveniently, if not perfectly approximated by autosaving if either A. the file has unsaved changes, or B. the file was last saved more recently than autosave_interval seconds ago, which can be checked from either the original file modification timestamp, or stored somewhere when a manual save is triggered).
• Include the checkbox option Clear all autosaves older than ____ days on Spyder exit with a Clear Now button next to it; 0 would be the default, which would clear all autosaves (the current behavior).
• Include an option in the File menu for Open backup directory, which would open the backup directory in the system file manager for that particular file, and the user could handle things from there (as most other editors do).
• Implement a "Revert to last autosave" command alongside the current "Revert" menu option, that just loads the file from the most recent autosave instead of the most recent manual save

[bayesfactor]

Is there an autosave currently available? If so, where does it write the temporary files? (I need to recover a wiped file!)

[CAM-Gerlach]

@bayesfactor There is, implemented in #7660, as well as other improvements to make writing files more robust and avoid the chance of the happening in the first place. Unfortunately, all of that its only present in Spyder 4 (our new, beta version), not Spyder 3.3.3, so there's not much we can do for you unfortunately. Really sorry about this. Perhaps you have a backup somewhere, or you have file history (Windows) or Time Machine (Mac) turned on?

[adriesse]

What is the current behavior of the Autosave feature? Or where is it documented?

[CAM-Gerlach]

Autosave currently just saves your open files with unsaved changes to the Spyder directory every 60 seconds, deleting old autosaves when either a new autosave occurs or after the original file is successfully saved. If Spyder crashes, hangs, closes unexpectadly or your file gets wiped, the backups in the Spyder directory are retained, and on next Spyder startup Spyder will offer to restore the files for you. A full description of this behavior is given in #7660 , the original PR by @jitseniesen that implemented it, with a number of followup PRs further enhancing it. We are currently in the process of completely updating our user documentation for Spyder 4, where we will also discuss it there. Does that answer your questions? Thanks!

[adriesse]

Thank you, this is very helpful. I had turned the feature off because I thought is was overwriting my original files. Now I will turn it back on.

A suggestion I would make is to save the back-up files in the .spyproject folder, if it exists.

[ccordoba12]

A suggestion I would make is to save the back-up files in the .spyproject folder, if it exists.

This is a very good suggestion! @jitseniesen, what do you think about it?

[jitseniesen]

@adriesse I am afraid I don't see the advantage of saving the back-up files in the .spyproject folder. Why are you suggesting this?

[adriesse]

I think this would be convenient if manual intervention is desired or required (bullet point 2 in the proposal). In my case the project files are on a server, and I tend to prefer having the backup files created there also. Anyway, it was just a minor suggestion.

[CAM-Gerlach]

I'm not sure I have a strong opinion either way, but for what its worth other editors, like Notepad++ (which I based their feature proposal on), save their backups in a hidden subdir in the same directory as the original

[jitseniesen]

The main advantage of putting the autosave files in a fixed directory (as is currently done) is that Spyder can find them when it starts up again after a crash. If the autosave files are put in either the project directory or in a subdirectory of the directory in which the original file is stored, then there needs to be a mechanism for Spyder to find the autosave files (i.e., we need to store some data in a fixed location), which introduces additional complexity and failure modes. Alternatively, we can depend on the user to restore the autosave files manually. I still think it's best to keep the autosave files where they are now.

Currently, the autosave files are transient, in that they are deleted when Spyder exits cleanly. If Spyder finds any autosave files on startup, then they should almost always be restored, so this is fairly automatic. The backup files proposed here have quite a different feel to me, in fact I'd say they are an orthogonal feature to the autosave files. I think it makes sense to store the backup files close to the original files.

[domuhe]

Is this simple auto-backup feature going to be implemented any time soon? I really miss the option to quickly go back one version.

[joooeey]

I just had a .py file open in two Spyder tabs. When switching to the other tab it claimed that the file was changed on disk (nobody changed anything [unless I was hacked]) and asked if I wanted to load it. I made the mistake of clicking okay. Now my changes were gone. So that's another reason why solving this issue would be a great idea.

I couldn't reproduce this behavior with a test file.