Enhanced backup restore dialog

[jonasstein]

This warning should tell the user exactly what the consequences of yes/no are. Some users want to create a backup or investigate the files manually, before answering the question.

A few ideas how to help the user:

• name the backup file literature.bib.sav for example, so that the user can look for it
• will recover overwrite the .bib file?
• will the .bib.sav be deleted, if NO is selected?
• what is the difference between the two files? size/date/number of entries/perhaps suggest the user to run a diff tool like diff or meld
• if the user can not answer this question now, give a chance to cancel.
• allow to ignore the question, if the user wants to start jabref and work on another file, but can not answer the recover question now.

[koppor]

Thank you for the long issue description including proposals. Can you create a pull request? 😇 In other dialogs we had a similar discussion. E.g., when closing a database with unsaved changes. @oscargus

[koppor]

Refs https://github.com/koppor/jabref/issues/149

[notuntoward]

I'll just add that I never used to see this popup but now see it all the time, whether or not I've saved the .bib file before exiting jabref.

[j0hannes]

Me too, almost every time I open JabRef the popup appears.

I'm usually clicking yes assuming that the backup file is a newer version of the bib file. I'm a right?

[chrisrapson]

I've noticed that the backup files (.bib.bak or .bib.sav) are often NOT newer than the .bib file. So I now open up my file manager, navigate to the folder and check the modification times before making my decision.

There are times when it's good to have a backup that's older than the .bib file, for example when I delete an entry by accident and want to recover it. But most of the time I want the most recent changes. So maybe a button that can expand the window with "details" like modification times would be useful. The window expansion could include the answers to jonasstein's other questions. Going further, a diff would be the ultimate way of knowing which file I want to use.

[koppor]

Really nice ideas! Could you offer some implementation help or know someone who could support us? 😇 We could arrange a JabRef hackday where some of our core developers are online for questions. In Germany, we have in Stuttgart a kind of development lab for an onsite hack event.

[chrisrapson]

I've got a bit much on for the next month or so, but if this is still open after that I might have a look at it. (I won't mind if someone jumps in and takes care of it before then!)

Quick related question: what is the reason for having a backup file and an autosave file?

[koppor]

The backup file is a relict from old JabRef times. The bak file was created before saving the database. The reason was to be be able to recover from failed saves. In 2016 we decided to keep this feature. However, none of the users reported broken bibliography files, so the bak feature can be removed when working on this. - We would be really happy if you found time this summer. Most of our developers are also online at https://gitter.im/JabRef/jabref so that we can answer questions there.

[pat-richter]

This issue still persists with JabRef 5.3. I get annoyed/irritated quite often, but still can't detect the issue behind.

A simple solution would be to add some more information (modification date), as already suggested. Instead, I would like to see a third button.

Button 1: Use current file Button 2: Use backup file Button 3: Compare files

The third button should open the dialog which is also shown, when the file has been changed by an external program.

With this solution one could

• easily see whats going on (no more guessing) and then choose the right file
• also keep only some changes, e.g. when the last edit of the library file has lead to some confusion¹

¹ I've had some times, when I copyied and pasted the bibtex code, without changing the bibtex key. This has led to broken bib file. I currently have my bib in a dropbox. But also in a git repo (which allows my to compare to a previous version easily). I would like to dispense on that additional "backup" step.

[ThiloteE]

f7648e82168e180f635b0e783262117d75c87fa5 introduced multiple backup files (10 to be exact), but the problem here is still valid:

• Give user more information about these multiple backup files, so that
• they can choose the correct one to import,
• or dismiss importing altogether.

[koppor]

The current dialog reads as follows:

[yenniejunvu]

Hi! Is this issue still available? If so, can I try it?

[koppor]

Sure, I assigned you @yenniejunvu

[koppor]

Note that this issue also requires thinking of the cases which can happen and the respective UI. In other words: Do a proper Software Engineering requirements engineering.

[gnandini156]

Hi @jonasstein can you please provide me the steps to get a popup for a backup found .

[ThiloteE]

@gnandini156

To trigger the popup:

1. open/create a library with JabRef
2. save said library
3. Change something in the library, e.g. adding a comment or adding a new entry.
4. Wait ~ 20 seconds for a backup to be created
5. Close JabRef and the library WITHOUT saving.
6. Open JabRef and said library again
7. Now the popup should trigger.

[ThiloteE]

See also https://docs.jabref.org/advanced/autosave for where backups are stored

[KotalaRadhika]

Hi @jonasstein @koppor

Could you please elaborate what to be fixed here?

i'm getting below popup

Also all 3 buttons are working fine.

[koppor]

@KotalaRadhika Good to see that you could bring up the dialog. Sadly, you did not show what happens at "Review backup"

Now, let's walk through the original posters steps.

This warning should tell the user exactly what the consequences of yes/no are. Some users want to create a backup or investigate the files manually, before answering the question.

This is a general remark. Fortunately, the reporter refined it.

A few ideas how to help the user:

Heading for the following list. OK, let's read on.

name the backup file literature.bib.sav for example, so that the user can look for it

This is already done. We see it in blue in the dialog.

will recover overwrite the .bib file?

By the button "Review backup", the next step should be clear

will the .bib.sav be deleted, if NO is selected?

It should also be clear with the buttons. Hopefully

what is the difference between the two files? size/date/number of entries/perhaps suggest the user to run a diff tool like diff or meld

This should be in the dialog.

• [ ] A table

Property	Current file	Backup file
Name	bug_reproduce.bib	...07.bak
Size	100kb	0kb
Date	2024-02-22 10:05	2024-02-21 23:49
Number of entries	100	95
Changes	-	removed: 7, added: 2, modified: 3

That information should be put into the dialog. I think, this is possible using the Bibdatabase Diff. -- The numbers for "Number of entries" and "Changes" should be calculated in a Background Task, because for large files, this could take time.

• [ ] Moreover, launching directly an external diff tool would be great.

1. New button "External diff" (that launches the diff tool)
2. Let the tool be configured at "External programs"

No need for place holders. Pass first the backup file and then the .bib file.

The default tool is git diff --no-index (see https://stackoverflow.com/a/17194704/873282)

if the user can not answer this question now, give a chance to cancel.

I think, with "Ignore Backup" this is fulfilled.

allow to ignore the question, if the user wants to start jabref and work on another file, but can not answer the recover question now.

Also done with "Ignore Backup"

---

Hope, this helps

In general, the issue refs This somehow refs https://github.com/JabRef/jabref/issues/10853.

.

[koppor]

Even more refined: https://github.com/JabRef/jabref/issues/11454#issuecomment-2353703805

[koppor]

Technical hint: Use git and appdirs.

• Option .bak file (even with git-bundle): Not good, because if used on Dropbox, conflicts may arise.
• Option git allows for "removing" intermediate backups
• Option rcs: There is no good Java support - and squashing revisions is not supported.

[khola22]

Hello, we are the group of students who would like to work on this issue within the framework of our school project. We are commenting so that you assign the issu to us. Thank you !

I note that the group is composed of : @khola22 me @Daren42 @Gillan0 @safaa0812 @ilias-lwa3r @nawalchahboune

[koppor]

@khola22 I can only assign students commenting on this PR. I think, for formality it is OK to assign one person of your group?

[github-actions[bot]]

Welcome to the vibrant world of open-source development with JabRef!

Newcomers, we're excited to have you on board. Start by exploring our Contributing guidelines, and don't forget to check out our workspace setup guidelines to get started smoothly.

In case you encounter failing tests during development, please check our developer FAQs!

Having any questions or issues? Feel free to ask here on GitHub. Need help setting up your local workspace? Join the conversation on JabRef's Gitter chat. And don't hesitate to open a (draft) pull request early on to show the direction it is heading towards. This way, you will receive valuable feedback.

Happy coding! 🚀

[koppor]

@khola22 I am so happy that a group of students chose this task. Please start from https://github.com/JabRef/jabref/issues/11454#issuecomment-2353703805. That should be enough for a requirements document. Then, you can add a directory for backups to org.jabref.logic.util.Directories. Then, you can add the use of git to handle files in this directory. You will probably rewrite org.jabref.gui.autosaveandbackup.BackupManager completely.

You could even craft "real" a requirement document. See https://github.com/JabRef/jabref/blob/main/docs/requirements/index.md for hints.

[nawalchahboune]

@koppor

Hello!! We are the group of students working on improving the version management of libraries in JabRef using JGit. We have implemented a feature that allows modifications to be saved as commits while giving users the ability to restore a previous version.

We have evaluated two possible approaches for version restoration:

first approach

Moving the HEAD to the restored version: This method shifts the HEAD to the selected version, making it the latest commit. However, this approach rewrites the commit history, preventing the user from restoring intermediate versions.

second approach

Rewriting the current version with the content of the restored version: This method preserves the commit structure by copying the content of the restored version to the current version. This way, the entire history remains intact, and new commits naturally follow the initial HEAD.

We would like to get your input on which approach you consider most appropriate for JabRef, taking into account flexibility and user experience.

@nawalchahboune me @khola22 @Daren42 @Gillan0 @safaa0812 @ilias-lwa3r

[koppor]

while giving users the ability to restore a previous version.

I don't know if this was requested. (I was in the wrong context)

Please do not re-invent git concepts.

Never ever rewrite history!!!

In short: Restore the file system contents. Then do a commit.

Alternatives:

You know that going back to a previous version is equal to a "git checkout commitid". And then git warns you that one works in a detached head. Then, one could work on a new branch.

One could also reset the current branch to that commit. Then, the remote branch will diverge and will need more handling.

[khola22]

Hello @koppor ! We have new codes implemented for the new BackupManagerGit class and also for the UI, we are trying now to write tests so we can be sure that it is working / correct and enhance our code. We want to have some insights on how to lead the tests for the BackupManagerGit class because it seems a bit complicated, given the whole structure of Jabref and how BackupManager is used in many other classes ... We want also to be sure that it s okay to have the backup manager and the UI codes merged, or should we keep them seperated. Thank you !

[koppor]

We have new codes implemented for the new BackupManagerGit class and also for the UI,

Nice! Where is the PR? 😅

We want also to be sure that it s okay to have the backup manager and the UI codes merged, or should we keep them seperated.

None of us can judge without seeing the code.

In general, think of JabRef also as CLI tool without any UI components. Then, JabRef should also work. Thus, the UI should be "dumb" and most logic should not be part of the UI...

[khola22]

Hello @koppor ,

We apologize for the delayed response. We've been working on validating our code and relied heavily on loggers to debug (apologies for the clutter). Here's an update: backups are now tracked in a Git repository under jabref/backups, where .bib files are copied every 19 seconds to maintain a Git history. However, change detection currently occurs only when .bib files are overwritten, which is triggered by actions like Save, Save All, or the save prompt during exit. Our reliance on listeners for tracking edits imposes strict conditions, which limits dynamic change detection and leads to two main issues: 1/ It is not possible to autosave changes without manually clicking Save or Save All. 2/ Recent changes are not recorded if the application closes unexpectedly.

The UI is mostly complete, but we are encountering challenges with the checkout functionality. We are uncertain if our current approach using JGit to revert commits is correct. On the positive side, commit retrieval, discarding changes, saving, and committing all work as intended. Since we replaced the BackupManager with a BackupManagerGit class, we adapted other components to integrate this new system.

Being new to contributing on GitHub, we are unsure whether we should open a pull request (PR) to share our incomplete work for feedback or if there is another recommended process. Your guidance would be greatly appreciated!

P.S.: We did not modify classes like CoarseChangeFilter for autosaving functionality.

Best regards, The team

[koppor]

@khola22 Thank you for the update. Please create a pull request. This enables us to investigate.

[khola22]

@koppor the pull request is done !