Open koppor opened 2 years ago
At https://github.com/JabRef/jabref/issues/8431 a user described his issues with current JabRef not having any git support.
Hi, my colleague and I are students from the University of São Paulo in Brazil, and we were asked to help in the open-source community. We would like to take a look into this issue and try to implement it. Could we have it assigned to me? Do you have any information to share that would be nice to know beforehand?
Dear Leonardo! Thank you for choosing JabRef! I reserved the issue for you.
Please note that this is a big project.
I shared information for behavior in the issue description. You can start investigating current behavior at SaveDatabaseAction.java.
Here our general newcomer text (which is also information to share)
As a general advice for newcomers: check out Contributing for a start. Also, guidelines for setting up a local workspace is worth having a look at.
Feel free to ask here at GitHub, if you have any issue related questions. If you have questions about how to setup your workspace use JabRef's Gitter chat. Try to open a (draft) pull-request early on, so that people can see you are working on the issue and so that they can see the direction the pull request is heading towards. This way, you will likely receive valuable feedback.
@lbenicio Any updates? Any UML diagrams? Concepts? UI sketches? Initial code?
Hi, sorry for the lack of updates.
From now, the idea for the simple implementation seems simple. Here what we have:
We were struggling with the point of external user interaction with the git to understand if it changes any behavior.
Now we are looking into how to open the UI with the changes.
From what I understand, we should present a solution proposal and then follow up with the code, right?
We were scheming the solution, but we did not have a diagram. We are on 2 people working 4h/week/each on the project, so the progress may be a little slow?
But what concerns me is that we did not figure what the complexity of the project yet. It looks like direct implementation.
Should we “reserve” an issue just when we get a proper solution?
Hi, sorry for the lack of updates.
No worries.
From now, the idea for the simple implementation seems simple. Here what we have:
1. Verify .git 2. call git sync method 3. issue the commands 4. on failure, open a UI.
Sounds OK. No need for a real UI. We have "dialogService". Example:
dialogService.notify(Localization.lang("Issue on GitHub successfully reported."));
We were struggling with the point of external user interaction with the git to understand if it changes any behavior.
I don't know, which of these steps outlined above are unclear:
git pull --rebase
is executed.git pull
git push
From what I understand, we should present a solution proposal and then follow up with the code, right?
In Software Engineering, one uses diagrams etc to communicate. UML sequence diagrams are a good thing to learn. -- You can also write code and submit a work-in-progress pull request to JabRef. To discuss things.
We were scheming the solution, but we did not have a diagram. We are on 2 people working 4h/week/each on the project, so the progress may be a little slow?
I don't know about your final date. Slow progress is very OK. Better 4h "deep work" hours than 8 non-focussed ones. I would suggest that you come back with intermediate results after each working block. We can open a separate skype group for you if you want.
I would also like to share the thought that this is a work educating you to become a better programmer and a better software architect. One needs 10.000 hours to become an expert. Thus, each hour invested here brings you closer to the goal becoming an expert.
In October, there is #hacktoberfest. I would recommend to use that opportunity to level-up your coding skills even more.
But what concerns me is that we did not figure what the complexity of the project yet. It looks like direct implementation.
The project is flagged as "large", because it is not straight-forward code. One needs to understand the current saving code and where to plug-in. It could be that the final "productive" code is 50 lines of code. Nevertheless, there will be 500 to 1000 lines of code for testing (JUnit tests, ...).
Should we “reserve” an issue just when we get a proper solution?
It is very OK to reserve an issue. We take back the reservation if the solution is implemented :) or the student resigns. We have enough other issues where other persons can work on.
Great, thank you the help and very detailed attention! Much appreciated!
In Software Engineering, one uses diagrams etc to communicate. UML sequence diagrams are a good thing to learn. -- You can also write code and submit a work-in-progress pull request to JabRef. To discuss things.
We will try to submit a work in progress solution/diagram to exemplify the solution as fast as possible.
The project is flagged as "large", because it is not straight-forward code. One needs to understand the current saving code and where to plug-in. It could be that the final "productive" code is 50 lines of code. Nevertheless, there will be 500 to 1000 lines of code for testing (JUnit tests, ...).
perfect, I believe we are on the right track.
I will take the opportunity to introduce my partner @marcelojunior1
Hi @koppor , I'm Marcelo and Leonardo's partner in our class. Thank you for your support and we hope to be able to contribute to JabRef.
@marcelojunior1 also welcome on bord!
I think, the place in the code is /src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java#L257
The are already test cases for SaveDatabaseAction
. You can add git-based tests.
For another project, I needed to create a git repository within JUnit tests. Maybe, the code is an inspiration for your tests, too? -- https://github.com/koppor/heylogs/blob/34c84ebb101b310bd0d715c669ab38e6d0300e65/heylogs-api/src/test/java/internal/heylogs/GitDiffRuleTest.java#L120
Great! The help is much appreciated, thank you!
We intended to update the git handler class and use it to manage this kind of git repositories as well.
About our progress: we are just finishing our solution proposal
Yes, the tests are really the complexity here, and this will help a lot.
After a long wait, here is our solution proposal:
We should start writing unit tests next week.
Maybe we add our code to an “intermediate” pull request, so you can track our progress
Thank you for sharing the UML flow chart.
I have following comments:
.bib
file "bib file" - "bibbi" seems something non-English.Code hint: On org.jabref.logic.git.GitHandler you see some git magic. I think, that class cannot be used directly: It creates a git repository on start. Either you modify it - or your start with your own class. Your own class could limit the the life cycle of the GitRepository when doing a save - and then throw away that handle (try-with-resources).
Notes
org.jabref.gui.collab.DatabaseChangesResolverDialog
.Advanced: The SLR functionality of JabRef (included in org.jabref.logic.git.GitHandler
) has support to handle patches (SlrGitHandler.java#L95). I think, this cannot be directly used, but I have not thought about it.
The core idea should be to find the modified entries by the citation key (AKA BibTeX key) locally and remotly. And then displaying a merge entries dialog if the bibtex key is remote and locally the same, but different content. -- You can call the org.jabref.gui.collab.DatabaseChangesResolverDialog
for that.
One could even reuse that dialog:
Put in a flow:
Screenshot of the Merge Entries dialog (as of today)
Wow, great, thanks!
I don't see how "commit bibbi file" can work without any saving (I think, the file should be saved before any interaction with a git remote. Thus, "saveDatabase()" should be called, and then the git "magic" handled
Totally forgot that. You are correct. We should wait for file save before calling for git save.
you need to make JabRef aware of the remote changes: The file needs to be loaded again in JabRef. Search for the usage of org.jabref.gui.collab.DatabaseChangesResolverDialog.
Forgot that too. Totally makes sense to update JabRef internal database with the updated changes from git pull.
I will update our flow chart to reflect the changes and post here.
Thank you! - As a side comment: Please ensure that gitk
is running on your machine. Then you can view the state of the repository easily. Tutorial at: https://lostechies.com/joshuaflanagan/2010/09/03/use-gitk-to-understand-git/.
So, this is the final idea. The user should be on a loop to fix conflicts until the user cancels the action, and we summarize to the normal flow and finish the git save method.
From this, we believe we should have an implementation and test strategy (which would be bigger then the code itself)
Sorry, I accidentally remove my assignment. :(
Regarding the diagram. I thought, it was an UML flow chart, now you are using another syntax. We computer scientists are picky on the syntax of things. Please choose an existing thing (and do not re-invent something else). Example: Casing OUTPUT and output. I think, the extra casing is not necessary. -- Round circles versus boxes versus boxes with rounded corners versus a yellow box. -- Stick with plain rectangles please.
I put the comments straight, I hope, this is OK
saveGitDatabase()
is a method, you will implement. The flowchart suggests that some git checking happens afterwards. This is strange. Just remove saveGitDatabase()
Ok, great! Yes, we decided not to use a proper diagram format, it was supposed to a sketch sorry, it was not a good idea.
We got some progress, and managed to create the commit, we should submit a “preview” PR, so you can track our progress soon.
Ok, great! Yes, we decided not to use a proper diagram format, it was supposed to a sketch sorry, it was not a good idea.
10.000 hours to become an expert. Drawing one diagram and getting feedback makes you a better diagrammer. Diagrams are useful for software developers and architects. Keep practicing!
Note that drawing and discussing a diagram takes shorter time than implementing everything. This is the main benefit of diagrams.
We got some progress, and managed to create the commit, we should submit a “preview” PR, so you can track our progress soon.
Nice. Looking forward!
Note that drawing and discussing a diagram takes shorter time than implementing everything. This is the main benefit of diagrams.
Yes, we agree! We were trying to do push/commit operations using ssh, but it was very complicated to import and configure org.eclipse.jgit.ssh.apache package. Now it's working and we can think more about the diagram.
We just submitted a pull request, so everybody can track our progress and provide suggestions.
Hi @koppor
Writing to update you about our progress. We almost finished the implementation and are moving to tests. We got the preferences' menu working and the git ssh, and user/pass login working as well as the login retrieval from env variables and preferences menu.
Next sprint, we are moving to unit tests (if nothing unexpected appears)
Our ETA would be 4 weeks to finish everything. I hope we can make it (since our semester would end).
Thank you for the update. It would be nice if you reopened the PR or commented if the time budget is spend on your side. No need to implement GPG signing. Just get the "basic" functionality running properly. Since this touches a core of JabRef, clean and working code is more important than features.
Especially, the user experience has to be great ^^. One can check how other tools do it. At https://www.reddit.com/r/LaTeX/comments/1179mdx/texstudio_git_integration_for_easy_committing/ there is a good explanation for TeXStuido.
Side note: I would have relied on https only, but OK for me that you worked on SSH, too. 😅
Ok, finally, we finished all unit testing, e2e testing and submited the PR. https://github.com/JabRef/jabref/pull/10586/
@koppor just to let you know, we did not abandon the project. We are in the final weeks, many finals tests to do, we missed the class dates to finish the PR, but we are going to finish it. Maybe the first or second week of December.
For requirements refinement, please read on related work, e.g., TeXStudio git integration: https://www.reddit.com/j9az1u9?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=2
As user, I would like to have a seamless integration in git. For "straight-forward" cases, JabRef should do "magic".
Implementation drivers
Opening a library
Saving of a library
No action taken (to save number of commits)
Saving a library
git pull --rebase
is executed.git pull
git push
Advanced implementation
A user might want to trigger pushing