Closed jbsarrodie closed 1 year ago
@jbsarrodie Can we plan the merge workflow? Should we allow merging only into master? Can we be clear about what the exact process is?
Before answering, I'm copying some interesting part of our global discussion about MVP...
Have you any thoughts of the user interaction for a merge function (to master)?
Well, I thought about it a bit...
Originally posted by @jbsarrodie in https://github.com/archimatetool/archi-modelrepository-plugin/issues/67#issuecomment-431142184
I’ve thought about merging a little bit too. There are some decisions that must be made.
At first there has to be a choice made regarding branching strategy (i.e. the Git workflow). There are several pre-defined workflows for different purposes. A possible, quite simple, workflow is feature branches where branches are branched off from master.
+--9--10--12--13 Architecture Asset 2
/ / \
1--2--5--8--------11--------14 Accepted Architecture Landscape
\ \ /
+--3--4--6--7 Architecture Asset 1
For merges there are other choices. The first choice that has to be made is whether commits should come along into the master branch or a single merge commit should represent the sum of the work for a certain feature. These two different approaches are made in separate ways in git and I’m not sure of how the merge view will support the rebase function that adds all the commits made in one branch to another. I say, go for the one merge commit approach, might be most straight forward.
There are also different schools about in which order the various operations are to be performed - just merge into master or first merge master branch changes into feature branch before merge into master.
From an Archi user perspective both directions of merging probably have to be available. An architect that is working on an architecture asset during a long time has to be up to date with the accepted architecture landscape from start to finish, so Git functionality like ‘update current branch from master’ must be available. From the other perspective of an architecture workflow, there has to be a function for accepting a new architecture asset to the accepted architecture landscape, in Git terms, merge branch into master.
The sum of the above is two new actions (aside the existing Refresh):
Maybe we should (at least for some time) only allow merge when remote is accessible.
Yes, a common flow of activities of merging includes pulls of the included branches before merging as
git checkout source_branch
git pull
git checkout target_branch
git pull
Not including the pull in a merge-flow will probably trigger a new merge when pushing to a remote.
Originally posted by @mjardemalm in https://github.com/archimatetool/archi-modelrepository-plugin/issues/67#issuecomment-431323272
A possible, quite simple, workflow is feature branches where branches are branched off from master.
I agree that in practice this might be the simplest workflow for users, but I don't think we should put some constraint in the plugin's code, unless this really simplify things at this level too.
For merges there are other choices. The first choice that has to be made is whether commits should come along into the master branch or a single merge commit should represent the sum of the work for a certain feature.
My view on this is to use the same strategy that the one currently in place when merging local work into remote master: don't force a rebase nor a squash, use a simple (potentially fast-forward) merge.
There are also different schools about in which order the various operations are to be performed - just merge into master or first merge master branch changes into feature branch before merge into master.
Good remark. The sole purpose of this being to avoid solving conflicts during a merge that involves two different branches (and not the same local/remote branch), we could add an option to enforce this (ie. automatically abort a branch merge if there are conflicts).
From an Archi user perspective both directions of merging probably have to be available
I agree for the same reasons: Architecture work can be long :-)
The sum of the above is two new actions
Well, we could also decide to simply add options to current "Refresh" and "Publish" (so we could refresh from, and publish to, another branch.
Originally posted by @jbsarrodie in https://github.com/archimatetool/archi-modelrepository-plugin/issues/67#issuecomment-431746440
Lots of information to digest. I'm going to need some help to break this down into a sequence of events. :-)
Perhaps it would be easier (for me at least) if we could create one (test) Action and then use that as a starting point and adjust it as needed?
Scenario:
User is either in "master" branch or some other branch. User wants to merge a branch into the current branch.
Workflow:
Remember that we cannot assume that the user is online or not online, only whether an action such as pull or push is successful or not.
Lots of information to digest. I'm going to need some help to break this down into a sequence of events. :-)
I'll make it digest later today ;-)
So let's try to clarify....
Scenario: User is either in "master" branch or some other branch. User wants to merge a branch into the current branch (the merge commit will be created in the current branch). User will stay in the current branch after the merge. The other branch is not impacted (ie. not deleted).
I wonder if we should allow merge locally? This could be useful in some cases, but there is a high risk that someone add other commits on remote branches, leading to unwanted/unexpected situation. Maybe we should (at least for some time) only allow merge when remote is accessible.
Having thought about it, I finally think we should start (small) with offline merge and later manage online merge.
Workflow for "offline" merge:
OtherBranch
) into the current branch (CurrentBranch
)git merge OtherBranch
FWIW, here is Workflow for "online" merge:
OtherBranch
) into the current branch (CurrentBranch
)git merge OtherBranch
I've just committed a basic merge Action which is the "Workflow for "offline" merge" above. We need to test this now to see if this is correct or for side-effects.
I've just committed a basic merge Action which is the "Workflow for "offline" merge" above. We need to test this now to see if this is correct or for side-effects.
I've just tested. This seems mostly correct but I noticed that in the conflict UI, "Theirs" shows bad information (it seems that "theirs" model is not extracted from the right commit). You can easily reproduce this by creating a branch and then doing some conflicting edits on both branches and then trying to merge them.
I also noticed some weird behavior: after having created a branch I did some changes and commited them. When switching to another branch, I'm asked to commit while I didn't do any new changes. Looking to the resulting commits in SmartGit shows that the first commit doesn't contain my changes, only the second does. I don't know how to reproduce it, I might check previous actions (involving several merges tests).
I've just tested. This seems mostly correct but I noticed that in the conflict UI, "Theirs" shows bad information (it seems that "theirs" model is not extracted from the right commit).
I've just committed a change would should hopefully fix this. The MergeConflictHandler was hard coded to use a remote Ref for "theirs" so this now uses the Ref for the merged branch.
I also noticed some weird behavior...
That's strange. I was going to suggest possible EOL problems, but seems different.
That's strange. I was going to suggest possible EOL problems, but seems different.
Yes, that's really different.
It is like if some changes made during a merge to solve some conflicts had not been commited as part of the merge commit. So triggering the Commit action didn't asked me to save the model, but instead went directly to the commit of the pre-recorded changes (it seems like if model was not exported to grafico at this time but instead previous export had been used). And then, when switching to another branch, this time I was asked to save my model, which certainly triggered a grafico export and then the "right" commit content.
As said, this happened after a merge in which I had some conflicts, but as the conflict solver had some issues with identifying the right "Theirs" commit, it might be the root cause. I'll test again tonight to see if I can reproduce the issue or not.
We need to review the sequence of events in MergeBranchAction
to check whether we've made the right assumptions about what needs to happen.
We need to review the sequence of events in MergeBranchAction to check whether we've made the right assumptions about what needs to happen.
What does a git.merge()
defaults to? I see that both setFastForward
and setSquash
is commented out, but it looks like JGit defaults to a fast forward when there are no conflicts and maybe a rebase when there are conflicting changes. Is this right? (Personally, I prefer a squash since I prefer the feature branch strategy which also keeps the target branch clean of work-in-progress commits.) Might that affect the behavior?
I've just committed a change would should hopefully fix this.
I've just tested and it now works as expected.
As said, this happened after a merge in which I had some conflicts, but as the conflict solver had some issues with identifying the right "Theirs" commit, it might be the root cause. I'll test again tonight to see if I can reproduce the issue or not.
I try to reproduce the issue but can't. So I assume it has been fixed with the other issue.
We need to review the sequence of events in
MergeBranchAction
to check whether we've made the right assumptions about what needs to happen.
I checked it and this is aligned with my proposition and seems the right one (at least IMHO).
What does a
git.merge()
defaults to? I see that bothsetFastForward
andsetSquash
is commented out, but it looks like JGit defaults to a fast forward when there are no conflicts and maybe a rebase when there are conflicting changes. Is this right?
From my test the result is similar to the one of the usual Publish action: a fast forward when possible and a merge commit (not a rebase nor a squash commit) in other cases.
Personally, I prefer a squash since I prefer the feature branch strategy which also keeps the target branch clean of work-in-progress commits.
Well, I personally prefer keeping the whole commit history, I find it more intuitive for user (squash commit is really a git thing).I suggest we keep it this way for the MVP. Maybe at some point we could offer user the choice to "group" all changes from the merged branch inside one (might be a better description for a squash commit), in this case the commit message could be auto-generated as the concatenation of all commit messages.
I’ve done some testing and I must say that it feels really good testing merge in Archi. =)
My reflections:
A successful merge, when there’s no changes on the merge target, the git.merge()
will do a fast forward and does not seem to care about the setCommit(true)
. The commits are merged as expected into the target with a straightforward traceability.
If there are changes in both target and source branch the commits are merged as expected but succeed of an empty commit with the message “Merged Branch ‘branch_name’”. For some reason the setCommit(true)
then applies, I’ll guess it’s why it’s a merge and not a fast forward. What merge-strategy that JGit uses I’m not able to analyze but resolve or recursive seems most likely, probably depending on how changes are analyzed before merge.
From a user point of view, these mergers are pretty much the same, even though JGit handles it in different ways, but one is made with an empty merge commit where the other isn’t. It does not feel like a consistent behavior.
This seems like an expected behavior. The one thing is the automatical commit. If the commit should be made automatically the message could be clearer. The other possible way is to not do the automatic commit and let the user commit the changes manually.
Are there any options on the MergeCommand
that we need to set or unset?
What merge-strategy that JGit uses
Looking at the source code of MergeCommand
it seems to be MergeStrategy.RECURSIVE
If JGit is doing inconsistent things then we need to look at the available options, and see if we've made the right assumptions. It may be a JGit quirk.
If the commit should be made automatically the message could be clearer.
Any suggestions? I think it can be set easily.
The other possible way is to not do the automatic commit and let the user commit the changes manually.
Not sure if leaving the state as "unmerged" is safe?
I've looked at the source code for MergeCommand
and seen that if an option is not set explicitly JGit will use settings in the config file for the branch. To be safe, I set them as:
MergeResult mergeResult = git.merge()
.include(mergeBase)
.setCommit(true)
.setFastForward(FastForwardMode.FF)
.setSquash(false)
.setMessage(message)
.call();
To be consistent, either the fast-forward should add an merge-commit (setFastForward(FastForwardMode.NO_FF
)) or the recursive merge should remove the merge-commit (setCommit(false)
). I think either way will work but I think consistency is important.
About the merge strategy, I think it’s a good choice to use recursive. If JGit don’t try something else in other situations, default values with fast forward or recursive merge will work fine.
Regarding the next stage for branch merging, how should we achieve this?
Is it the case that the user needs to do some actions first before a merge?
Do we want these actions to be automatically done before a Merge Branch action? If so, I think that things could go wrong and the user might not know what to do. How do we determine a "success" or "fail"?
Regarding the next stage for branch merging, how should we achieve this? Do we want these actions to be automatically done before a Merge Branch action? If so, I think that things could go wrong and the user might not know what to do. How do we determine a "success" or "fail"?
My feeling is that we should decide what is the "normal" what to merge, ie. online or offline. I would argue that, for a merge to be really a success, it should be done online, so I would go for an online merge by default, unless the user decides the opposite.
In addition, as merge is a multi-step action, I suggest we display some progress information.
So this merge action could become something like:
So merge workflows become:
Workflow for "offline" merge:
OtherBranch
) into the current branch (CurrentBranch
)git merge OtherBranch
Workflow for "online" merge:
OtherBranch
) into the current branch (CurrentBranch
)git merge OtherBranch
My fear is that's a lot of dialog boxes and stages that need to be strung together (and where a "fail" could happen at any stage).
It's probably not a good idea to have several dialog boxes one after the other, but assembling these stages into a single UI wizard would mean re-writing these actions in existing code and dialogs into one UI and handling things differently.
Are we ideally thinking of a multi-stage wizard? Or are we asking the user to do these operations manually?
Boiling this down, it amounts to doing these Archi code Actions:
Current Branch - PushModelAction
Switch Branch - SwitchBranchAction
Other Branch - PushModelAction
Switch Branch - SwitchBranchAction
Current Branch - MergeBranchAction
(Stages 1 - 8 are the RefreshAction
)
Thinking aloud...
What about keeping only the first (and main) dialog that explains the whole process and ask user to continue (online), continue (offline) or abort. Once the user has decided he will know what happens, so no need for additional dialogs along the way.
I'll add an experimental POC second action, something like "Merge Branch (exp)" (which won't have the explanatory dialog for now) to see if it's possible to sequence the existing actions together and if it works...
I've pushed a new branch - "online-merge". This is experimental work.
I've added a new context menu item to Branches View - "Experimental Merge", It does a sequence of actions as follows:
Current Branch - PushModelAction Switch Branch - SwitchBranchAction Other Branch - PushModelAction Switch Branch - SwitchBranchAction Current Branch - MergeBranchAction
This allows us to re-use code from the existing Actions. You may see more than one ProgressMonitor because of this.
Please test!
I've done some testing and it seems to work, but we need to really try different scenarios. If the sequence works, I can then improve the UI with a unified dialog as suggested.
There's a problem with the way that JGit merges after resolving a merge conflict. Here's an image taken after merging in Archi:
The "origin/master" branch is not on the timeline, so the next time you pull it will merge this back into local "master."
Here's an image taken after doing a manual merge in SmartGit:
The "origin/master" branch is on the timeline so it does not create a problem.
I've tried setting different merge strategies in JGit but still have this problem.
Edit - FIXED! See below.
There's a problem with the way that JGit merges after resolving a merge conflict.
I'm not sure to understand the issue, can you explain it a bit more (its consequences)?
Please test!
Will do soon.
Look at the second image above. That's how it should look after a merge conflict. "origin/master" is behind "master" in the timeline.
However, the first image shows that "origin/master" is not on the timeline after a merge commit. Thus, you cannot push until you pull to merge origin into master.
I have found the problem and submitted a fix:
When a merge conflict is resolved a commit is necessary to commit the merge result, It was being committed with "amend last commit" set to true. Setting it to "false" solves the problem.
Here is the sequence of actions performed in the "Experimental Merge" menu action:
That's a lot of stuff to do, so it can take a while.
Ok, I've just done some tests and while this works as expected, I have some remarks:
I can then improve the UI with a unified dialog as suggested.
I guess we can move on with this. BTW, is the text of Merge Conflict dialog easily customizable depending on the stage at which it is called? If yes we could tune the text a bit so that user know at which step he is and which branches are compared (local branch X versus remote branch X, local branch Y versus remote branch Y or local branch X versus local branch Y).
When switching branch while I did some (uncommitted) changes, I am asked to save but not to commit.Not being forced to commit makes sens because I could want to commit these new changes in another branch, but being offered the choice to commit or not would be even better.
Yes, agreed. Now the logic is this:
Update - the GitHub branch "online-merge" is now gone and we are back to the main development branch "branching"
There is now only one "Merge into Current Branch" Action. This shows a dialog with the choices of "online", "local" or "cancel".
The final Merge should be followed by a push to remote (we're online after all).
Done.
We should decide how to manage the deletion of a branch after a merge. Maybe a dialog at the end: "You've successfully merged branch X into Y. You can now safely delete branch X." with two options: "Delete" and "Don't Delete" ?
Done. The options are "Yes" and "No" to the question "Do you want to delete branch x?"
BTW, is the text of Merge Conflict dialog easily customizable depending on the stage at which it is called? If yes we could tune the text a bit so that user know at which step he is and which branches are compared (local branch X versus remote branch X, local branch Y versus remote branch Y or local branch X versus local branch Y).
There are two places where the Merge Conflict dialog can appear:
I've added some text depending on whether we are pulling and merging from the remote or merging from local branch to local branch.
Please test!
Please test!
Ok, I've just tested. It works but I noticed some small bugs:
I've added some text depending on whether we are pulling and merging from the remote or merging from local branch to local branch.
I'd suggest to check if both branches have the same name: if yes we can assume that we are publishing, if no we are merging 2 branches. So the message could be either:
We should not offer to remove the merged branch if it is master (deleting master is never a good idea)
Fixed
It seems that the Change History view only lists branches which exist locally while Branches view also lists remote branches.
This was intentional because I'm not sure if user actions were suitable for remote branches that have not been checked out. However, I've now enabled remote branches here. Let's test that.
Very strange: a publish action with conflict seems to amend previous commit
This is actually happening on the "Refresh" (pull) action prior to "Publish". I seem to remember a case where we wanted this behaviour. However, I've now set amend to false. Let's test that.
@jbsarrodie Can we plan the merge workflow? Should we allow merging only into master? Can we be clear about what the exact process is? For example:
Originally posted by @Phillipus in https://github.com/archimatetool/archi-modelrepository-plugin/issues/67#issuecomment-435660072