Closed eposse closed 6 months ago
@eposse @j26151
I'd argue that BEFORE we merge those PRs, that the CX version should be upgraded. We should create a streams branch for the current "papyrus" branch that exists today so that development can continue on that 2.X version if we need it. Then the HEAD of the papyrus branch can become 3.0 with these two PRs on top of that.
Thoughts?
That all makes sense to me.
Let me confirm I understand. The normal process that we do when upgrading a version is as follows:
Assuming the last version released is 2.N, the version on the main papyrus branch is 2.(N+1) and we have a 2.N tag (on the commit of the release) and a streams/v2.N.x-maintenance branch off the 2.N tagged commit. Currently the last version released was 2.5.0 and the version on the papyrus branch is 2.6.0.
The steps we do are:
So If I understand you, we should do the following, in the order specified:
Specifically we do not create a 2.6.0 release, which is why I think the 2.6.0 tag would be optional (unless you think it might be useful for something). Note that the only commits in 2.6.0 (the papyrus branch) that have been made since the 2.5.0 release are changes made automatically by GitHub's dependabot updating the GitHub's workflow file, which just updates versions for some actions in that file (all have been reviewed and verified). In other words, there are no functional differences at all with 2.5.0.
Please confirm if this is what you want.
Thanks
I think we're on the same page. The release steps I certainly agree with. Because the 2.6.0 tag only has dependabot changes, I'm fine with y'all not making a formal release for that version and just making the 2.6.x streams branch for it.
I don't know that we need you to do steps 6-10 though. We could probably just keep rolling on 3.0.0 changes unless you want to make a 3.0.0 release that only contains the bump in the papyrus version (not to minimize all the work you've done in those PRs). But if you think that change big enough to drop a new release, go for it!
Yeah, I don't think NGC needs a 3.0.0 release, especially since we haven't testing anything yet and could have changes or issues when testing our production models. I would prefer to just see the head of papyrus be targeting a future 3.0.0 release instead of one be released and tagged just yet.
Right, so steps 6-10 would be for a 3.0.0 release, and I was assuming that you wanted to make that release. The change is big, considering the very large number of commits for patches and fixes required to remove log4j 1 dependencies and upgrade to Papyrus 6.5, but it is true that the end-user might not notice significant differences, as all those changes are under the hood, except that is, for changes in Papyrus itself, since we went from 4.8 to 6.5 (two major versions and several minors). But it does make sense to wait until we have incorporated some more substantial features and bug-fixes like the one described in Issue #450. So I'm ok with not releasing 3.0.0 for now.
And this reminds me: how should I prioritize the next tasks? #450 is a big one, but I saw there are a few new ones reported. Also, I want to contribute the patches to the different open source projects, and at least the latest fixes for the "architecture domain merger" in Papyrus, which I think it's pretty important. What do you think?
We likely will want to make that 3.0.0 release, just not yet. We know the change is big and certainly warrants a major version bump like 3.0.0 but we need to test it internally and might wish to add a few more features/bug fixes between now and when we need to lock to that 3.0.0 version (at which time we'd ask for a formal release).
But as I said before, if you and your team want to make that release now, you certainly can as you own this repo/product. NGC just isn't going to pay for it since we don't need it at the moment.
Yeah, Michael hit the nail on the head. NGC most likely won't need this 3.0.0 release until Q3/Q4 if I had to guess and almost none of our developers will be using this new version until then.
I've been thinking a bit more about this, and I want to make sure we are really on the same page.
From the points you made above:
Just to clarify, I do not have a strong opinion about releasing 3.0 now, and as I mentioned before, it does make sense to wait until we have added more user-visible features and bug-fixes.
Having said that, I want to clarify a few things, before bumping the version on the main papyrus branch.
So I have a few questions:
Thanks
PS: Just to add, in case you missed the comment in #471, a build of #480 is ready for you to test. You can get it here.
Some answers to your questions.
- From the above, I infer that you do not intend to have any releases from us between the current 2.5 and 3.0 sometime in Q3/Q4, even if we integrate new features and bug-fixes between now and then. Is this correct?
Correct. NGC only requires a release when we, NGC, release our product. Since CX is open source, anyone can make releases, but NGC isn't going to pay for them until NGC needs one.
- Assuming the answer to the previous question is yes, all new features and bug-fixes (including e.g. https://github.com/ZeligsoftDev/CX4CBDDS/issues/450) will be only on version 3.0.0, correct?
I wouldn't say all new features and bug fixes. It is possible that we find a bug in CX 2.5.0 that we want fixed on the 2.5.0 maintenance stream and the branch that could eventually be 2.6.0 (as well as 3.0.0 if it exists there as well). Additionally, it is possible, but not probably that in the future we want a 2.6.0 release for one reason or another. But in any event, any new bug tickets or large features should definitely go on 3.0.0 and possibly go on 2.5.0 and the branch that could eventually be 2.6.0
- Once you have tested the latest build from PR https://github.com/ZeligsoftDev/CX4CBDDS/pull/480 (which addresses both Issue https://github.com/ZeligsoftDev/CX4CBDDS/issues/471 and https://github.com/ZeligsoftDev/CX4CBDDS/issues/476), can we merge it into the main papyrus branch? (I think this only makes sense if the version in the main papyrus branch has been bumped to 3.0.0)
Yes, this large change to remove the old log4j issue should only be applied to the 3.X branch. the 2.5.0 maintenance stream and future 2.X stream/branch should not have all of these changes.
Do you currently have any timeline and resources for testing PR https://github.com/ZeligsoftDev/CX4CBDDS/pull/480?
How do we prioritize the currently open issues?
We are discussing internally these topics. Unfortunately, when the new year begins, we have some open questions that can take some time to get situation.
Can I report in the time-sheets, the time spent on reporting the Papyrus bug on the "architecture domain merger" and provide its fix? (It takes some time to create the proper report, minimal reproducer and patch)
Can I report in the time-sheets, the time spent on contributing the multiple patches to other projects (Apache, ECP, Eclipse, GMF, MWE, Nebula, OCL, Papyrus, Papyrus Compare, Xpand, Xtend, and Xtext)? (Also takes time, including contacting the projects teams).
I am not entirely sure what "architecture domain merger" is. But, I think as a rule, any time you spend related to tickets NGC approves to work, should definitely be reported in the time sheet so that you aren't doing any work unaccounted for. I think if there are any large issues such as "It will take 40 hours to get XXX change applied to Xpand" we should have another conversation, but since you have the patches done, I would imagine the work to get it applied to not be extraneous
I have a couple of followup questions to clarify a couple of things.
Some answers to your questions.
- From the above, I infer that you do not intend to have any releases from us between the current 2.5 and 3.0 sometime in Q3/Q4, even if we integrate new features and bug-fixes between now and then. Is this correct?
Correct. NGC only requires a release when we, NGC, release our product. Since CX is open source, anyone can make releases, but NGC isn't going to pay for them until NGC needs one.
- Assuming the answer to the previous question is yes, all new features and bug-fixes (including e.g. Add support for ExF to CX modeling tool #450) will be only on version 3.0.0, correct?
I wouldn't say all new features and bug fixes. It is possible that we find a bug in CX 2.5.0 that we want fixed on the 2.5.0 maintenance stream and the branch that could eventually be 2.6.0 (as well as 3.0.0 if it exists there as well). Additionally, it is possible, but not probably that in the future we want a 2.6.0 release for one reason or another. But in any event, any new bug tickets or large features should definitely go on 3.0.0 and possibly go on 2.5.0 and the branch that could eventually be 2.6.0
Ok, so you think that you may want a 2.6.0 release, right? If so, then we shouldn't bump the version in the papyrus branch to 3.0.0, as agreed a few messages before. If this is the case, then we should leave the papyrus branch alone, with the current 2.6.0 version, and only bump the version to 3.0.0 on the branch that upgraded Papyrus and added all the patches. Is this correct?
Alternatively, we could create a separate 2.6.0 branch (different from the 2.6.x-maintenance branch) and bump the version to 3.0.0 in the papyrus branch and merge all the changes. But I imagine you would prefer the first option as you haven't tested the new build yet, right?
By the way, since the new branch upgraded Papyrus and you won't be using it for now, it is very possible that some of the bugs found are applicable only to the old version of Papyrus. Furthermore, the fix of some bugs may not be the same on the two versions, which means that for bug or feature will take more time to resolve or implement, as some APIs and code have changed, and debugging and testing will have to be done on both versions 3.0 and 2.5 (or 2.6?). It's the cost of keeping older versions of software while upgrading to newer versions in development.
- Once you have tested the latest build from PR Papyrus Issue #471 #480 (which addresses both Issue Upgrade Papyrus version #471 and Remove Log4j1 from Papyrus. Replace with Log4j2 #476), can we merge it into the main papyrus branch? (I think this only makes sense if the version in the main papyrus branch has been bumped to 3.0.0)
Yes, this large change to remove the old log4j issue should only be applied to the 3.X branch. the 2.5.0 maintenance stream and future 2.X stream/branch should not have all of these changes.
Right, but as mentioned above, if you do want to potentially release a 2.6.0 version, then we either bump the version to 3.0.0 only on the branch with the patches and not on the papyrus branch, or we create a separate 2.6.0 branch and bump the version to 3.0.0 on the papyrus branch and merge all the changes.
Do you currently have any timeline and resources for testing PR Papyrus Issue #471 #480?
How do we prioritize the currently open issues?
We are discussing internally these topics. Unfortunately, when the new year begins, we have some open questions that can take some time to get situation.
Ok, I'll work on submitting the patches to the different projects in the meantime, and preparing the bug report to Papyrus.
Can I report in the time-sheets, the time spent on reporting the Papyrus bug on the "architecture domain merger" and provide its fix? (It takes some time to create the proper report, minimal reproducer and patch)
Can I report in the time-sheets, the time spent on contributing the multiple patches to other projects (Apache, ECP, Eclipse, GMF, MWE, Nebula, OCL, Papyrus, Papyrus Compare, Xpand, Xtend, and Xtext)? (Also takes time, including contacting the projects teams).
I am not entirely sure what "architecture domain merger" is. But, I think as a rule, any time you spend related to tickets NGC approves to work, should definitely be reported in the time sheet so that you aren't doing any work unaccounted for. I think if there are any large issues such as "It will take 40 hours to get XXX change applied to Xpand" we should have another conversation, but since you have the patches done, I would imagine the work to get it applied to not be extraneous.
The issue with the "architecture domain merger" is the main bug I encountered when testing the 3.0.0 version after upgrading to Papyrus 6.5, and which I mentioned in Issue #471, and took me a couple of weeks to debug and fix. It was a big bug that affected many things in Papyrus, including the following: unable to open deployment plan editors, unable to open diagrams, unable to create components, both monolithic and assemblies, empty "New Diagram" pop-up menu, and others. The cause of this error was the upgrade to Papyrus 6.5 as a central aspect of Papyrus, called the "architecture domains manager" went through a major revamp that broke our "architecture context model". Without getting into details, an "architecture context" is a language like UML, SySML, or AXCIOMA, an "architecture domain" is something like "Software Engineering" or "Systems Engineering", which defines one or more architecture contexts. Each context defines a set of "representation kinds" like types of diagrams, and a set of "viewpoints" which have an associated set of representation kinds. The idea is that each model is associated to a context (e.g. AXCIOMA) and you can have one or more viewpoints (in AXCIOMA we have only one), but the viewpoint determines what elements are available in the UI, for example, what entries are shown in the "New Diagram" menu, or in the palette, what kind of diagrams you can create or view, etc. The "architecture domain manager" handles all this, and it has a component called the "merger" which collects and merges all relevant, active domains, contexts, viewpoints and representation kinds. This changed considerably and had a very subtle and difficult to detect bug. This is the one that took me a while to find, and the one I am working to report to Papyrus with the fix. This has taken a bit of time because it means creating a minimalistic Papyrus-only sample model to reproduce the bug, get the appropriate developer setup for plain Papyrus, reapply the fix on the latest branch (6.6 or 6.7) and test it there.
Similarly, submitting the patches to the other projects does take a bit of time, because it requires tracking and contacting their developers, figuring out how to submit the patches (each project has its own way), determine if it can be submitted on an older branch (as is the case for most patches), etc. And as I mentioned, there are quite a few of these: Apache HTTP components, Apache XMLgraphics, ECF, Eclipse, GMF, MWE2, Nebula, OCL, Papyrus, Papyrus Compare, Xpand, Xtend/Xtend2, and Xtext.
I'm working on all of these. I assume that is a reasonable way to proceed while you decide about the priorities for the next tasks, and I want to make sure I can report these in the timesheets. Is that right?
Ok, so you think that you may want a 2.6.0 release, right? If so, then we shouldn't bump the version in the papyrus branch to 3.0.0, as agreed a few messages before. If this is the case, then we should leave the papyrus branch alone, with the current 2.6.0 version, and only bump the version to 3.0.0 on the branch that upgraded Papyrus and added all the patches. Is this correct?
Alternatively, we could create a separate 2.6.0 branch (different from the 2.6.x-maintenance branch) and bump the version to 3.0.0 in the papyrus branch and merge all the changes. But I imagine you would prefer the first option as you haven't tested the new build yet, right?
I think we are on the same page. We should have 2 branches, one for the possible future 2.6.0 release that does not have any of the new papyrus updates or log4j changes, and one for the future 3.0.0 release that does have all of those changes.
By the way, since the new branch upgraded Papyrus and you won't be using it for now, it is very possible that some of the bugs found are applicable only to the old version of Papyrus. Furthermore, the fix of some bugs may not be the same on the two versions, which means that for bug or feature will take more time to resolve or implement, as some APIs and code have changed, and debugging and testing will have to be done on both versions 3.0 and 2.5 (or 2.6?). It's the cost of keeping older versions of software while upgrading to newer versions in development.
Completely understand.
Ok, I'll work on submitting the patches to the different projects in the meantime, and preparing the bug report to Papyrus. Great Thank you
... I'm working on all of these. I assume that is a reasonable way to proceed while you decide about the priorities for the next tasks, and I want to make sure I can report these in the timesheets. Is that right?
Yes, these are all related to the original task of upgrading CX and Papyrus to no longer have log4j and to work to contribute back these fixes where accepted and possible. The only place NGC would possibly draw the line in the future is if working with owners of the other projects start to take increasing amounts of time and effort due to issues with working with the other projects.
Ok, so you think that you may want a 2.6.0 release, right? If so, then we shouldn't bump the version in the papyrus branch to 3.0.0, as agreed a few messages before. If this is the case, then we should leave the papyrus branch alone, with the current 2.6.0 version, and only bump the version to 3.0.0 on the branch that upgraded Papyrus and added all the patches. Is this correct?
Alternatively, we could create a separate 2.6.0 branch (different from the 2.6.x-maintenance branch) and bump the version to 3.0.0 in the papyrus branch and merge all the changes. But I imagine you would prefer the first option as you haven't tested the new build yet, right?
I think we are on the same page. We should have 2 branches, one for the possible future 2.6.0 release that does not have any of the new papyrus updates or log4j changes, and one for the future 3.0.0 release that does have all of those changes.
Ok, will do. If it's all the same to you, I'll keep the papyrus branch with 2.6.0, and create a branch called "papyruscx-3" or something like that. There is a question about how do we handle PR #480. I can think of the following options:
Personally I would prefer option 2, as I don't like to leave an Issue and a PR indefinitely open.
Which option would you prefer? And do you have any idea if you will be testing that anytime soon?
... I'm working on all of these. I assume that is a reasonable way to proceed while you decide about the priorities for the next tasks, and I want to make sure I can report these in the timesheets. Is that right?
Yes, these are all related to the original task of upgrading CX and Papyrus to no longer have log4j and to work to contribute back these fixes where accepted and possible. The only place NGC would possibly draw the line in the future is if working with owners of the other projects start to take increasing amounts of time and effort due to issues with working with the other projects.
Ok, understood. Thanks.
Personally, I too would like option 2. I would like to get @J17359 opinion on the matter as well before we go ahead.
Agree. Option 2 sounds best - merge now and write new tickets in future if needed
I created a new branch called 'papyruxcx-3' to be the main branch for the 3.x version, but I'm not entirely happy with that name, or the name of the main branch for the 2.x versions, which is 'papyrus', not very descriptive. So considering that we have many branches called "streams/v2.x.y-maintenance", I thought that it might be a good idea to have a more uniform convention and rename the new branch to "streams/v3" and the 'papyrus' branch to "streams/v2". I've discussed it with Paul and Young-Soo and they agree. Would it be OK with you if we rename these branches (specially the existing 'papyrus' branch)?
Fine with me. We'll just have to make a few changes internally to find the new branches since we've hardcoded papyrus
in some of our build scripts, but I don't see that being terribly difficult.
Ok, would you like me to wait until you make those changes, or can I rename them now?
Rename them whenever you want. We'll notice the breakage when our CICD jobs start failing 😄 so it'll force us to fix them.
Thanks. Will do.
Ok, I've finished all the tasks for this. We now have two branches:
streams/v2: the old 'papyrus' branch; this is where work towards a 2.6.0 version will be. streams/v3: the new branch, based on Papyrus 6.5 and with all the patches replacing Log4j 1 with Log4j 2.
Both builds are successful.
If you are ok with it, I'll close this issue.
Close away!
Issue and tracking information
Developer's time Estimated effort to fix (hours):
Developer's Actual time spent on fix (hours)
Issue reporter to provide a detailed description of the issue in the space below
Once we merge #471 and #476, the new version of CX should be upgraded to 3.0.