search

ZeligsoftDev / CX4CBDDS

CX4CBDDS component modeling and generation tool
Apache License 2.0
8 stars 5 forks source link

Update version to 3.0 #490

Closed eposse closed 6 months ago

eposse commented 6 months ago

Issue and tracking information

Developer's time Estimated effort to fix (hours):

Developer's Actual time spent on fix (hours)

Issue reporter to provide a detailed description of the issue in the space below

Once we merge #471 and #476, the new version of CX should be upgraded to 3.0.

J17359 commented 6 months ago

@eposse @j26151

I'd argue that BEFORE we merge those PRs, that the CX version should be upgraded. We should create a streams branch for the current "papyrus" branch that exists today so that development can continue on that 2.X version if we need it. Then the HEAD of the papyrus branch can become 3.0 with these two PRs on top of that.

Thoughts?

emammoser commented 6 months ago

That all makes sense to me.

eposse commented 6 months ago

Let me confirm I understand. The normal process that we do when upgrading a version is as follows:

Assuming the last version released is 2.N, the version on the main papyrus branch is 2.(N+1) and we have a 2.N tag (on the commit of the release) and a streams/v2.N.x-maintenance branch off the 2.N tagged commit. Currently the last version released was 2.5.0 and the version on the papyrus branch is 2.6.0.

The steps we do are:

  1. Tag the latest commit on the papyrus branch with 2.(N+1).0.
  2. From that same commit, create a new streams/v2.(N+1).x-maintenance branch
  3. Bump version numbers in the papyrus branch to 2.(N+1).0
  4. Bump version numbers in the streams/v2.(N+1).x-maintenance branch to 2.(N+1).1
  5. Create release based on the 2.(N+1).0 tag.

So If I understand you, we should do the following, in the order specified:

  1. (Optional) Tag the latest commit on the papyrus branch with 2.6.0
  2. From that same commit, create a new streams/v2.6.x-maintenance branch
  3. Bump version numbers in the papyrus branch to 3.0.0
  4. Bump version numbers in the streams/v2.6.x-maintenance branch to 2.6.1
  5. Merge PR #480 (addresses both Issues #471 and #476) to the papyrus branch (after you test it)
  6. Tag the commit on the papyrus branch with 3.0.0
  7. From that same commit, create a new streams/v3.0.x-maintenance branch
  8. Bump the version numbers in the papyrus branch to 3.1.0
  9. Bump the version numbers in the streams/v.3.0.x-maintenance branch to 3.0.1
  10. Create release based on the 3.0.0 tag

Specifically we do not create a 2.6.0 release, which is why I think the 2.6.0 tag would be optional (unless you think it might be useful for something). Note that the only commits in 2.6.0 (the papyrus branch) that have been made since the 2.5.0 release are changes made automatically by GitHub's dependabot updating the GitHub's workflow file, which just updates versions for some actions in that file (all have been reviewed and verified). In other words, there are no functional differences at all with 2.5.0.

Please confirm if this is what you want.

Thanks

J17359 commented 6 months ago

I think we're on the same page. The release steps I certainly agree with. Because the 2.6.0 tag only has dependabot changes, I'm fine with y'all not making a formal release for that version and just making the 2.6.x streams branch for it.

I don't know that we need you to do steps 6-10 though. We could probably just keep rolling on 3.0.0 changes unless you want to make a 3.0.0 release that only contains the bump in the papyrus version (not to minimize all the work you've done in those PRs). But if you think that change big enough to drop a new release, go for it!

emammoser commented 6 months ago

Yeah, I don't think NGC needs a 3.0.0 release, especially since we haven't testing anything yet and could have changes or issues when testing our production models. I would prefer to just see the head of papyrus be targeting a future 3.0.0 release instead of one be released and tagged just yet.

eposse commented 6 months ago

Right, so steps 6-10 would be for a 3.0.0 release, and I was assuming that you wanted to make that release. The change is big, considering the very large number of commits for patches and fixes required to remove log4j 1 dependencies and upgrade to Papyrus 6.5, but it is true that the end-user might not notice significant differences, as all those changes are under the hood, except that is, for changes in Papyrus itself, since we went from 4.8 to 6.5 (two major versions and several minors). But it does make sense to wait until we have incorporated some more substantial features and bug-fixes like the one described in Issue #450. So I'm ok with not releasing 3.0.0 for now.

And this reminds me: how should I prioritize the next tasks? #450 is a big one, but I saw there are a few new ones reported. Also, I want to contribute the patches to the different open source projects, and at least the latest fixes for the "architecture domain merger" in Papyrus, which I think it's pretty important. What do you think?

J17359 commented 6 months ago

We likely will want to make that 3.0.0 release, just not yet. We know the change is big and certainly warrants a major version bump like 3.0.0 but we need to test it internally and might wish to add a few more features/bug fixes between now and when we need to lock to that 3.0.0 version (at which time we'd ask for a formal release).

But as I said before, if you and your team want to make that release now, you certainly can as you own this repo/product. NGC just isn't going to pay for it since we don't need it at the moment.

emammoser commented 6 months ago

Yeah, Michael hit the nail on the head. NGC most likely won't need this 3.0.0 release until Q3/Q4 if I had to guess and almost none of our developers will be using this new version until then.

eposse commented 6 months ago

I've been thinking a bit more about this, and I want to make sure we are really on the same page.

From the points you made above:

  1. You will most likely not need the 3.0 release that upgrades Papyrus to 6.5 and removes Log4j 1 dependencies until Q3 or Q4 of this year.
  2. You may want to add features and bug fixes between now and when the 3.0 is ready
  3. Therefore we should bump the version in the papyrus branch to 3.0.0 and create the 2.6.1 maintenance branch.
  4. But you would be ok with use releasing 3.0 before if we wanted to.

Just to clarify, I do not have a strong opinion about releasing 3.0 now, and as I mentioned before, it does make sense to wait until we have added more user-visible features and bug-fixes.

Having said that, I want to clarify a few things, before bumping the version on the main papyrus branch.

So I have a few questions:

  1. From the above, I infer that you do not intend to have any releases from us between the current 2.5 and 3.0 sometime in Q3/Q4, even if we integrate new features and bug-fixes between now and then. Is this correct?
  2. Assuming the answer to the previous question is yes, all new features and bug-fixes (including e.g. #450) will be only on version 3.0.0, correct?
  3. Once you have tested the latest build from PR #480 (which addresses both Issue #471 and #476), can we merge it into the main papyrus branch? (I think this only makes sense if the version in the main papyrus branch has been bumped to 3.0.0)
  4. Do you currently have any timeline and resources for testing PR #480?
  5. How do we prioritize the currently open issues?
  6. Can I report in the time-sheets, the time spent on reporting the Papyrus bug on the "architecture domain merger" and provide its fix? (It takes some time to create the proper report, minimal reproducer and patch)
  7. Can I report in the time-sheets, the time spent on contributing the multiple patches to other projects (Apache, ECP, Eclipse, GMF, MWE, Nebula, OCL, Papyrus, Papyrus Compare, Xpand, Xtend, and Xtext)? (Also takes time, including contacting the projects teams).

Thanks

PS: Just to add, in case you missed the comment in #471, a build of #480 is ready for you to test. You can get it here.

emammoser commented 6 months ago

Some answers to your questions.

  1. From the above, I infer that you do not intend to have any releases from us between the current 2.5 and 3.0 sometime in Q3/Q4, even if we integrate new features and bug-fixes between now and then. Is this correct?

Correct. NGC only requires a release when we, NGC, release our product. Since CX is open source, anyone can make releases, but NGC isn't going to pay for them until NGC needs one.

  1. Assuming the answer to the previous question is yes, all new features and bug-fixes (including e.g. https://github.com/ZeligsoftDev/CX4CBDDS/issues/450) will be only on version 3.0.0, correct?

I wouldn't say all new features and bug fixes. It is possible that we find a bug in CX 2.5.0 that we want fixed on the 2.5.0 maintenance stream and the branch that could eventually be 2.6.0 (as well as 3.0.0 if it exists there as well). Additionally, it is possible, but not probably that in the future we want a 2.6.0 release for one reason or another. But in any event, any new bug tickets or large features should definitely go on 3.0.0 and possibly go on 2.5.0 and the branch that could eventually be 2.6.0

  1. Once you have tested the latest build from PR https://github.com/ZeligsoftDev/CX4CBDDS/pull/480 (which addresses both Issue https://github.com/ZeligsoftDev/CX4CBDDS/issues/471 and https://github.com/ZeligsoftDev/CX4CBDDS/issues/476), can we merge it into the main papyrus branch? (I think this only makes sense if the version in the main papyrus branch has been bumped to 3.0.0)

Yes, this large change to remove the old log4j issue should only be applied to the 3.X branch. the 2.5.0 maintenance stream and future 2.X stream/branch should not have all of these changes.

  1. Do you currently have any timeline and resources for testing PR https://github.com/ZeligsoftDev/CX4CBDDS/pull/480?

  2. How do we prioritize the currently open issues?

We are discussing internally these topics. Unfortunately, when the new year begins, we have some open questions that can take some time to get situation.

  1. Can I report in the time-sheets, the time spent on reporting the Papyrus bug on the "architecture domain merger" and provide its fix? (It takes some time to create the proper report, minimal reproducer and patch)

  2. Can I report in the time-sheets, the time spent on contributing the multiple patches to other projects (Apache, ECP, Eclipse, GMF, MWE, Nebula, OCL, Papyrus, Papyrus Compare, Xpand, Xtend, and Xtext)? (Also takes time, including contacting the projects teams).

I am not entirely sure what "architecture domain merger" is. But, I think as a rule, any time you spend related to tickets NGC approves to work, should definitely be reported in the time sheet so that you aren't doing any work unaccounted for. I think if there are any large issues such as "It will take 40 hours to get XXX change applied to Xpand" we should have another conversation, but since you have the patches done, I would imagine the work to get it applied to not be extraneous

eposse commented 6 months ago

I have a couple of followup questions to clarify a couple of things.

Some answers to your questions.

  1. From the above, I infer that you do not intend to have any releases from us between the current 2.5 and 3.0 sometime in Q3/Q4, even if we integrate new features and bug-fixes between now and then. Is this correct?

Correct. NGC only requires a release when we, NGC, release our product. Since CX is open source, anyone can make releases, but NGC isn't going to pay for them until NGC needs one.

  1. Assuming the answer to the previous question is yes, all new features and bug-fixes (including e.g. Add support for ExF to CX modeling tool #450) will be only on version 3.0.0, correct?

I wouldn't say all new features and bug fixes. It is possible that we find a bug in CX 2.5.0 that we want fixed on the 2.5.0 maintenance stream and the branch that could eventually be 2.6.0 (as well as 3.0.0 if it exists there as well). Additionally, it is possible, but not probably that in the future we want a 2.6.0 release for one reason or another. But in any event, any new bug tickets or large features should definitely go on 3.0.0 and possibly go on 2.5.0 and the branch that could eventually be 2.6.0

Ok, so you think that you may want a 2.6.0 release, right? If so, then we shouldn't bump the version in the papyrus branch to 3.0.0, as agreed a few messages before. If this is the case, then we should leave the papyrus branch alone, with the current 2.6.0 version, and only bump the version to 3.0.0 on the branch that upgraded Papyrus and added all the patches. Is this correct?

Alternatively, we could create a separate 2.6.0 branch (different from the 2.6.x-maintenance branch) and bump the version to 3.0.0 in the papyrus branch and merge all the changes. But I imagine you would prefer the first option as you haven't tested the new build yet, right?

By the way, since the new branch upgraded Papyrus and you won't be using it for now, it is very possible that some of the bugs found are applicable only to the old version of Papyrus. Furthermore, the fix of some bugs may not be the same on the two versions, which means that for bug or feature will take more time to resolve or implement, as some APIs and code have changed, and debugging and testing will have to be done on both versions 3.0 and 2.5 (or 2.6?). It's the cost of keeping older versions of software while upgrading to newer versions in development.

  1. Once you have tested the latest build from PR Papyrus Issue #471 #480 (which addresses both Issue Upgrade Papyrus version #471 and Remove Log4j1 from Papyrus. Replace with Log4j2 #476), can we merge it into the main papyrus branch? (I think this only makes sense if the version in the main papyrus branch has been bumped to 3.0.0)

Yes, this large change to remove the old log4j issue should only be applied to the 3.X branch. the 2.5.0 maintenance stream and future 2.X stream/branch should not have all of these changes.

Right, but as mentioned above, if you do want to potentially release a 2.6.0 version, then we either bump the version to 3.0.0 only on the branch with the patches and not on the papyrus branch, or we create a separate 2.6.0 branch and bump the version to 3.0.0 on the papyrus branch and merge all the changes.

  1. Do you currently have any timeline and resources for testing PR Papyrus Issue #471 #480?

  2. How do we prioritize the currently open issues?

We are discussing internally these topics. Unfortunately, when the new year begins, we have some open questions that can take some time to get situation.

Ok, I'll work on submitting the patches to the different projects in the meantime, and preparing the bug report to Papyrus.

  1. Can I report in the time-sheets, the time spent on reporting the Papyrus bug on the "architecture domain merger" and provide its fix? (It takes some time to create the proper report, minimal reproducer and patch)

  2. Can I report in the time-sheets, the time spent on contributing the multiple patches to other projects (Apache, ECP, Eclipse, GMF, MWE, Nebula, OCL, Papyrus, Papyrus Compare, Xpand, Xtend, and Xtext)? (Also takes time, including contacting the projects teams).

I am not entirely sure what "architecture domain merger" is. But, I think as a rule, any time you spend related to tickets NGC approves to work, should definitely be reported in the time sheet so that you aren't doing any work unaccounted for. I think if there are any large issues such as "It will take 40 hours to get XXX change applied to Xpand" we should have another conversation, but since you have the patches done, I would imagine the work to get it applied to not be extraneous.

The issue with the "architecture domain merger" is the main bug I encountered when testing the 3.0.0 version after upgrading to Papyrus 6.5, and which I mentioned in Issue #471, and took me a couple of weeks to debug and fix. It was a big bug that affected many things in Papyrus, including the following: unable to open deployment plan editors, unable to open diagrams, unable to create components, both monolithic and assemblies, empty "New Diagram" pop-up menu, and others. The cause of this error was the upgrade to Papyrus 6.5 as a central aspect of Papyrus, called the "architecture domains manager" went through a major revamp that broke our "architecture context model". Without getting into details, an "architecture context" is a language like UML, SySML, or AXCIOMA, an "architecture domain" is something like "Software Engineering" or "Systems Engineering", which defines one or more architecture contexts. Each context defines a set of "representation kinds" like types of diagrams, and a set of "viewpoints" which have an associated set of representation kinds. The idea is that each model is associated to a context (e.g. AXCIOMA) and you can have one or more viewpoints (in AXCIOMA we have only one), but the viewpoint determines what elements are available in the UI, for example, what entries are shown in the "New Diagram" menu, or in the palette, what kind of diagrams you can create or view, etc. The "architecture domain manager" handles all this, and it has a component called the "merger" which collects and merges all relevant, active domains, contexts, viewpoints and representation kinds. This changed considerably and had a very subtle and difficult to detect bug. This is the one that took me a while to find, and the one I am working to report to Papyrus with the fix. This has taken a bit of time because it means creating a minimalistic Papyrus-only sample model to reproduce the bug, get the appropriate developer setup for plain Papyrus, reapply the fix on the latest branch (6.6 or 6.7) and test it there.

Similarly, submitting the patches to the other projects does take a bit of time, because it requires tracking and contacting their developers, figuring out how to submit the patches (each project has its own way), determine if it can be submitted on an older branch (as is the case for most patches), etc. And as I mentioned, there are quite a few of these: Apache HTTP components, Apache XMLgraphics, ECF, Eclipse, GMF, MWE2, Nebula, OCL, Papyrus, Papyrus Compare, Xpand, Xtend/Xtend2, and Xtext.

I'm working on all of these. I assume that is a reasonable way to proceed while you decide about the priorities for the next tasks, and I want to make sure I can report these in the timesheets. Is that right?

emammoser commented 6 months ago

Ok, so you think that you may want a 2.6.0 release, right? If so, then we shouldn't bump the version in the papyrus branch to 3.0.0, as agreed a few messages before. If this is the case, then we should leave the papyrus branch alone, with the current 2.6.0 version, and only bump the version to 3.0.0 on the branch that upgraded Papyrus and added all the patches. Is this correct?

Alternatively, we could create a separate 2.6.0 branch (different from the 2.6.x-maintenance branch) and bump the version to 3.0.0 in the papyrus branch and merge all the changes. But I imagine you would prefer the first option as you haven't tested the new build yet, right?

I think we are on the same page. We should have 2 branches, one for the possible future 2.6.0 release that does not have any of the new papyrus updates or log4j changes, and one for the future 3.0.0 release that does have all of those changes.

By the way, since the new branch upgraded Papyrus and you won't be using it for now, it is very possible that some of the bugs found are applicable only to the old version of Papyrus. Furthermore, the fix of some bugs may not be the same on the two versions, which means that for bug or feature will take more time to resolve or implement, as some APIs and code have changed, and debugging and testing will have to be done on both versions 3.0 and 2.5 (or 2.6?). It's the cost of keeping older versions of software while upgrading to newer versions in development.

Completely understand.

Ok, I'll work on submitting the patches to the different projects in the meantime, and preparing the bug report to Papyrus. Great Thank you

... I'm working on all of these. I assume that is a reasonable way to proceed while you decide about the priorities for the next tasks, and I want to make sure I can report these in the timesheets. Is that right?

Yes, these are all related to the original task of upgrading CX and Papyrus to no longer have log4j and to work to contribute back these fixes where accepted and possible. The only place NGC would possibly draw the line in the future is if working with owners of the other projects start to take increasing amounts of time and effort due to issues with working with the other projects.

eposse commented 6 months ago

Ok, so you think that you may want a 2.6.0 release, right? If so, then we shouldn't bump the version in the papyrus branch to 3.0.0, as agreed a few messages before. If this is the case, then we should leave the papyrus branch alone, with the current 2.6.0 version, and only bump the version to 3.0.0 on the branch that upgraded Papyrus and added all the patches. Is this correct?

Alternatively, we could create a separate 2.6.0 branch (different from the 2.6.x-maintenance branch) and bump the version to 3.0.0 in the papyrus branch and merge all the changes. But I imagine you would prefer the first option as you haven't tested the new build yet, right?

I think we are on the same page. We should have 2 branches, one for the possible future 2.6.0 release that does not have any of the new papyrus updates or log4j changes, and one for the future 3.0.0 release that does have all of those changes.

Ok, will do. If it's all the same to you, I'll keep the papyrus branch with 2.6.0, and create a branch called "papyruscx-3" or something like that. There is a question about how do we handle PR #480. I can think of the following options:

  1. Leave it as is, and once you have tested it, merge it into the new "papyruscx-3" branch.
  2. Merge it into the new "papyruscx-3" branch now, (which we won't release yet) and create new Issues and PRs for any issue that arises when you test it.

Personally I would prefer option 2, as I don't like to leave an Issue and a PR indefinitely open.

Which option would you prefer? And do you have any idea if you will be testing that anytime soon?

... I'm working on all of these. I assume that is a reasonable way to proceed while you decide about the priorities for the next tasks, and I want to make sure I can report these in the timesheets. Is that right?

Yes, these are all related to the original task of upgrading CX and Papyrus to no longer have log4j and to work to contribute back these fixes where accepted and possible. The only place NGC would possibly draw the line in the future is if working with owners of the other projects start to take increasing amounts of time and effort due to issues with working with the other projects.

Ok, understood. Thanks.

emammoser commented 6 months ago

Personally, I too would like option 2. I would like to get @J17359 opinion on the matter as well before we go ahead.

J17359 commented 6 months ago

Agree. Option 2 sounds best - merge now and write new tickets in future if needed

eposse commented 6 months ago

I created a new branch called 'papyruxcx-3' to be the main branch for the 3.x version, but I'm not entirely happy with that name, or the name of the main branch for the 2.x versions, which is 'papyrus', not very descriptive. So considering that we have many branches called "streams/v2.x.y-maintenance", I thought that it might be a good idea to have a more uniform convention and rename the new branch to "streams/v3" and the 'papyrus' branch to "streams/v2". I've discussed it with Paul and Young-Soo and they agree. Would it be OK with you if we rename these branches (specially the existing 'papyrus' branch)?

J17359 commented 6 months ago

Fine with me. We'll just have to make a few changes internally to find the new branches since we've hardcoded papyrus in some of our build scripts, but I don't see that being terribly difficult.

eposse commented 6 months ago

Ok, would you like me to wait until you make those changes, or can I rename them now?

J17359 commented 6 months ago

Rename them whenever you want. We'll notice the breakage when our CICD jobs start failing 😄 so it'll force us to fix them.

eposse commented 6 months ago

Thanks. Will do.

eposse commented 6 months ago

Ok, I've finished all the tasks for this. We now have two branches:

streams/v2: the old 'papyrus' branch; this is where work towards a 2.6.0 version will be. streams/v3: the new branch, based on Papyrus 6.5 and with all the patches replacing Log4j 1 with Log4j 2.

Both builds are successful.

If you are ok with it, I'll close this issue.

J17359 commented 6 months ago

Close away!