Merge with one with pde.ui?

[vogella]

@vik-chand, @HannesWell @akurtakov @iloveeclipse WDYT? Can we merge this one into the main PDE repo? Any objections? We have a bigger discussion for platform repos but IMHO for PDE is pretty obvious in my eyes that it is useless to have build separate.

[iloveeclipse]

I know no reason why there are two PDE repos. Probably some historical split coming from IBM times?

If nothing speaks against it, one could try.

I've never merged two git repos. I imagine, if the file trees are different, shouldn't be a big deal. There are few technical questions which arise for this concrete merge. I have no answers...

• Note, the files from "build" repo would change paths in the new one, because the projects in the "build" were located directly under the repo root, in the pde.ui they will be most likely inside new directory.
• So it is not only "simple" merge but also path change, which might have side effects on build scripts / whatever else?
• One probably would need to move one directory up first?
• In both repos the root files are duplicated. So how is that handled? Should the files be removed first from one repo?
• There are identical tags (I-Builds, R-Builds etc) in both repositories. Now, we can't have two identical tags on two different commits. How is that handled?

[mickaelistria]

Note, the files from "build" repo would change paths in the new one, because the projects in the "build" were located directly under the repo root, in the pde.ui they will be most likely inside new directory. [...] One probably would need to move one directory up first?

+1, this could and should be done ASAP here, before the merge, to facilitate next steps.

So it is not only "simple" merge but also path change, which might have side effects on build scripts / whatever else?

As long as things build with Maven/Tycho, it should be OK. Those particular repo content are AFAIK not referenced in build scripts directly.

In both repos the root files are duplicated. So how is that handled? Should the files be removed first from one repo?

I suggest to delete the root files from this repo before doing the merge. Maybe that can be done in a branch if there is a strong need to not break anything at all. But I think if all required changes (eg to aggregator, to this repo...) are prepared in advance, it should be a matter of ~20 minutes to complete the merge; so maybe not worth working on a branch.

There are identical tags (I-Builds, R-Builds etc) in both repositories. Now, we can't have two identical tags on two different commits. How is that handled?

That's a very good question and I don't have any idea! Maybe we can just not merge the tags anyway and only leave them available in the former repository?

[vogella]

Annotated tags are unique even if the same label is used.

I merged pde.core and pde.ui a while ago at this time pde.build was considered as "dead" and Dani did not want to merge it too. But as it is still used, I guess we should merge now.

[HannesWell]

I merged pde.core and pde.ui a while ago at this time pde.build was considered as "dead" and Dani did not want to merge it too. But as it is still used, I guess we should merge now.

pde.build is responsible to perform the builds triggered from the UI when one hits export Product, Plug-in or Feature etc, isn't it? I already wounder/thought if it would be possible to replace that by a Maven/Tycho build for which the root pom.xml is generated automatically? This would only require the maven.runtime of m2e and probably the launching facility.

We could then just drop pde.build entirely.

The only thing we would loose are the ANT tasks, but are they public API or just used internally?

[vogella]

@HannesWell it would be great if PDE build could be replaced with Maven. PDE build does not work reliable anymore and replacing it with something working would be great but I assume that may be a big task.

[HannesWell]

@HannesWell it would be great if PDE build could be replaced with Maven. PDE build does not work reliable anymore and replacing it with something working would be great but I assume that may be a big task.

I cannot estimate the effort exactly at the moment, because I'm not yet made myself fully aware of what pde.build is doing exactly besides exporting/building Plug-ins, Features, Products and Update-Sites/P2-Repos/category.xml (did I forget something?)?

But if it's only this I would estimate it to be only a moderate amount of work, because 'only' a corresponding root pom.xml would have to be generated. It could be generated into a temporary working directory and could use absolute paths of the modules to build. The target-platform could also be generated as a list of bundle-paths of the active workspace TP in case it is in a format that Tycho does not support (like Oomph-Targlets).

Besides that it depends on if we are willing to let parts of PDE depend on parts of m2e and if we are willing to remove some parts completely without replacement (I'm thinking about the ANT-tasks but this could also effect some properties of the export).

CC @mickaelistria and @laeubi in case they are interested or want to add comments.

[mickaelistria]

PDE Build has been dead to me for many years. From my POV, there is no need to keep it at all; nor to find a replacement if we want to get rid of it. Is anyone among PDE contributors still using it? Is anyone in need of keeping it alive?

[iloveeclipse]

@mickaelistria : even if no one uses it from PDE contributors, org.eclipse.pde.build.IFetchFactory is a public API that should be there as long as it is not deprecated / announced for deletion etc.

Beside this, in our (customized) IDE we've built some ant related builder tasks around AbstractBuildScriptGenerator & Co to generate custom ant build scripts. Ant is not dead :-)

[vik-chand]

I think PluginExport is used quite commonly which uses BuildScriptGenerator. Also I'm sure many releng tasks may be using pde.build.

[akurtakov]

For Eclipse SDK build releng pde.build is no longer used AFAICT.

[vik-chand]

PDE Build has been dead to me for many years. From my POV, there is no need to keep it at all; nor to find a replacement if we want to get rid of it. Is anyone among PDE contributors still using it? Is anyone in need of keeping it alive?

As many functionalities are still in pde.build as mentioned below by @HannesWell

I cannot estimate the effort exactly at the moment, because I'm not yet made myself fully aware of what pde.build is doing exactly besides exporting/building Plug-ins, Features, Products and Update-Sites/P2-Repos/category.xml (did I forget something?)?

And since there is no known and easy replacement, we should not delete pde.build

Instead I recommend merge pde.build to pde.ui repo as suggested by @vogella

[laeubi]

And since there is no known and easy replacement, we should not delete pde.build

As Tycho can build all the PDE stuff, why not migrate this to maven/tycho instead of ant/pde.build? Given that I think we can simply keep pde-build repo as is so if we one day don't need it in platform anymore, we can simply keep the reporistory and don'T need to split it (as with CVS repository) if it is still usefull for legacy/others...

[vogella]

@vik-chand shall I send a PR for merging build into ui?

[HannesWell]

Christoph's remark is reasonable for me. Merging pde.build into pde.ui could look like some kind of a revival. Maybe we should keep it here and work instead more actively on deprecating/replacing it by Maven/Tycho and let it go through the regular 2-years deprecation/removal process.

@iloveeclipse are the ANT-tasks giving you anything that Maven/Tycho does not offer? Or is it just that you are happy using it and don't want to migrate because you don't need to?

I have to admit that I'm not aware how the jars of ANT-tasks are fetched so this question might not be the smartest one, but does it break existing happy-running ANT builds, if we would remove pde.build now? In Maven existing plug are forever available in Maven-Central, so if this works in the same way for ANT, users could just use old versions if they don't want to migrate their build. Given that pde.build is not maintained this would not make a big difference.

[iloveeclipse]

@HannesWell : few things regarding ant. We are not just Eclipse build, we build ~200 Eclipse plugins as a tiny part of a bigger C/C++ project with various build drivers, and ant is just one of them (triggered by make). The whole thing is over 15 years old with constantly increasing complexity and around 15.000.000 LOC.

1) for CLI build we use ant and don't use tycho because we don't use maven, nowhere. It is simply too slow for us. 2) for IDE we use custom plugin that is based on AbstractBuildScriptGenerator and automatically updates ant build files on changes in the project structure or dependencies. 3) we have additionally custom ant build scripts that run different generators (xtext, xcore, custom emf etc).

So if PDE build will be removed, we would need to fork it, because moving to maven/tycho is not an option.

[vogella]

PMC decides that we should merge the two projects into pde.ui.

[HannesWell]

@HannesWell : few things regarding ant. We are not just Eclipse build, we build ~200 Eclipse plugins as a tiny part of a bigger C/C++ project with various build drivers, and ant is just one of them (triggered by make). The whole thing is over 15 years old with constantly increasing complexity and around 15.000.000 LOC.

So if PDE build will be removed, we would need to fork it, because moving to maven/tycho is not an option.

Understand, thanks for your explanation.

In general I think the export capability for Plug-ins/Features/Update-Sites/Products from the IDE is a tool that's sometimes very convenient so I would not remove it. And since it is currently behind Tycho in terms of supported elements, will probably loose further (because it stands still and Tycho moves on) and has some bugs I think a Tycho-based export would be nice to have. Maybe it would even be possible to create a basic Tycho setup for a given project (if it has a structure close enough to the usual one). But I won't have the time to work on this in the near future.

If the pde.build plug-in is based on Tycho one day and since the ANT-tasks are still needed (as @iloveeclipse expressed) we could consider to extract them to a dedicated Plug-in then (o.e.pde.build.ant maybe?).

[laeubi]

In general I think the export capability for Plug-ins/Features/Update-Sites/Products from the IDE is a tool that's sometimes very convenient

The important part is sometimes here, as it always failed for me for non trivial setup.

I think a Tycho-based export would be nice to have.

One don't need specifically tycho for this but could use P2, as tycho also delegates to P2 as well...

PMC decides that we should merge the two projects into pde.ui.

I still don't think to merge a build tool (pde build) with the ide tool (pde). Especially as the repository name eclipse.pde.ui is already confusing (it does contains apitools, **ui**, ua, ds subfolders), so is eclipse.pde.ui/ui the real UI? Not really as it contains several code and ui bundles especially org.eclipse.pde.ui so is eclipse.pde.ui/ui/org.eclipse.pde.ui maybe more UI than other UI ;-)

[HannesWell]

In general I think the export capability for Plug-ins/Features/Update-Sites/Products from the IDE is a tool that's sometimes very convenient

The important part is sometimes here, as it always failed for me for non trivial setup.

Although I also encountered some flaws it wasn't that bad for me, so I have some hope. :)

I think a Tycho-based export would be nice to have.

One don't need specifically tycho for this but could use P2, as tycho also delegates to P2 as well...

Indeed. I just had that idea because I'm more used to use Maven+Tycho and it seemed relatively simple to "just" have to generate a corresponding root-pom (and hopefully no need for project-specific poms). But using P2 directly is definitely an option to consider and would not require Maven/m2e, which is nice because it avoids the up-link in the inter-project dependency tree.

PMC decides that we should merge the two projects into pde.ui.

I still don't think to merge a build tool (pde build) with the ide tool (pde). Especially as the repository name eclipse.pde.ui is already confusing (it does contains apitools, **ui**, ua, ds subfolders), so is eclipse.pde.ui/ui the real UI? Not really as it contains several code and ui bundles especially org.eclipse.pde.ui so is eclipse.pde.ui/ui/org.eclipse.pde.ui maybe more UI than other UI ;-)

The naming issue of pde.ui is addressed as well: https://github.com/eclipse-pde/eclipse.pde.ui/issues/35

Lars is diligently merging and renaming repos for the better :)

[vogella]

I'm the biggest fan of using Tycho for export. Once we have this we could deprecated pde build functionality.

[laeubi]

I'm the biggest fan of using Tycho for export.

As noted above it does not makes much sense to use Tycho, as it is a maven-plugin and you most probably need to better control it (e.g. output folders, compress, select what to export), but could simply call P2 (what is already part of PDE), e.g. Tycho uses the MirrorApplication to build features/products.

[vogella]

I'm the biggest fan of using Tycho for export. Once we have this we could deprecated pde build functionality.

Let me rephase: I'm the biggest fan for using something which is still used and maintained by the Eclipse platform build. In the past my clients experienced issues with PDE build and the reported problems were not addressed, as platform does not rely on the functionality anymore.

[vogella]

Process of merging:

Add new remote to the repo to be merged (source, in this case the pde.build) to the destination repo (pde.ui) Fetch from new remote Merge source/master with origin/master branch Resolve conflicts Adjust pom files to build also the new elements

[laeubi]

Merge source/master with origin/master branch

If you use

git remote add pde-build ...
git fetch pde-build --tags
git merge --allow-unrelated-histories --no-commit --strategy-option ours  pde-build/master
git remote remove pde-build

you should get anything in including any history ... and as they both use different subfolders you could tell git to always use the origin instead of the source then you only need to add the new modules to the pom without any conflicts.

[vogella]

@laeubi that is what I'm doing only via the EGit UI.

[vogella]

Merge done, the rest of the work happens in other repos (aggregator and common) to adjust the build.

[HannesWell]

Thanks Lars for your work.

I think this repository can be archived now? https://docs.github.com/en/github-ae@latest/repositories/archiving-a-github-repository/archiving-repositories

We could add one last commit that adds a README, which states that the content of this repository has been merged into eclipse.pde.

[vogella]

https://github.com/eclipse-pde/eclipse.pde.build/issues/10 is the archive. I push an update to the readme.